Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > Forum / Site Suggestions & Help

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th May 2023, 14:37   #1  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,981
Google no longer searches forum

For some reason, Google no longer returns results of forum search on eg
Code:
avisynth site:forum.doom9.org
https://www.google.co.uk/search?q=av...orum.doom9.org

or
Code:
avisynth  site:https://forum.doom9.org
https://www.google.co.uk/search?q=av...orum.doom9.org

Is it problem with Google, or with forum.



EDIT:
This works,
Code:
ukraine site:bbc.co.uk
https://www.google.co.uk/search?q=uk...te%3Abbc.co.uk

EDIT: Perhaps forum has added some kind of "robots" thingy, preventing scanning of site.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 19th May 2023 at 14:50.
StainlessS is offline   Reply With Quote
Old 19th May 2023, 15:31   #2  |  Link
Thundik81
Registered User
 
Join Date: Jul 2004
Posts: 23
https://forum.doom9.org/robots.txt
https://developers.google.com/search...ots/robots_txt
Thundik81 is online now   Reply With Quote
Old 19th May 2023, 15:49   #3  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,981
Thanks for that, Thundik81. [EDIT: I forgot how that robots thingy works, but remembered was 'robots' related]
It would then seem that GoogleBot has been denied access to search the D9, one wonders why that is.
Presumably then, you will never get any Doom9 forum hits when doing a google search.

Planned obsolescence ???
(If you are not already a Doom9 user, you will never [well hardly ever] find out that it exists)

EDIT: "Yandex Spider" and "BingBot" are currently scanning forum. (See Quick Links/Who's online).
Why the hatin' on Google, but not Yandex nor Bing ?
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 19th May 2023 at 21:21.
StainlessS is offline   Reply With Quote
Old 19th May 2023, 17:37   #4  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 121
I was also attempting to use the Google Search Engine to search the forum with short keyword and had the same issue. I don't think this robots.txt change is good.
cubicibo is offline   Reply With Quote
Old 19th May 2023, 19:48   #5  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,982
I always use google to search the forum.
Not only Google used to provide far better search results than the internal search inside the forum, but it was also how the overall majority of users came to Doom9 in the first place.
They searched something on the web related to encoding, Google provided topics in the forum as answers, they opened the related topics, found the replies and said "you know what? I'm gonna stick around, those people know what they're talking about".

Blocking Google from scanning the forum is a big mistake and will inevitably lead to a sharp fall in viewers/guests and an inevitable decline in new users...


p.s even the .net is blocking Google:

Code:
avisynth  site:forum.doom9.net
returns nothing

Last edited by FranceBB; 21st May 2023 at 10:11.
FranceBB is offline   Reply With Quote
Old 20th May 2023, 01:33   #6  |  Link
Balling
Registered User
 
Join Date: Feb 2020
Posts: 544
Quote:
Originally Posted by StainlessS View Post
Thanks for that, Thundik81. [EDIT: I forgot how that robots thingy works, but remembered was 'robots' related]
It would then seem that GoogleBot has been denied access to search the D9, one wonders why that is.
http://forum.doom9.net/robots.txt
Balling is offline   Reply With Quote
Old 20th May 2023, 03:52   #7  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,981
Yeah I got that Balling. [Thundik81 already posted where to find it]
Quote:
one wonders why that is.
Meaning, Why was Google denied access by D9 glitterati.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 20th May 2023, 09:13   #8  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,348
When I do a google search against "avisynth site:forum.doom9.net" I see the info that results were removed because of some DMCA claim,... which points to https://www.lumendatabase.org/notices/10021144
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 20th May 2023, 09:36   #9  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,365
Quote:
Originally Posted by Selur View Post
When I do a google search against "avisynth site:forum.doom9.net" I see the info that results were removed because of some DMCA claim,... which points to https://www.lumendatabase.org/notices/10021144
Thanks for the useful info👍, Selur.
Now we understand why Yandex was not blocked by the forum admin(s):
Gluegow is a slave to the c0pyrite wh0r&$, but Yandex is not.😇
__________________
«Your software patents have expired.»
filler56789 is offline   Reply With Quote
Old 20th May 2023, 09:46   #10  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,365
Quote:
Originally Posted by Selur View Post
When I do a google search against "avisynth site:forum.doom9.net" I see the info that results were removed because of some DMCA claim,... which points to https://www.lumendatabase.org/notices/10021144
What browser were you using?
With Opera version 93.etc, both "avisynth site:forum.doom9.net" and "avisynth site:forum.doom9.org" return that mega-stupid mega-ugly mega-annoying fishertroll animated GIF🤮

https://forum.videohelp.com/threads/...gned-by-DD00TS
__________________
«Your software patents have expired.»
filler56789 is offline   Reply With Quote
Old 20th May 2023, 10:17   #11  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,363
Quote:
Originally Posted by Selur View Post
When I do a google search against "avisynth site:forum.doom9.net" I see the info that results were removed because of some DMCA claim,... which points to https://www.lumendatabase.org/notices/10021144
That claim is 9 years old. Either indexing was broken for all this time and you never noticed, or its not related.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 20th May 2023, 12:17   #12  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,981
I last searched @ site:forum.doom9.org [.org only] about December, worked back then. [EDIT: I was inactive about Jan->March]

Quote:
That claim is 9 years old. Either indexing was broken for all this time and you never noticed, or its not related.
Astute observation.

EDIT:
Quote:
Originally Posted by filler56789 View Post
With Opera version 93.etc, both "avisynth site:forum.doom9.net" and "avisynth site:forum.doom9.org" return that mega-stupid mega-ugly mega-annoying fishertroll animated GIF
I'm using FireFox, and for the .net search, I could not see the DMCA message as was off the bottom of the page, required scrolling to see it.
(W10 settings/System/Display/Scale & Layout=150%, ALSO, I got FireFox at Zoom 150%, 4K monitor)
EDIT: And I love my SUPER MASSIVE <requires about:config editing> scroll bars, so easy to hit [clickme, twice].
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 20th May 2023 at 12:45.
StainlessS is offline   Reply With Quote
Old 20th May 2023, 12:44   #13  |  Link
Thundik81
Registered User
 
Join Date: Jul 2004
Posts: 23
Quote:
Originally Posted by StainlessS View Post
I last searched @ site:forum.doom9.org [.org only] about December, worked back then. [EDIT: I was inactive about Jan->March]
Wayback Machine is your friend:
http://web.archive.org/web/*/https:/...org/robots.txt
>
http://web.archive.org/web/202212240...org/robots.txt
then Santa Claus came to Doom9
http://web.archive.org/web/202212280...org/robots.txt
Thundik81 is online now   Reply With Quote
Old 20th May 2023, 12:51   #14  |  Link
VoodooFX
Banana User
 
VoodooFX's Avatar
 
Join Date: Sep 2008
Posts: 997
As I understand doom9's robot.txt doesn't disallow Google searching the forum threads.

Strange that it doesn't show result from doom9, DMCA complains removes one link, not a whole site. Last DMCA I see was in 2021.

duckduckgo works, if I remember correctly they actually are using data from Google: https://duckduckgo.com/?q=avisynth+s...rg&t=h_&ia=web
VoodooFX is offline   Reply With Quote
Old 20th May 2023, 12:52   #15  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,981
Thanx again Thundik81,
(And I just found out that my ISP is no longer blocking Wayback Machine.)
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 21st May 2023, 00:22   #16  |  Link
Brazil2
Registered User
 
Join Date: Jul 2008
Posts: 542
Quote:
Originally Posted by VoodooFX View Post
duckduckgo works, if I remember correctly they actually are using data from Google
AFAIK data is from Microsoft Bing:
https://spreadprivacy.com/more-priva...-transparency/
Quote:
...our use of Bing as a source for our private search results
Using 'netstat -ano' or TcpView you can also see that Duckduckgo connects to Microsoft IP addresses when you perform a search.
Brazil2 is offline   Reply With Quote
Old 21st May 2023, 03:04   #17  |  Link
Swede
Deputy
 
Swede's Avatar
 
Join Date: Jan 2002
Location: Sthlm, Sweden
Posts: 1,456
I don't know why google have banned us but I'm quite sure that disallowing "User-agent: Googlebot/Nutch-1.7" is not the problem (you can google it ). Just to be sure I've removed that restriction and will monitor the abuse of Nutch that we were hit with at that time as well as looking into why google doesn't like us anymore.
Swede is offline   Reply With Quote
Old 21st May 2023, 17:42   #18  |  Link
gonca
Registered User
 
Join Date: Jul 2012
Posts: 1,226
Could it be a search algorithm update?
They had one in the second half of March
gonca is offline   Reply With Quote
Old 21st May 2023, 22:48   #19  |  Link
Balling
Registered User
 
Join Date: Feb 2020
Posts: 544
Quote:
Originally Posted by Swede View Post
I don't know why google have banned us but I'm quite sure that disallowing "User-agent: Googlebot/Nutch-1.7" is not the problem (you can google it ). Just to be sure I've removed that restriction and will monitor the abuse of Nutch that we were hit with at that time as well as looking into why google doesn't like us anymore.
I noticed that every time I searched previously it was actually the print pages not forum pages and only from .net mirror, that has no valid https cert, because the cert has no .net alternate name.

Last edited by Balling; 21st May 2023 at 22:50.
Balling is offline   Reply With Quote
Old 21st May 2023, 23:44   #20  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 1,365
Quote:
Originally Posted by Balling View Post
I noticed that every time I searched previously it was actually the print pages not forum pages and only from .net mirror, that has no valid https cert, because the cert has no .net alternate name.
MOSTLY from the "printer-friendly archive", yes, but not always.
And in the better old days, G00gle also indexed the "semi-official" Doom9 mirrors by Koepi and Nic.
__________________
«Your software patents have expired.»

Last edited by filler56789; 21st May 2023 at 23:52. Reason: clarity
filler56789 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:26.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.