Google no longer searches forum

[FranceBB]

And... we're back, baby.
Google slowly started indexing pages again.
Not only I could see the spider, but if you click on StainlessS first link (or indeed search avisynth site:forum.doom9.org yourself on Google) you'll see that we now have 1 topic:
"The power of Avisynth: restoring old 8mm films."

I wonder how long it will take for Google to index back all the thousands of topics in the forum and if we can do something to speed things up...
Perhaps setting up an account with the Google Webmaster Tool would help? https://search.google.com/search-console/about

[StainlessS]

We're gettin' a few hits on

avisynth+  site:forum.doom9.org

https://www.google.co.uk/search?q=av...orum.doom9.org
but still currently only one hit on 'avisynth' (non +) search alone.

Most, (but not all) 'avisynth+' hits have 'avisynth+' in the thread title. (not searching body text ???)

[filler56789]

Originally Posted by StainlessS

We're gettin' a few hits on

avisynth+  site:forum.doom9.org

https://www.google.co.uk/search?q=av...orum.doom9.org
but still currently only one hit on 'avisynth' (non +) search alone.

Most, (but not all) 'avisynth+' hits have 'avisynth+' in the thread title. (not searching body text ???)

Since 2013 at least, Google's databases are programmed to become more and more "selective"
(read: self-corrupted) :-|

[StainlessS]

Maybe its now due to AI being more selective, and the thread
"The power of Avisynth: restoring old 8mm films.", is the only one that
it's AI deems worthy [ you watchin' fred ]

[FranceBB]

Nah, I think it's just gonna be a matter of time.

Searching with link shows that we're already getting lots of pages back, but it looks like it's gonna take a while before the whole forum is indexed back.

I'm afraid it's not gonna be all laugh and smiles, though, 'cause although pages are back now that the ban is gone, Google is really not ranking Doom9 highly, so much so that even searching Avisynth doesn't show Doom9 in the first page and we're probably somewhere further down in Google, like in some remote high page number.
I really hope I'm wrong and I know that the admins were trying their best, but it looks like the effects of this self inflicted penalty (by mistake) will last for quite some time...

[StainlessS]

Today at time of posting, showing (for "site:forum.doom9.org/")

About 29,200 results (0.14 seconds)

Whats it gonna say tomorrow. [EDIT: Ie number of results]

[filler56789]

Originally Posted by StainlessS

Today at time of posting, showing (for "site:forum.doom9.org/")

About 29,200 results (0.14 seconds)

Whats it gonna say tomorrow.

Doesn't matter. Because "About NNNN results (0.nn seconds)" depends on which part of the planet you're in, to begin with :-/

Besides, by setting 20 results per page, and clicking on the link to "page 10", you will be already greeted with the horrendous fishertroll thing🤮

Remember, the false-god was never intended to be a replacement for the original Altavista,
which actually indexed AND showed everything it could index and show...
this is the reason why Gluegow always says
"in order to show you the most IRrelevant results,
blah-blah-blah".

https://forum.videohelp.com/threads/...D-x265-sucks-P

[StainlessS]

OK, OK, lets see what it says about number of results BEFORE paring them down. [ie the About 29,200 results thingy]

[VoodooFX]

Originally Posted by filler56789

Remember, the false-god was never intended to be a replacement for the original Altavista,
which actually indexed AND showed everything it could index and show...

But everyone switched from Altavista to Google because it suuucked.
Anyway, my Member Berries goes to Astalavista.

[FranceBB]

Originally Posted by StainlessS

Whats it gonna say tomorrow.

It's far more than I predicted, with a sharp increase from 29,200 to 37,800 results now!

Over to you, StainlessS, for tomorrow's update xD

[Balling]

All works now! https://www.google.com/search?q=http...=1210&dpr=1.71

[FranceBB]

Originally Posted by FranceBB

a sharp increase from 29,200 to 37,800 results now!

And we're now at 41,400 results.
Let's keep going, guys!

[FranceBB]

OH NO!
We're losing indexing again!

http://forum.doom9.net/robots.txt

is blocking User-agent: Googlebot again!
In fact, we're falling AGAIN!

We just went from 41,400 results to 32,200 results!!
Who put the Googlebot block in robots.txt again and most importantly why?!

[nevcairiel]

The robots file does not seem to exist.

[FranceBB]

Yep, they have now deleted it.
Interesting.
I think it's the right choice, though.
Getting rid of it entirely, I mean.
If it's not there, you can't get it wrong hahahahahaha

Anyway, hopefully now all spiders will be able to find us once and for all.

[FranceBB]

Well, deleting the robots.txt seems to be working.
We're bouncing back again:

34,400 results

[StainlessS]

It certainly allowed me to find thread with "guitar" clip back in 2009.

[Balling]

Why does it have both http and https results? You can apparently do 301 server redirect: https://support.google.com/webmaster...sgid=107978668

[filler56789]

Originally Posted by Balling

Why does it have both http and https results? You can apparently do 301 server redirect: https://support.google.com/webmaster...sgid=107978668

Maybe the admins should have disabled http PLUS the "printable archive" ages ago...
Anyway, it's probable that Google is simply "recycling"
(and recycling very-poorly, to say the least...)
its jurassic databases.
I serious doubt their ""spider"" is actually re-indexing Doom9 from ground-zero.
*woman shrugging emoticon*

Originally Posted by FranceBB

Well, deleting the robots.txt seems to be working.
We're bouncing back again:

34,400 results

IMNSHO, the "bigger problem" of Doom9's forum is not it not being indexed well by Gluegow,
the "bigger problem" is: there exists no read-only FULL backup of all its threads and posts on the Web.
Nope, the Wayback Machine is not an option, that site sucks more and more each day, and for various reasons it stops archiving images, ZIP files (attachments), etc, which makes it not-so-useful.
This problem was briefly discussed in the deceased doom10 of Dark Shikari, but nothing practical happened since then and so far :-|

[FranceBB]

Originally Posted by filler56789

IMNSHO, the "bigger problem" of Doom9's forum is that [...] there exists no read-only FULL backup of all its threads and posts on the Web.

Yep, I totally agree.
Although I hope Doom9 isn't gonna go anywhere anytime soon, I do think that we should really have a safe backup of all its contents if, one day, it was to go down for good.
There are too many valuable information and people spent far too many hours of their lives for years and years on it. I mean, it would be a shame if one day it just... disappeared into nothingness.