New hardware

[Swede]

Thanks to the donors I didn't have to put up all the money for a new server myself. :thanks:

We're now on 20 cores instead of 4 so you should notice a quicker response. I'll be monitoring the load closely for some time to make sure we can cope with the AI-bot-storms that roams-the-earth.

/Swede

Some stats from this month:
http://leffster.se/forum0526.png

[Z2697]

Do we have backup(s)?

[Selur]

Nice! Thanks for the info!

Do we have backup(s)?
Real man, don't need backups,.... but real man cry sometimes. ;)

Cu Selur

[Columbo]

I would be shocked and amazed if Swede did not do backups.

[Swede]

I've been around for to long to not have backups...
The schedule is that every week the database is archived to a NAS, RAID-5, and there are 2 months worth of backups saved. Since that NAS is physically at my place it gets synced to another NAS, RAID-1, somewhere else.

So, the only thing not backed up is myself...

/Swede

[microchip8]

20 cores? Can't be AMD as they don't have 20 core CPUs. Must be either Intel or ARM? I'm guessing Intel, yes?

[Swede]

20x 12th Gen Intel(R) Core(TM) i7-12700T (and yes, for all you die-hards out there, it's 20 threads not cores, but hey, who's counting?) :o
And at least they're all ours, no shared HW at my place!

[hajj_3]

which model was the old cpu?

[Swede]

Intel Core i5-7400T

[microchip8]

Intel Core i5-7400T

Ha! I have the same one in my NAS, but without the T. It's a Kaby Lake generation (improved Skylake)

[Z2697]

I have the same one (w/o T) in my old laptop :)
The laptop is a pile of components now but the CPU should be functional... in theory.

[rwill]

New user record today, 86k. The spiders scale with the resources available...

[huhn]

where did you get the ram from? so i'm the only one that has gone AMD for consumer grade mainboards with ECC(not reg)?

[Z2697]

New user record today, 86k. The spiders scale with the resources available...

We need Anubis :)

[Swede]

Kind of fun sitting here, listening to the servers humming quietly at "Currently Active Users: 71850" :cool:

[Columbo]

Just curious, do you just allow anything to come in and request stuff? No anti-bot/scraper mitigation?

[Swede]

Well, I do IP-ban some networks, mainly Alibaba and the likes, but other than that I just take what the throw at me. The problem with all the anti-something is that there will always be false positives, legit bots and users. And there is the (small) annoyance with all these 'Wait while we decides if you are worthy'-pages.
I also don't like the aproach that some of those anti-scrapers have, like Cloudflare, as they might listen in on your 'https-secure' traffic.

Besides, I guess with all this traffic, ~4-5 mil hits / day, it will cost me some money.

[Z2697]

Perhaps, only anti-bot the "Guest" visitors?

[clsid]

Perhaps adjust cache-control and expire headers based on user-agent or ip-range? So spiders update less often.

[Z2697]

Perhaps adjust cache-control and expire headers based on user-agent or ip-range? So spiders update less often.

"AI" spiders break rules (e.g. robot.txt). I don't think this will help.

[huhn]

Well, I do IP-ban some networks, mainly Alibaba and the likes, but other than that I just take what the throw at me. The problem with all the anti-something is that there will always be false positives, legit bots and users. And there is the (small) annoyance with all these 'Wait while we decides if you are worthy'-pages.
I also don't like the aproach that some of those anti-scrapers have, like Cloudflare, as they might listen in on your 'https-secure' traffic.

Besides, I guess with all this traffic, ~4-5 mil hits / day, it will cost me some money.

where does the cost come from no flat rates?

[Swede]

Gah, sometimes the universe seems to be against you! This blackout had nothing to do with bots or new servers or anything. It was a major breakdown of my providers backbone here in my part of the world.
Hopefully all new IPs and so forth will update shortly.

[-Murali-]

Sorry if this has been answered before, any reason not to move to cloud for hosting? I am not aware of any cloud provider hosting forums though - maybe renting a compute and having it as backup?


Sent from my iPhone using Tapatalk

[Emulgator]

@-Murali-
What if the hoster decides against content, or is forced to take down ? Or changes TOS ? Charges more ? Or just disappears ?
All this has happened before to others, so we are very lucky to have a safe haven with Swede.

[FranceBB]

With the new server Doom9 feels good and responsive again, even with thousands of visitors, so I guess we're gonna be fine :)

As tradition, we want a new picture of the closet now ehehehehehehehehe

[Swede]

Well, there are (as Emulgator says) a few reasons. The first and most obvious is that when I offered to host the forum and www things were different that today. Hosting was expensive, cost would increase with bandwidth, almost impossible to get by all the nooks and crannies of the agreements/disclaimers/rules.

I had the bandwidth and some CPU to spare so doom9 accepted my offer and that was the start of it all. Neither one of us knew the amount of time/work/money it would become/require but I'm still here, after ~25 years and currently pushing ~1,5TB a month using, I don't know which server in line but there has been a 'few', the hardware I am willing to pay for.

I've learned a lot during these years of hosting huge forums, some of you might remember that I had to shut down doom9 during the tsunami in 2004 when I used all my resources to host another forum back when facebook and such wasn't an option to bring people together, and for this knowledge I'm grateful.

I'm not as cunning in the field of audio/video that most of you are but I'm doing my best to give you a platform to express yourselfs.

(Oh, and since the new server looks just like the old one there won't be a new pic this time. :cool:)

/Swede

[hajj_3]

Will the forum software ever be updated to something newer like vbulletin v6 or something else? Not having large images rescale is one of the annoying things about old forum software like vbulletin v3.

[Columbo]

My ISP terms of service prevent me from hosting a site like Doom9. Do you not get any flack from your ISP?

[GeoffreyA]

I'm doing my best to give you a platform to express yourselfs.


It's much appreciated. Today, knowledge can disappear at the click of a finger. I think of Anandtech, which now remains on the Web Archive, the main site not even working any more.

[Swede]

Will the forum software ever be updated to something newer like vbulletin v6 or something else? Not having large images rescale is one of the annoying things about old forum software like vbulletin v3.
We've actually tried vb6 but since it will break all links and there's a tons of other things that, at least I, concider too much of change, it won't happen. (I'm an old guy who doesn't like to much of a change)

My ISP terms of service prevent me from hosting a site like Doom9. Do you not get any flack from your ISP?
So far no, and since we're not breaking any laws here I don't anticipate any problems. Remember, I'm not in the US or any other strict country so no one can even make me give up anything (your emails or such).

All this AI and what-ever made me change the way the archive works. Normaly the archive is a 'Search-friendly' way of .. well.. archiving the forum but since it updates in realtime the load is *heavy*. I've now, since the addition of nginx, changed this to a static archive, updated whenever I feel like it (perhaps once a month) to take care of the forumload. Maybe I should create an .tar with all those pages, just so that there will be something left if I vanish.

[Blue_MiSfit]

You rock, Swede :) Thank you for everything you do!

[Swede]

Thanks for all those kind words.

I've now zipped the current archive (you can find the online version at https://forum.doom9.org/archive/index.php) and the link, that I don't want the bots to find is archive tar gz at the main domain (this one without the forum)

Beware though that the gz-fil is ~1/2 GB and unzipped it's in the vicinity of <3GB using +330.000 files so don't download just for fun.

[FranceBB]

To be fair, I actually expected it to be much bigger.
455 MB for the entirety of Doom9 is a bargain XD

All jokes aside, I've now stored a copy on my RAID6 server called "Doom9 Backup 20260702 archive.tar.gz" in case something ever happens (hopefully it never will). :)

[Z2697]

Looks like I'm no smarter than bots :( :(
I can't find the link :(

[FranceBB]

Looks like I'm no smarter than bots :( :(
I can't find the link :(

I've sent you a PM ;)

[VoodooFX]

Here is that archive 7zip'ed to 199MB: https://www.swisstransfer.com/d/c24d6934-5d46-4cb2-8522-7b29628391fd

[GeoffreyA]

Looks like I'm no smarter than bots :( :(
I can't find the link :(

Same. I couldn't find it :)

[FranceBB]

Same. I couldn't find it :)

Sent you a PM too. ;)

[Blue_MiSfit]

Backed up locally, to backblaze, and to Amazon S3 Glacier Deep Archive :)

[oibaf]

We've actually tried vb6 but since it will break all links and there's a tons of other things that, at least I, concider too much of change, it won't happen. (I'm an old guy who doesn't like to much of a change)

What about the security vulnerabilities of using such an old version?
Maybe consider migrating to phpbb or other forum software?

[FranceBB]

To be fair, we haven't really had any incidents other than one in May 2016 in which the hashes were leaked, but not the actual passwords of course.
Anyway, if you're worried about that, just use a password randomly generated by a password manager (which is a good standard practice anyway).

By the way, just like many people here, I'm also a big fan of homeostasis, meaning that once I get used to things I would rather not see them change.
Besides, old software is amazing, it allows you to login with a simple username and password, maintains the session open indefinitely and allows you to see the forum and post immediately.
No fuss, no two factor authentication where you have to pick up your phone, no annoying CAPTCHA, no "accept cookies" infobar popping up every time, no extremely short sessions with the cookies expiring every few days and requiring you to login again etc etc etc.


I know that many security people are gonna cringe about what I just stated above, but old software came from a moment in time in which there was trust among people. After all, the internet had just started and was literally the evolution of what used to be the Arpanet among universities.

That being said, even if there were vulnerabilities, there's really no point for anyone to ever try to do something bad. We're an open source community working on free software that is made available to thousands of people across the world. We always welcomed everyone from any side of the globe and we don't really care or talk about politics. We're a group of passionate people who dedicate their spare time doing what we love. Attempting something malicious here wouldn't achieve anything as we're no profit. There's no gain for an attacker as we don't have sensitive information stored here either (most of us use nicknames as well), it's not like an ecommerce website that stores credit card info or private people details etc.

[GeoffreyA]

Agree with FranceBB's sentiment. Doom9 is a bit like a living encyclopedia.

[Jamaika]

What about the security vulnerabilities of using such an old version?
Maybe consider migrating to phpbb or other forum software?
What kind of data can be stolen here? User forum password.
As for the additional password for the smartphone, it is annoying and inconvenient for older people.

[Mr_Khyron]

Have tried to use anubis? https://github.com/TecharoHQ/anubis
To deal with the AI bots?