OS2World.com Server Replacement – After 14 Years of Faithful Service!

[Kim]

Hi everyone,

After more than 14 years of rock-solid service, we have finally retired the old os2world.com server and moved everything to a brand-new, modern infrastructure. We thought you might enjoy a bit of history and some fun facts about the hardware and software that quietly kept the site alive all this time.

A quick trip down memory lane
On 17 January 2012 we put the old server into production. It replaced an even older HP ProLiant ML150 G2 tower server with dual Xeon CPUs that on average chewed through 1 kW per hour – basically an electric heater with network ports (our electricity bill was not happy).

The 2012 machine was a much more compact and power-efficient little rig built in a Thermaltake Element Q Mini-ITX case:

Motherboard: Intel DH67CF Media Series mITX (H67 chipset, DDR3, PCI-E, SATA 3.0, USB 3.0)
CPU: Intel Core i5-2405S 2.5 GHz (6 MB cache, low-power Sandy Bridge, Socket 1155)
RAM: Corsair XMS3 16 GB (2×8 GB) 1600 MHz DDR3 CL11
Storage: Intel 320 Series 120 GB SATA II SSD (boot drive from day one) + 500 GB HDD (secondary storage)

Software & extra services
The server ran Debian Linux the entire time. Yes, we know — it’s not OS/2, but it was the only platform that could reliably host everything we wanted to offer. Some of you might also remember that for many years we provided free hosting for OS/2-related projects, including email accounts. A few of those @os2world.com accounts are still actively maintained today!

Fun facts (and the end of an era)

• That little Thermaltake Element Q rig ran non-stop for over 14 years – longer than many people keep their phones or even their cars!
• The only storage change we ever made was when the 500 GB HDD failed — everything else (CPU, motherboard, RAM, SSD, case) stayed exactly as installed in 2012.
• We did replace the power supply once, but lately both the PSU fans and the CPU fan had started making some rather unhealthy noises… a clear sign it was time to retire the old warrior.
• The unofficial sysadmin motto was “if it ain’t broke, don’t touch it”… and for a very long time it simply wasn’t broke.

Why we finally replaced it
In the last couple of years the hardware was no longer 100 % reliable and the OS plus all the supporting software had turned into a colourful patchwork of updates, hacks and work-arounds. Maintenance was becoming a real pain.

The final straw came in the form of some very nasty DDoS attacks over the last few months. In just a few days the old server had to deal with over 53 million SYNs dropped at the LISTEN sockets and nearly 3.4 million SYN retransmissions — that’s easily 100–1000 times more connection attempts than we would normally see in 2–3 days of regular traffic. The little i5 fought bravely with SYN cookies and every trick in the book, but it was clearly time for something more modern and resilient.

What we have done now
We have moved the entire site to a containerised environment based on Proxmox VE. To make the setup much more robust we have also split the services across separate sub-domains and isolated containers:

• Main site → its own container
• Forum + Wiki → their own containers

That way, if one part ever has a problem, the rest of the site stays online.

Current status & ongoing work
Everything is already live and mostly working well, but we are still seeing quite a lot of ongoing SYN and HTTP flood attacks that are currently targeting Small Machine Forum in particular. We are actively working on further tweaks and performance improvements. Just to be clear: the new container environment and Proxmox VE are not the bottleneck — it’s simply the Apache2 and PHP configuration that needs some tuning and optimisation. We expect to have most of these issues sorted out during the next couple of days.

A big thank-you to the team involved:

• Tommy from Njordium for spending a huge amount of time migrating the sites, repairing database issues and getting everything running smoothly.
• Martin for his patience during the whole process.

If you spot anything that doesn’t behave as expected, please just reply in this thread and we’ll fix it ASAP.

A huge thank-you to the old server (and its trusty Intel SSD) for its incredible service life — it truly earned its retirement. And a big thank-you to all of you for your patience during the transition.

Here’s to many more years of OS/2 news, discussions, and resources!

Best regards,
The OS2World Team

[Martin Iturbide]

Thank you Kim and Tommy for all the efforts during all this time.

I can not believe that OS2World had been 26 years online. We defined OS2World official birth date as March 27 (next friday).
I still remember that I wrote you back on 2000 and told you I wanted to help with the news of the site.

I'm 47 years old and I had been collaborating with the OS/2 Community for more than half of my life, which I feel it as something special :_)

Regards

[David Graser]

I am no expert on DDos attacks and others.  I was wondering if it would help to have a check box saying I am not a robot before allowing access to the site.

I have been using OS/2 since 1993.  The windows blue screen of death and OS/2 work place shell converted me to an OS/2 user.  Along with FM/2, I have never regretted my decision. 

[Kim]

I am no expert on DDos attacks and others.  I was wondering if it would help to have a check box saying I am not a robot before allowing access to the site.

Unfortunate a simple “I’m not a robot” checkbox (like reCAPTCHA) is great at stopping some automated spam or basic bots, but unfortunately it won’t help much against the two specific types of attacks we’re seeing right now on Small Machine Forum.

Here’s a quick, high-level explanation why:

SYN flood attacks: These happen at a very low level (the TCP connection stage) and long before the web server even shows you the page with the checkbox.
The attackers flood the server with millions of fake “hello, can we connect?” requests and never finish the handshake. The server’s connection queue fills up and starts dropping everyone, including real users. The checkbox never gets a chance to appear because the connection itself never completes.

HTTP flood attacks: These are the “normal-looking” requests that do reach the website. Even if we put a checkbox in front of every page, the attackers can 1) Use botnets where each computer can solve the CAPTCHA, or 2) Send so many requests per second that the server is still overwhelmed just processing the flood (before or while checking the CAPTCHA).

What we’re dealing with are malicious attempts to overwhelm the site. The attacks are currently flooding in from roughly 400–800 different IP addresses at any given time (mostly compromised home computers and servers, based on the IP ranges involved).

We’re already working on several more effective protections, including better rate-limiting, connection tuning, and Apache/PHP optimisations. We have also placed the server behind a reverse proxy and made tweaks to the firewall and gateway to catch and mitigate attacks as early as possible. In addition, Martin has kindly agreed to upgrade Small Machine Forum (SMF) to the latest version, as the current one has a few known issues and doesn’t handle bots as well as the newest release.

We’ll keep you updated here in this thread as we roll out these changes over the next few days.

[Kim]

I can not believe that OS2World had been 26 years online. We defined OS2World official birth date as March 27 (next friday). I still remember that I wrote you back on 2000 and told you I wanted to help with the news of the site.

Yes, it has been quite a while. Although I haven’t been very active in the community for a number of years, I’ve still been indirectly involved by helping Tommy with the server maintenance. When we originally started the site many years ago, I was working at IBM in Stockholm.

[RTAN]

Hi Kim,

That was a fun read and an interesting trip down memory lane - thanks for sharing that!

I also echo Martin's thanks to both you and Tommy for all of your efforts.

Richard.

[Kim]

So some additional progress has been made and so far some errors on the Wiki has been fixed including that the old Ultraboard (2003-2007) which was our first forum script to use has been recovered as well now and next up is the first installation of Small Machine forum (2008-2012) is next up and will be added to the The Archives menu section. We've also added rewrite rules so any old hard coded links used in the forum or articles will still work and not throw any errors.

The OS2 Gaming site is also available and sits directly under os2world.com/gaming but the idea is to break it out to it's own sub-domain, gaming.os2world.com, and it's own LXC container

[galaxian]

Hi Kim and Martin. Great update and thanks a lot for all the efforts. I'm just reading all this today and was wondering what had changed, as the site has been loading super quick! Nice to see these improvements. Performance is vastly improved from my end in Canada.

[Gregg Young]

Hi

I am seeing and untrusted connection error using Seamonkey. It won't let me add an exception. The rest of the site seems fine. Dooble connects fine. Haven't tried Firefox. Thanks

Gregg

"This Connection is Untrusted

You have asked SeaMonkey to connect securely to forum.os2world.com, but we can't confirm that your connection is secure."

[Kim]

"This Connection is Untrusted

You have asked SeaMonkey to connect securely to forum.os2world.com, but we can't confirm that your connection is secure."

What does the certificate say if you check that; should be Let's Encrypt -> forum.os2world.com ? Installed and check here on my side as well with SeaMonkey and I don't get any certificate error; just works without any problem.

[Dave Yeo]

I had a similar error, though a few days ago. SM said it was self signed certificate for localhost. Since SM has been loading the site fine with only occasional slow downs.

[Dariusz Piatkowski]

Thank You to all those involved in making our little 'OS/2 Humble Abode' stand the test of time!!!

Re: the certificate error, FF here, yup got that message too, was a little suprised by it at first but given the server changes I simply told FF to accept it through it's built-in exceptions management and it's been smooth since.

[Kim]

I had a similar error, though a few days ago. SM said it was self signed certificate for localhost. Since SM has been loading the site fine with only occasional slow downs.

Do try to remove the risk acceptance for the certificate and try again; believe that it might be caching issue based on that we in short time moved from old server to new server environment and there as well made some changes in the site structures with new certificates and as well using rewrite rules for */forum/* pointing to forum.os2world.com.

[Pete]

Hi All

The new server seems to be working fine and no problems trying to reach os2world.com.

Well done the OS2World Team.


Regards

Pete

[Alfredo Fernández Díaz]

Even if I have been inactive for months...
Congratulations, and a heartfelt thank you!