Scoop Does Not Scale

For the last year Kuro5hin has been experiencing database corruption, timeouts, downtime, and packet loss of a frequency and volume both detrimental to site usage and inexplicable by any means. Now, after 12 months of unacceptable instability—especially in light of the $35,000 donated to Rusty to run Kuro5hin—I am moved to examine the outages and poor performance that have been plaguing this site. It is my responsibility to the average Kuro5hin user to expose the culprit.

You see, the problem is not the computer the database is running on, nor is it the presence or lack of multiple OC3 connections to the backbone. The plain fact of the matter is that Scoop just doesn't scale.

It is moot how much RAM or MHz you throw at Scoop. One could donate a quad 2.8 GHz Xeon server with 24 GB of RAM running FreeBSD 4.9 and Scoop and Kuro5hin would still time out after 600 seconds. After a certain point the muscle behind Scoop just doesn't matter: If the activity on a Scoop site reachers critical mass Scoop just won't be able to handle it, period. I was the admin of just such a site which is now no more due to a popularity soar and subsequent activity increase. Kuro5hin long ago exceeded its own critical mass and has been living on life support for some time.

Of course, Rusty Foster is certainly no computer scientist. Nor is he even a system administrator. He is barely qualified to consider himself a computer lab troll. No, gentle reader, Rusty Foster is simply a little blonde nature boy living in a little log cabin on a little secluded island near Maine who likes to code Open Source tools and write about his Earthen skills. Just as the Open Source heroes he worships, Rusty Foster ignores the rest of polite society and would rather use substandard tools and broken software.

And where does all of this leave Kuro5hin? Stuck on a burning site while the ruler fiddles. We've seen this before in an egg and it is not known as progress; it is called going bad.