I’ve now fixed all but one of the outstanding issues for 2.01. I think you’ll find that this site is now much faster and more stable than previous releases, which was the point of recent work I’ve done on it. My apologies for several bouts of downtime the site took over the last couple of days while I updated the code.
Most of the changes were on an infrastructure level. The Rails framework on which the site is built was upgraded to 0.11.1 (the latest and greatest). I also optimized the main page and the article display page (the most often hit pages), reducing the number of database accesses necessary to render the pages. The database itself has had several new indexes and hints implemented to serve up the data more quickly. The query that lists all articles on the main page is still rather ugly, and I will continue to work on that one.
The biggest change, and the biggest performance gain, was that I’ve moved away from the Apache webserver working in CGI mode. This old method required a new process for every page access and loaded the entire code-base from scratch every time. Now we’re running on lighttpd, a very fast, lightweight webserver. It’s using FastCGI, a container that is built to optimize the caching of several instances of the code and re-use them for requests as they come in.
Overall, the benchmarking has shown that the improvement is impressive: over 25x faster after all is said and done. I think you’ll find that this is now the fastest version of the website I’ve ever had. Database accesses for the most intense page (the front page) are now well under .05 seconds even under moderate load. And that page is the one with the killer unoptimized query.
Here’s an example of how long it took to create the main page before the update:
Completed in 1.794307 (0 reqs/sec) | Rendering: 0.257199 (14%) | DB: 1.632127 (90%)
And after:
Completed in 0.112052 (8 reqs/sec) | Rendering: 0.052303 (46%) | DB: 0.000865 (0%)
While I don’t expect future improvements to be so dramatic, there are still plenty of things to do to make the app faster. As you can see, rendering the page (handled by the Rails framework) is the biggest bottleneck now that the database has been reasonably optimized. The newest version of Rails has several new caching schemes available that I intend to try out, so that will probably be the next bump in performance.
I do still host the app in a shared environment, so load from other users can affect things, but I think things should be much better from here on out. As always, please let me know about things that might have been broken by this update.