Building Webpop on top of the Rackspace Cloud has been a fantastic experience for us, and we're happy to be their newest Cloud Tools partner. In this blog post, I'll dive into the details of how we've built a truly scalable CMS on top of their Cloud Servers infrastructure.
One of the things that always makes me sad, is when I follow a link from Reddit or any of the myriad of other social news sites, and get a page that simply times out or throws a server-side error. For someone with a blog or a company site, getting hit with lots of traffic should be a marvelous thing, an opportunity, an event to be celebrated. Not a moment of frantically struggling to get a server to survive while presenting people with your domain plus a broken site. Having your website die, just when you're finally getting a bunch of visitors sucks. For any small business, it's a wasted opportunity.
One of our goals, when we set out to build Webpop, was to make sure this would never happen to anyone hosted with us.
The first thing you'll want, while building a system to have as little downtime as possible, is redundancy. Servers will fail every once in a while and anything that fails from time to time always tend to pick the worst time to do it. If any part of a system depends on one lone server, all of it will fail when that one critical point goes down.
While building Webpop, we have taken great care to avoid depending on any single server. One key to achieving this right from the get go has been the inexpensive 256MB instances that Rackspace offers. There's a vast difference between developing for a single server and developing for a distributed system.
An app built to run on one server will not benefit from two of them, but an app built to run on two servers should easily be able to take advantage of five, ten or hundreds.
By using a small cluster of inexpensive mini-instances, we have been able to develop and test everything as it would work in a full production system, but at an extremely low cost. As Webpop has grown bigger we've simply replaced the old mini-instances with larger servers, while the basic architecture has stayed the same.
The Webpop Architecture
Webpop is primarily built using the programming language Ruby and our data is stored in MongoDB.
MongoDB is a document oriented database and part of the wave of new NoSQL databases. Being document oriented rather than relational, makes MongoDB a very good fit for a system like ours where each project can have its own content types. MongoDB is designed to be fast and easy to distribute with its built in support for replica-sets and auto-sharding.
The main Webpop application is split in two smaller applications each with their own area of responsibility:
Where Rackspace Cloud Servers really shines, is when it comes to provisioning new servers. Getting a new server online can be done in a few minutes; adding or removing servers can easily be accomplished with API calls, but taking advantage of this requires work however. Getting a server online is one thing. Getting a server configured, getting your load balancers to know about it and getting your app up and running on it, is something quite different.
At Webpop, we use a program called Chef to configure new servers. We have our own central chef server and a small script that will perform a few tasks. The script will automatically launch a new server with a specific role, get chef to install everything and deploy our app to it. Chef also handles configuring load balancers, so they know about the new server, opens the firewall on the database servers, so the new server can connect, sets up relevant checks and monitors for the new server as well as cooks us dinner (ok, that last part we're still working on).
This automated infrastructure means we can very easily add capacity to any part of our system if we see the load increase and it gives us a great capacity to respond to sudden peaks in traffic.
Monitoring, logging and historical data
To see if the load increases or decreases, and to make sure we know straight away if something is wrong, we use a number of tools to monitor and watch our servers. Perhaps the most important tool is Wormly, an external service that runs checks against our websites from several servers around the world. If any test fails, our phones start ringing straight away.
Normally, things should never get that far before we notice. Internally we use a monitoring application called Nagios to keep an eye on our servers and applications. Nagios will send us alerts if any queries in our database start running slowly, such as if we start having requests queued up in our load balancers, or if a server starts getting low on disk-space, memory, etc.
To keep an eye on long-term tendencies, we use another tool called Ganglia. Where Nagios will sound the alarms as soon as a check fails, Ganglia gives us an overview of where we might run into problems in the future by showing graphs on just about anything from disk use and CPU activity to network status and application-specific metrics.
We also use an external service called New Relic to keep an eye on both Amoeba and Primordial Soup.
All our logs go to a central logging server where we can easily search through them when investigating bugs or problems.
Built for the Cloud
This is a brief overview of how we've been using the Rackspace Cloud to create a truly scalable CMS, a CMS built for the cloud which offers a quality of hosting that goes far beyond what you would achieve with a shared host or a VPN and all this at an affordable price.
Tools like Rackspace Cloud Servers can enable companies like us to build scalable infrastructures in a way that simply was not economically viable just a few years ago. We firmly believe that traditional shared hosting is a thing of the past that makes less sense every day. Cloud tools like ours are the way of the future.
Public clouds make great things possible, but in order to take advantage of them the infrastructure should be built from the beginning with the cloud in mind and automated server management and configuration is a must.
By using Webpop to host your websites, you get all the benefits of true scalable cloud hosting without the hassle to take advantage of it.« Back to Blog Index