Backups and security update
Since the recent unpleasantness, I have been busy working on shoring up the Stellar’s backend. The RDS backup retention period at Amazon has been increased. (RDS is Amazon’s Cloud Relational Database Service, i.e. the thing I use for storing Stellar’s data.) Periodic offsite backups of the database are in effect…in fact, there’s one happening as I type this. The size, power, and speed of the RDS instance has been increased to make sure that the DB doesn’t run out of memory anytime soon. Amazon has a service called CloudWatch which allows you to set alarms that will email you if something happens (say the crawler server pegs at 100% CPU for five minutes)…I now have several of those set up for the DB and web server instances.
A big part of the site’s operation are the crawlers and streams on the backend that are constantly harvesting faves from the various sources. Almost 25,000 crawls are done each hour, along with almost 8500 accounts streaming faves in from Twitter. Those processes give the server a good workout and need to be restarted periodically. The process by which this happens has been updated, which has increased the stability of the crawler server and guards against missing data.
I’ve recently purchased a SSL certificate for the site and will soon make the sign-in process secure. Or maybe I’ll make the whole site secure…that seems to be heading towards best practice status these days.
And as always, Stellar’s source code is stored on my local machine, on a code repository in the cloud, and on multiple servers at Amazon.
I am optimistic (but remain vigilant!) these improvements will keep Stellar running smoother, faster, and more secure in the future.