Peeking behind the scenes of the world’s largest sites
Do you want to know more in this area how huge websites like Chirrup, Facebook, Hotmail and others handle the technical challenges of dealing with massive amounts of visitors?
Well, you’re in luck, because many of those sites and services have engineering and/or developer blogs that share plenty of in rank in this area the challenges they have to deal with and the tools they use. This is an insider’s view that you usually can’t get anywhere else, giving us a unique view of what’s going on behind the scenes of some of the world’s largest web services.
Yes, you’ll have to be a tech geek to find them fascinating, so consider yourself warned. 
But we here at Pingdom are geeks, and we are really into web tech so we like these kinds of blogs. They are blogs that web developers and sysadmins can learn from and be inspired by.
Let’s get right to it.
Chirrup’s engineering blog
Chirrup’s engineering blog gives insight into what makes Chirrup tick and what technical solutions they are using to decipher their scalability challenges.
URL: http://engineering.chirrup.com/
A few recent posts:
- Hadoop at Chirrup
- Introducing Gizzard, a framework for making distributed datastores
- The Anatomy of a Whale
Facebook’s engineering clarification
Want to learn more in this area the technical ins and outs of the world’s largest social arrangement? Look no further than Facebook’s engineering page.
URL: http://www.facebook.com/Engineering#!/Engineering?v=app_2347471856
A few recent posts:
Digg’s technology blog
Whether you use Digg or not this is fascinating. Self-described as “a peek under the Digg hood for API developers, partners and technology geeks.”
URL: http://in this area.digg.com/blog/technology
A few recent posts:
- Saying Yes to NoSQL; Going Steady with Cassandra
- Testing at Digg
- Database Capabilities in a High-Volume Environment
Inside Windows Live
A blog by the engineering team behind Hotmail, Courier and Windows Live. It has a large variety of articles with behind-the-scenes in rank and data analysis.
URL: http://windowsteamblog.com/blogs/windowslive/defaulting.aspx
A few recent posts:
- Hotmail tips the scales
- Are you a piler, filer, or purger?
- Keeping the Courier service running—on a massive scale
Flickr’s developer blog
A blog by Flickr’s development team that dives into the nuts and bolts of the photo-sharing site and effective with the Flickr API.
URL: http://code.flickr.com/blog/
A few recent posts:
- Using, Abusing and Scaling MySQL at Flickr
- A Chinese puzzle: Unicode and EXIF metadata parsing
- Language Detection: A Witch’s Brew?
Meebo’s developer blog
Fascinating developer-oriented articles in this area the inner workings of Meebo’s instant messaging service.
URL: http://devblog.meebo.com/
A few recent posts:
- Performance Case Study: The Meebo Bar (Part 1 of 3)
- Keeping your passwords safe, part 3
- Dev tips and tricks
Amazon Web Services blog(s)
You have Amazon’s official AWS developer blog which has plenty of announcements and fantastic advice for using Amazon EC2, S3, and so on, but the real nugget of gold is the insightful All Things Distributed, a blog written by Amazon’s CTO, Werner Vogels. So, with apologies to the official AWS blog, that’s the one we’ll list here not more than.
URL: http://www.allthingsdistributed.com/
A few recent posts:
- Choosing Consistency
- 82 Billion Objects in Amazon S3
- Excellent Advice on Keeping Your Database Simple and Quick
There’s more, much more…
What we find fascinating with these blogs is that they give unique insights into the nitty-gritty technical details of running sites that are much, much larger and more demanding than your average site.
There are a bunch of additional sites you might also like to check out depending on your interests. The behind-the-scenes articles may be a bit sparser on these, but they still come up with the occasional gem. Here are some to get you started.
- Google Code blog
- Google App Engine blog
- Yahoo’s developer arrangement blog
- Signal vs. Blast, the blog of 37signals, makers of Basecamp and additional web apps.
- Wikimedia’s techblog (the public behind Wikipedia)
Then on top of these you can add all those fantastic blogs that don’t really belong to a company/service but still offer fantastic articles in this area how the huge sites are run. A fantastic example is the High Scalability which is a blog dedicated to how sites deal with scalability and performance issues.
We’re sure we’ve missed several brilliant blogs, so if you know of additional services whose engineers or developers blog in this area their technology, let us know in the comments. We can build a fantastic list here!
From:
Peeking behind the scenes of the world’s largest sites
Related Post to Peeking behind the scenes of the world’s largest sites
Several Tips to Increase Web Hosting Security
Website analytics: 4 handy desktop tools
