Building robust applications

[Gnome]

Interesting video talking about Netlix's architecture:
http://www.infoq.com/presentations/netflix-architecture-resilience

A lot to be learned from their architecture, something like disaster recovery doesn't need to exist in their architecture because of the way they designed their system.

Very impressive and in my opinion this is where software development is moving, alas most SA software houses will likely take quite a few years to start moving to that architecture

 

[Batista]

Impressive but they are a huge company streaming data over the world and it needs to arrive fast.So i wouldnt have expected anything less.I didnt watch the 50 min vid though.

 

[JStrike]

Nothing too exciting there. But it will be interesting for those that are new to HA software
And the good SA software companies are most certainly doing similar architecture (Twitters Storm is far more interesting actually)

 

[semaphore]

Storm is nice.

 

[cguy]

The software I would write at my previous employer would go out to many millions of diversely configured machines. I learned a fair amount there about building robust software. Some tidbits:

In terms of infrastructure, there were entire data centers running tests 24/7 - directed, full application and scenrio replay tests, running on every conceivable piece of hardware, in any relevant configuration. Using some sophisticated maths to figure out which tests correlated in failure and performance, in order to avoid running redundant tests, and how to automatically detect performance regressions in noisy environments. Reproducing failures in the lab, and recovering from failures in a sane way without disabling the entire testing grid. Running a hierarchical level of tests (light auto-testing per-checkin, heavy auto-testing on nightly snapshots, and days of mostly full-grid testing prior to signing off on a release cut).

Apart from that, there was a name and shame approach used to discourage breaking stuff (email would go out to everyone if you caused a test failure). Very tight coding standards, and mechanisms to pre-test prior to submission, and a lot of defensive coding. It was quite a science - I had no idea these things were so complicated, and prior to this I never had full appreciation for the saying: you can't go forwards, until you stop going backwards.

 

[shauntir]

The software I would write at my previous employer would go out to many millions of diversely configured machines. I learned a fair amount there about building robust software. Some tidbits:

In terms of infrastructure, there were entire data centers running tests 24/7 - directed, full application and scenrio replay tests, running on every conceivable piece of hardware, in any relevant configuration. Using some sophisticated maths to figure out which tests correlated in failure and performance, in order to avoid running redundant tests, and how to automatically detect performance regressions in noisy environments. Reproducing failures in the lab, and recovering from failures in a sane way without disabling the entire testing grid. Running a hierarchical level of tests (light auto-testing per-checkin, heavy auto-testing on nightly snapshots, and days of mostly full-grid testing prior to signing off on a release cut).

Apart from that, there was a name and shame approach used to discourage breaking stuff (email would go out to everyone if you caused a test failure). Very tight coding standards, and mechanisms to pre-test prior to submission, and a lot of defensive coding. It was quite a science - I had no idea these things were so complicated, and prior to this I never had full appreciation for the saying: you can't go forwards, until you stop going backwards.

What type of software did you write?

 

[cguy]

What type of software did you write?

System software - drivers, kernel, etc. When writing the ring 0 stuff, it meant that we had to be really robust - accessing the wrong piece of memory could bring down the whole system.