What is the Accelerate Model?
The model discussed is described in the book: Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations. Simply put, the book outlines a large research project that quantitatively and qualitatively surveyed software engineering organizations. The output of the surveys proved correlation and causation between the maturity of capabilities in the organization and how they ultimately affect organizational performance.
The core metrics that are measured by the Accelerate model are: Deployment Frequency, Change Failure Rate (%), Mean Time to Resolution (MTTR), Cycle-time, and Reliability.
Our Accelerate Path @ ACV
In the fall of 2019 we started our path with Accelerate by recognizing that our weekly, after-hours release to prod was not working. We were a growing engineering team of about 50 people and the complexity was becoming too high to release with confidence. We quickly became overwhelmed at which point we retroactively measured our change failure rate. It was 47%. This meant it was effectively a coin toss of whether or not our release to production would “fail”. The issues ranged from DB migrations gone wrong and pods connecting to old deployments, to broken routing configurations and everything in between. We knew that a quick fix here or there would not cut it. Finally in October of 2019, after our 5th failed and painful Tuesday night release, we pulled the andon cord and stopped the our deployment process entirely.
We decided on a hackathon style approach because we knew we needed results fast. That Wednesday morning, we got all of our engineering managers together and immediately started a 3 day planning session. We knew very quickly what our goals were, and of course, came up with a name for our plan: Release Confidence Week.
Wednesday
Our objective was simple: Improve our Change Failure Rate from 47% to 20% with a stretch goal of 10%. We defined a few control metrics including customer NPS and developer churn, and we identified the top opportunities or themes.
Thursday & Friday
We brought in the entire engineering team to start working on a brain storming exercise. We collected the top ideas into a google slides and had over 20 different projects proposed by our engineers. We ended by grouping them together based on the themes from Wednesday. Friday morning, we printed them off and posted them on the 150’ whiteboard in our old office. By noon on Friday, every engineer made two votes on the projects and we settled on the winners.
That afternoon our engineers and managers self selected project sponsors and teams, and we coordinated with our Product Management team to reprioritize our roadmaps for the next two weeks. We were finally ready to get started.
Monday (Hacking Begins!)
We spent exactly two weeks with this modified organization and hackathon. We had hiccups here and there but ultimately made massive progress in our processes, reporting, architecture, infrastructure, and quality engineering.
Since then, we have grown leaps and bounds on our practice and process, and we have not had to make a major course correction since. Today we ship more than 150 times per week with < 10% change failure rate. We have also added investments in analyzing our source of delays to improve cycle-time, reliability and MTTR.
The Numbers
We have been tracking our software delivery performance metrics since the fall of 2019. It has become a more automated process, but is still simple. We measure the number of releases to production out of Kubernetes and report them to BigQuery and a google sheet. We have MTTR and failure data from PagerDuty that is correlated after the fact alongside our blameless postmortem process. Finally, we pull our cycle-time data directly from Jira.
We use Tableau to visualize these metrics and review them weekly with managers and present them out to the organization to today.
If you’d like to learn more about these practices please check back into our blog, as we will be sharing more content on these topics. And of course, read the Accelerate book!
