Scaling DevOps at Pearson

Aug 22, 2017 8:00:00 AM By Derek Weeks

seanmack1.png

Pearson might be one of the more influential companies you have never heard of. Their footprint in the educational publishing marketplace is expansive, and their scope was just one challenge when they made the move to DevOps.

Sean D. Mack (@SeanDMackNYC) is formerly the VP of Operations and Application for Pearson. To give you an idea of the scale of their operations when they began implementing DevOps, Sean oversaw 110 development teams and 3,000 different products across the globe. To further complicate it, Pearson is a mature company with both legacy and new products. This is a case study of how they approached implementing DevOps with the goal of helping you learn how to do it. Sean presented it at the 2016 All Day DevOps conference.

As they say, you often learn your biggest lessons with your failures. Sean noted that, ultimately, DevOps was a failure at Pearson due to organizational change. But, he presented the case study with the hope you will learn from what they did and take it forward. He continues to be a passionate evangelist for DevOps.

Pearson had their challenges. They were a siloed organization operating at a large scale, and they had a toss-it-over-the-wall mentality to dealing with problems. Out of sight, out of mind.

seanmack1.png

In addition, all of their teams were at different levels of maturity, from legacy teams running .Net on Windows 2000 servers while others were doing CI/CD to AWS. Obviously, this creates challenges when implementing DevOps, but we will see later how they tried to tackle that, specifically.

 

At the high level, the solution involved several aspects:

  • A DevOps Practice Area
  • A DevOps Steering Committee
  • Measuring progress based on global KPIs focused on measuring business impact
  • Measuring teams based on a DevOps scorecard
  • Push from the bottom - plenty of developers and operations who embraced and were eager for the change
  • Push from the top - support from key executives

 

Pearson’s DevOps Steering Committee set-up several workstreams:

  • Tracking tools
  • Metrics
  • Deployments automation and monitoring practices
  • Test automation
  • App engineering resource contention
  • Operationalization
  • Communications

While these are specific to Pearson, many may be applicable to your organization. In any case, it is important to step back, look at your organization, and decide what your organization needs.

Pearson also looked at four core competencies to track and measure: automation; independence - how independent is the feature; operationalization - how easy it is to maintain; and, continuous improvement. The below chart outlines each area in more detail.

Automation

Continuous Improvement

Independence

Operationalization

  • Unit tests
  • Critical Scenarios
  • Dependent Systems
  • Continuous Build
  • Continuous Deployment
  • Test Automation
  • KPI
  • SLA
  • Metric Tracking
  • Metric Reporting
  • Remediation
  • Isolated Configuration
  • Code Dependencies
  • Isolated Infrastructure
  • Monitoring and Alerting
  • Tier 1 Service Desk System Knowledge
  • Tier 2 Service Desk System Knowledge
  • Documentation

 

 

In order to help each team measure their competencies and improvement, Pearson built a self-reporting tool to empower teams but encourage visibility, and they developed business metrics and then a competency model to measure teams against to see how they were progressing. As Sean noted, “We were not trying to get everyone to a 5. Some teams, yes, but we also had legacy products that were running for years, successfully, with little or no maintenance. The investment to bring the teams up wasn’t worth the effort. There had to be a business reason.“

seanmack2.png

In the end, Pearson raised DevOps maturity for all of the teams. As noted earlier, though, this was eventually phased out, but Sean presented this case study with the hope you found something to drive DevOps in your company.

In the end, if you have just a couple take-aways, they are this:

  1. Begin by determining what your desirable outcomes are.
  2. Don’t just do DevOps to be cool. Do it to drive business results.

You can watch Sean’s entire talk online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free-of-charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed).  It’s all free and online on October 24th.