DevOps and digital transformation – leadership characteristics & key metrics

Gartner predicts that through 2022, 75% of DevOps initiatives will fail to meet expectations due to issues around organizational learning and change. Another study suggested that the DevOps market is expected to reach $6.6 billion in the same year.

DevOps is not a mere trendy buzzword anymore. It is expected to become the standard way of software development in each organization in the coming future as it transitions from the phase of early adoption to early majority uptake. As leaders realize the need to fundamentally transform the way their businesses operate and to keep customer-centricity at the core of their initiatives, DevOps adoption is expected to gain further momentum this year and beyond.

Jermaine Oldham, VP of Engineering & Technology at J.B. Hunt, recently talked in detail about the main characteristics required in a leader to drive a successful DevOps organization, impact of DevOps on digital transformation, and the key metrics necessary to determine the DevOps performance in an organization. This blog is an excerpt from the discussion on the QATalks podcast.

The key characteristics of Quality Engineering & DevOps leaders

The pace of change around us has accelerated many-folds with the ongoing pandemic. What was supposed to be a 4-year transformation plan for several organizations was realized in the span of three months. While keeping up with the speed of change, it is important that the leaders do not fall into a rut and put significant focus on innovation as well. Therefore, quality engineering and DevOps leaders need to embrace a holistic point of view while navigating the digital transformation or DevOps transformation.

Jermaine suggests four key characteristics that these leaders should embrace – “One, lead with confidence. Confidence is the key to know that innovation and change are out there, and you need to embrace it. Two, as a QE leader, you need to be willing to adapt and learn and grow. So, the key thing there is just being flexible to be able to adapt, learn, and grow so that you can evolve as a leader. Three, be willing to challenge your previous exceptions and understandings. Things may not be the status quo. So, you’ll have to certainly challenge your previous assumptions and understandings of what you thought you knew. And then finally, you need to be open to failing. But the key with failing in this space is failing and failing very fast.”

CI/CD/CT pipeline for failing fast

DevOps enables automation and augments and accelerates software delivery and innovation. With a Continuous Integration & Continuous Deployment pipeline, coupled with a continuous testing feedback loop, DevOps offers the scope to fail fast, but also to get up and bounce back faster.

Jermaine elaborates on the concept of failing fast as, “The key thing here is, as I attributed before, failing very fast. It needs to be a kind of a fast feedback loop and a fast kind of evolution of how you work through problems and situations. So, I’ve seen leaders before who get myself and my teams where we failed on something, we tried to implement something, we tried to adopt a new technology. We thought we had the use cases and documentation to support the business, but we didn’t actually hit the nail on the head the first time around. So, in those instances where I’ve seen that we failed in the beginning, we went back, and we iterate it to understand what we needed to do to improve on. We came back with a working model in order to solve the problem. So I would say just that there’s a continuous evolution where you may deliver a solution per the use cases and requirements that you gathered, but you may fail the first time, but don’t be swayed from the overall goal of trying to solve business problems. You want to fail fast and you want to come back and you want to be persistent.”

The rising importance of DevOps in digital transformation

The COVID-19 outbreak has hit the digital transformation initiatives as a wrecking ball. With the accelerated pace of digital transformation and adoption of digital practices, DevOps is also shifting from a ‘nice-to-have’ to a ‘must-to-have’ component within the digital transformation strategy. While offering recommendations to organizations trying to embrace this change coming at them so quickly, Jermaine says, “I would say one of the things that are important is preparing your team for the cultural change. When I think about DevOps and I think about quality, I think about really how you are developing the tool chain and a methodology. The first thing you need to think about is culture, the culture of the team. Prepare the team and yourself to embrace a cross or working model. When you think about the implementation of DevOps in the digital age, it’s cross functional team that consists of not only just developers and IT operations, but quality experts as well , so quality engineers and quality leaders and quality analysts to make that kind of working model. In addition, you need to prepare your team to own the quality as a key pillar to delivering software. I’ve seen teams not own quality. Quality is extremely important and in order to ensure that you’re going to deliver value fast and often to the business, you’ve got to own your own quality. And the last thing I would say on this is you also need to prepare the business to embrace getting valued to software delivery early and often. So it does you no good to have a very stringent pipeline where you’ve got dev and operations working together, where you’re able to build, where you’re able to test and release, and you’ve got continuous testing inside of the pipeline, but the business doesn’t have the capacity to consume that. So, you need to also prepare your business in order to receive this value on a frequent basis as well.”

The relevance and importance of Testing Centers of Excellence

With the introduction of Agile and DevOps methodologies, it seemed that the Testing Centers of Excellence or TCoEs lost their relevance. As quality started to shift left in the SDLC, TCoEs appeared to be redundant and are still in a challenging spot. Sharing an interesting perspective on the importance of TCoEs in a DevOps era – “There is a lot of shift in the overall IT itself is happening, and that includes quality as well. So, the testing center of excellence is becoming a challenging spot right now and it’s getting into a next evolution, I would say. Quality and testing is not going to go away. It will exist in different format. It could be quality engineering, and that’s where the industry shift is happening. I want to get my testers ahead of the game to start working from the requirement itself, I want each of my test engineers to be certified with the quality engineering concept or scaled agile framework concept. In the agile and DevOps model, the whole of the shift is towards building out pockets of excellence, which could be a quality engineering center of excellence, or digital center of excellence, or it could be automation, or it could be DevOps center of excellence.”

On further explaining the concept of the pockets of excellence, “Like the way how agile is split into multiple parts and you call it as a story, or you call it as a scrum and execute them, quality is also shifted into each of those pockets, testing underneath the IT domain itself. For example, if I’m creating an IT life cycle with respect to agile or DevOps, you form a small scrum team. And within that, you are bringing in this test-driven development concept, and bringing in the quality engineers, or bringing in the software development engineer in test, and part of that team itself and creating that CoE within that in-sprint, no development life cycle. So as TCoE is evolving into the next stages of the excellence with respect to quality, we are repositioning the entire IT with respect to continuous testing in the name of quality engineering.”

Impact of DevOps on digital transformation

DevOps has a significant impact on driving customer value through digital transformation. For creating a structured model and methodology for delivering software with quality through automation, it is important to consider automation holistically – automation of builds, automation of deployments, and automation of monitoring. Jermaine says, “When I think about the impact, I’m also thinking about creating a software delivery model that provides a quick feedback loop. So in the DevOps world in relation to digital transformation driving customer value, you’ve got to have a system checks and balances that allow you to fail and fail fast as I was mentioning before, but it’s got to be a continuous feedback. So if my bill failed, because I didn’t pass the security check, I get notification as a developer and I’m able to adjust on the fly and reinitiate that bill to ultimately get that value, which is the code that I’m developing and the features and functions that I’m trying to iterate on the software and deliver a new functions and functionality to the business as fast as possible.”

“I would say integration of testing as a practice needs to be first and foremost and embracing continuous testing is a requirement to success. I’ll tell you that a DevOps methodology in order to deliver software is just as good as testing that’s integrated. So, when I think about tests, I’m thinking about some unit testing of the code, I’m thinking about testing multiple modules together, I’m thinking about doing service calls and service testing, I’m thinking about functional testing, I’m thinking about non-functional testing, performance testing. In the perfect world, this would be integrated as a product of your delivery pipeline, which is your DevOps pipeline. And then you need to create a method to deliver software to the business fast and when they want it, but with higher quality. You never compromise quality for speed, but you want to do a little bit of both. You want to deliver it fast, which is the value proposition, but you want to do it with higher quality.”

Building a DevOps culture and shifting mindset

One of the key components for shifting to a DevOps culture is having a shared responsibility. A DevOps culture drives better communication in a cross-working model to facilitate higher collaboration between dev, IT Ops, and QA teams. Having a shared responsibility and accountability not only enables the delivery of high-quality software at high-speed but also establishes end-to-end transparency.

Talking about transparency and trust, Jermaine says, “When you think about the culture, you’ve got to work with others and you’ve got to work with others very frequently, and you can’t get in a situation where you’re hiding something or trying to suppress something on the development side, or on the IT operations side, or when you’re doing the testing aspects of it. You’ve to be a hundred percent transparent. And you’ve to trust. So in the DevOps culture, one of the underlying factors that you’ve to have is you got to trust one another. You’ve to bring people together, you’ve to communicate, and you’ve to trust one another. If you get that and things start clicking, then that’s when you’re able to really deliver software faster with higher quality.”

The key metrics for evaluating DevOps maturity in an organization

There are five key metrics that allow organizations to evaluate their DevOps maturity – deployment frequency, lead time, change failure rate, meantime-to-recovery, and the total cost of quality.

Talking about these metrics, Jermaine elaborates, “One is deployment frequency – how able, how fast, and how frequent am I able to deploy to production, to lay binaries to production services and servers, and what’s the frequency of that. Another is lead time that really measures the amount of time that occurs between starting a work item until it’s deployed. So that really allows you to measure the time of value from the time that you’ve logged that user story using your scrum or Kanban methodologies and from the time that it’s logged to the time it’s deployed. Another key KPI and metric in this space is change failure rate. You want to deploy frequently, but you don’t want to deploy frequently if you’re failing. You want to be able to measure your change failure rate. From a time that I’ve deployed, what’s my failure rate with that because that’s going to give you that quick feedback loop in order to change to make your deployments more seamless. And meantime to recovery – from the time that you do deploy and you do have a failure, this metric really helps you track how long does it take to recover from failures. That’s important because not all deployments, not all the solutions deployment will be successful. So, measuring meantime recovery will allow you to measure how long it took for you to recover so that you can get faster and faster and faster, and ultimately really delivering value to the business at a fast rate with higher quality with the least amount of thought as possible.”

Explaining the total cost of quality, “You want to see how you’re improving the total cost of quality and the meantime to recover – How quickly I fixed it, how quickly I tested it and released it to production, and have your quality as the metrics.”

To sum up

Implementing DevOps testing practices helps CxOs bring about a cultural shift that leads to greater collaboration across teams, improves the quality of code, increases the response rate of business changes, inculcates shift-left practices, & creates an environment of growth for Software Development Engineers in Test (SDETs) – all of which are necessary to provide DevOps solutions.

At Cigniti, we standardize efforts and ensure accelerated time-to-market with DevOps testing solutions. We focus on delivering improved deployment quality with greater operational efficiency. Our DevOps testing specialists with their deep experience in Continuous Integration (CI) testing & Continuous Deployment (CD) help configure & execute popular CI/CD tools supporting your DevOps transformation & application testing efforts.

Schedule a discussion with our DevOps testing experts to know more.