Shifting Left: Using quality cost as a force to overcome resistance

A Petro-chemical company is collaborating on the requirements for a new order transaction type that must meet the needs of customers requesting that orders be split across multiple shipments. Rep from transportation group is absent. The team defines the split shipment requirement to the business analyst team for design.

The vital piece of information that never made it into the requirements was the compartment capacity of individual tanker truck types. The lifecycle continues with the requirements defect now nested in the solution, technical specifications are drafted, the new order type is coded, interfaces are modified, data objects are defined and mapped, test cases are written and executed.

Not until the User Acceptance Test is the design flaw identified. At that point, it is estimated that the effort to resolve the defect, including modifying impacted technical and data objects, changing supporting designs, and re-writing test cases, will run into thousands of man-hours. In fact, several studies have indicated that the cost of fixing this defect during UAT will cost about 50 times more than what it would have cost to resolve during the design phase.

Causes for Test Squeeze

Some of the causes for Test Squeeze includes underestimated development timelines, Scope creep, and Complexity of requirements. Also, the habit of recreating what we do now and lack of investment in process optimisation leads to unnecessary mods.

Few other causes include Underestimation of the downstream impacts especially when legacy systems are involved. The perception of “test” is not value add and the language has to be changed to quality management / assurance.

The Waterfall habitually leaves all testing to the end rather than dropping modules into test. The high defect rate with defects built in so much harder to change.

The data quality is often poor, leading to invalid test cases. For example, GML Reporting: 100% success rate on data integrity testing reported.

Digging further showed that out of a data set of 10s of 1000s of rows, the Test Data set had been reduced to 19 rows, all of which succeeded.  Every other row had failed. The “happy path” testing does not allow us to anticipate issues in production. There is a need to model “what could possibly go wrong?”

The cause of the test squeeze is also due to the underestimation of the scale of test effort, exacerbated by unrealistic estimates. For instance, the quote to test vanilla Oracle RMS core functionality > 11-man years.

What drives this behavior, cognitive dissonance?

Retailers are very cost sensitive: focus is always on reducing costs, maximizing profit and IT projects are high cost up front, often with long paybacks. Reluctance to invest at the beginning leads to panicked “whatever it takes” at the end.

• Lack of holistic focus over the project timeline
• The fallacy of higher net cost associated with involving testing early.
• “We’ll focus on testing later” philosophy
• Dual use of BA’s as testers later in the cycle, at a time when stage gate violations render them unavailable.
• Perspective of testing being a decelerating ‘Gate Keeper’
• Use of the Test Phase as a buffer / shock absorber for upstream overruns, aka Test Squeeze.

We never really take account of the true UEL of a solution.  5-7 years for deprecation doesn’t really reflect how long a system has been in use. We have systems that have been in place for years and others for just 15 years, but we under-invest at the beginning to achieve a quick payback. A lack of understanding of the difference between config and dev means that we don’t always target the right areas for thorough testing. Mods are delivered towards the end of the project lifecycle, and then we find defects to fix rather than designing them out at the beginning.

Lack of understanding the REQUIREMENT rather than the DESIGN means tests are designed, and can be passed, based on code not function so code then fails in UAT and must go back for further fixes/test cycles. The requirements change over time and are not well expressed.

The acceptance criteria become negotiable, resulting in cost and timeline uncertainty:

• I have been asked for firm costs before Discovery starts.
• Costs are ALWAYS challenged, regardless of the experience of the estimator
• “cone of uncertainty”
• Not asked “what can you do for this budget” but “squeeze the scope into this budget”.

The availability and skill set of the end user group are often overestimated. The level of organisation required and the preparation of test cases and data are underestimated. It goes hand in hand with the underestimation of non-production environments.

Universal impact of squeezing QA to the right

The impact of squeezing QA to the right is that it increases fix costs by up to 100 times. It leads to a diminished Test Coverage, sacrifice of testing independently and impartially, and inability for production like test.

These factors lead to Undetected Defect Leakage, Massive and Frequent Regression Tests, Increased labor costs, Unwanted staged releases, Workarounds, and Go-Live Delays.

Quality from Day Zero – The Fundamentals of Shift Left

A quality environment foresees quality activities kicking off and running in parallel to the entire development lifecycle. So, to be successful, it should be orchestrated by a single organizational entity. This could be the PMO or Project Manager, but for larger projects a better solution would be a dedicated resource with a background in testing. This elevated role would have responsibility for defining the overall quality strategy, assembling the resources and assets required to implement it, and have the clout to stand up to project leadership on implementation, particularly at Quality Gates as collaboration points.

Enable Quality through the Empowerment of the QA Team: Appoint a program or group-wide head of quality assurance. Empower this person to engage from day one of any project, from scoping through production. Bestow the authority to make project-wide decisions as part of a steering committee. Educate the entire project team on the importance of quality from Day Zero.

Arm a Quality Program with Industry-Leading Processes:

Deploy Quality-Driving Tools: Deploy tools which support the management of deliverables created during the project lifecycle, including requirements, designs, development objects, test cases, and defects. Deploy tools that support the generation of efficiencies and enable more comprehensive test coverage. Deploy tools that support the validation of quality in design and structure.

Here is the link to access the on-demand webinar that was conducted on August 5^th on Shifting Left: Using Cost of Quality as a Driver Against Resistance.

Why Should You Watch the Webinar?

Gartner Research finds that “Application leaders are under increasing pressure to maintain quality while increasing the velocity of delivery.”

QA leaders are aware of practices such as Shift Left Testing and their ability to create a highly productive development lifecycle that helps achieve faster time-to-market. However, they are often faced with organizational resistance when it comes to adoption.

Watch this webinar as Robb La Velle, Vice President – UK and Europe, Cigniti Technologies, and Maryanne Fleming, Head of Technology, Monsoon Accessorize, shared their insights on making a case for Shift Left, establishing an end-to-end strategy, anticipating challenges, and managing change through a real-world enterprise case study.

Who should watch?

QA and IT leaders from enterprises in any industry or region will find this webinar insightful.

Conclusion

Being a worldwide leader in independent quality engineering services, Cigniti is a strong advocate of Quality Assurance and its implementation right from the initial stages of the software lifecycle. We encourage customer feedback and believe in including such feedback in our broader quality assurance approach. We take great measures to make sure that we are fully equipped with state-of-the-art services and have partnered with other experts that specialize in providing testing services. Talk to us.