The role of Artificial Intelligence in Quality Assurance

Software testing efficiency and software testing effectiveness are two key metrics that determine the overall progress of a test strategy. Artificial Intelligence (AI) and Machine Learning (ML) in testing essentially focus on these two parameters. AI & ML can optimize risk coverage, prevent redundancies, perform portfolio inspection, detect false positives, diagnose defects automatically, and analyze user experience.

It is estimated that more than 60% of the test cases in an enterprise test case portfolio are redundant. AI identifies such test cases that are physically as well as logically identical and eliminates the duplicates, which do not add any business value and can be removed without decreasing the business risk coverage. AI is capable of maximizing defect detection and risk coverage while minimizing costs, execution time, and the number of test cases by identifying the optimal test sets. It can uncover weak spots in test case portfolios by tracking flaky test cases, unused test cases, untested requirements, and those test cases that are not linked to the requirements. Additionally, AI has self-healing automation properties, which means it can heal the broken automated test cases and make test automation better resilient to changes.

All in all, AI makes software testing smarter while promoting higher efficiency and effectiveness. Kevin Pyles, Director of QA at Domo Inc, discussed the impact of Artificial Intelligence in Quality Assurance on our recent podcast. This blog is an excerpt from their interview on QATalks podcast.

The present state of AI-driven software testing

AI has been buzzing around since the 1900s and it still upholds the hype across the globe. Everyone keeps talking about the possibilities of the role of AI. However, there is still a wide gap between where AI has reached today and where it has to go. Kevin explains the present state of AI in software testing as, “The vision, the hope for everybody is that someday, AI will be able to do the testing for us. We’re not there yet. I’m not promoting that. But what is definitely here is AI-based tools and AI that helps us with our jobs. So, we shouldn’t look at it as AI replacing testers yet, we shouldn’t look at it as AI replacing really most of our processes yet. What AI does right now is it helps us be better testers, meaning it takes out some of that mundane work that we wouldn’t like to do anyway. Or maybe as we’ll hear a little bit later, AI can help us do things like prediction or analytics better than we’ve done in the past, which just allows us to do our jobs better.”

AI resolving the ‘Test Automation trap’

Software testing is a time-consuming and cost-intensive activity. A challenge with regular test automation is that by the time the test code is completed, the requirements start changing and applications start evolving with regards to business functionality and UI. This means that the whole effort put into developing the test code goes into vain and you need to adapt the test automation needs accordingly. Kevin calls it the ‘Test Automation trap”. He explains, “Test automation trap is when the test teams are not getting enough time to be able to do the failure triage from the previous test run before building the next test automation code. That is where AI can be really used to solve this dilemma and to accelerate the manual testing. With some of our clients, we are able to apply AI in the context of prioritizing test cases and also maintaining the test automation code in an automated manner, as opposed to manually investigating what needs to be changed. And I expect that over a period of time, we’ll see that it can play a great role in the context of analyzing the test results and also deciding on what needs to be tested and things like that, which can happen freely without human intervention.”

Realizing the DevOps transformation with Intellectual Property

In a way, the DevOps transformation is similar to the previous transformations, say, Agile or Waterfall. Even DevOps has been rebranded to DevSecOps or QASecDevOps so that everybody is involved in this transformation. Kevin advises, “What you really need to do is look at your business, what does your business need, and what best practices are good practices out there that can be applied to your company? And why are we always looking externally for solutions when we probably have a lot of really smart people internally that could help us develop our own techniques and business processes, dev processes, and techniques. We should probably do a better job there and we can improve what we have. And so, we need to look at what makes sense for our companies and apply whatever new terminology we want to apply. But more importantly, apply practices that will help us develop better products that suit the needs of our customers and our businesses.”

AI and Ml-driven dashboard for greater productivity

There is no dearth of data in the present ecosystem. However, there is a concerning dearth of the abilities to collate the available data in one place, derive valuable insights from this information, and apply it into the day-to-day operations for improving productivity. Intelligent dashboards, like the one Kevin developed at Domo, allows the stakeholders to pull and visualize the data from anywhere and share it across the company for real-time status updates. While elaborating his perspective on the power of the dashboard, Kevin states, “They give a quick status update. And I believe they answer just about any question that executives would ask – So, what’s the status of the project? Are we ready to ship? How many bugs are coming in? And then how do we feel about the release? And all of those questions are answered using this dashboard.”

As more and more organizations are moving into agile and DevOps and there is a great need to get real–time insights and analytics, the teams at Cigniti are able to act decisively in terms of the changes that need to be done to the projects. And of course, something like this is also needed to be able to make release readiness decisions. The dashboard that we have built comes with the ability to analyze the data and provide the data from a descriptive, diagnostic, predictive, and also prescriptive point of view. So not only it can tell you what happened, but with the AI/ML features, it also has the ability to predict what could happen based on their previous projects of similar size and scope. In essence, the dashboard is ultimately built to drive better business outcomes and improve predictability and also accelerate the transformation that is happening within the organizations.

To sum up

There is certainly a lot of hype around AI in Quality Assurance, and constant efforts are being made to narrow the gap between this hype and reality. We might not be where we want to be in terms of AI-led testing, yet. But we will be there in the near or distant future.

Cigniti’s BlueSwan™ is a next–generation Proprietary Testing platform. Aligning with the needs of the market in the digital era, BlueSwan is Cigniti’s enabler towards being the leader in independent software testing services. It also fortifies our IP led software testing services approach along with the Cigniti 4.0 services and Domain centric approach for offering world–class testing services to businesses across different verticals. To know how our innovative solutions and Intellectual Property can help your organization, schedule a discussion with us.