DevOps Quality Trends – Around the World


User Experience is quintessential with DevOp’s “continuous everything” and for keeping customer loyalty and acquisition. Customers look for companies that provide the best services and that too at effective costs.

With the increasing customer expectations and ever-changing business paradigms, it is tough for organizations to keep them loyal to them.

So, how can one really keep up with the competition and their customers all along providing quality services. Well, implementing the best development practices is the first step to delivering quality services and automation is the second. Automation when handled properly along with quality metrics during development-testing-deployment phases can work wonders to deliver faster results.

Also, according to a study by Forrester on the importance of DevOps Quality Metrics the findings shows global usage and valuation of the metrics.

Global Results

3 Key Outcomes:

  • DevOps experts modifies quality metrics based on organization’s perception about the risk factors
  • DevOps experts focus mainly on contextual metrics while others focus on counting metrics
  • DevOps experts are more likely to measure the user experience across an end-to-end transaction while others rely on application-specific or team-specific metrics

All in all, the findings show that one can not base DevOps entirely on just automation but also on the realization and alteration of business risks in order for faster market release.

Europe DevOps Quality Metrics Trends

Let’s look at the top “20” changes in the Europe region are as follows:


Stress on Measuring Quality Metrics 

European organization members recorded the measuring of DevOps quality metrics at a higher level i.e. around 6% higher than the global average. Other metrics such as time, coverage, risk, effectiveness, and efficiency were recorded to be higher than 14%. Hence, this shows that European organizations’ lays more emphasis on metrics relates to time and resource utilization.

Risk and Coverage Metrics

European organization members also recorded the risk and coverage metrics 21% higher than the global average. As most of the respondents were from the Financial and Insurance sector along with Healthcare and Government sector, their focus was more on risk analyzation. Hence, this shows that they value the corporate brand value in comparison to other factors.

Test Data Preparation Time

In comparison to their global peers, European respondents recorded measuring (+18%) and highly-value (+23) Test Data Preparation Time. After the alleged test data restrictions placed by GDPR in May 2018, a change was seen in the management process of the European organizations. Not only that, they even monitored the reflections of the changes on the overall process carefully.

Asia-Pacific DevOps Quality Metrics Trends

Now let’s look at the top 20 Quality Metric Trends in the Asia-Pacific region:

Importance to end-to-end testing metrics

Measurement of end-to-end metrics was recorded higher among the Asia-pacific respondents than their global peers. Automated Tests were 47% and highly-valued by 84% along with Risk Coverage measurement was around 49% and highly-valued by 71%. This concludes that this region focuses more on delivering exceptional user experience as well as digital transformation.

Importance to API testing metrics 

Similarly, Asia-Pacific recorded more API testing measurements globally I.e. around 16% (where API Test Coverage was 63% vs 99% globally and API Risk Coverage was 79% vs 62% globally). The reason behind this inclination is their greater focus on the Financial and Insurance sector.

Higher Quality Metrics Measurement Gap

There is a huge evident gap between the DevOps leaders and laggards in the Asia-Pacific region. The study shows that the overall average of the leaders in this region was lower than their global peers however they did measure the quality metrics at a similar level. Talking about the laggards of the Asia-Pacific, their measurement of quality metrics was quite lower globally. Hence, it can be said that only certain organizations are really concentrating on incorporating DevOps in their development cycle whereas the laggards have a lot of learning up to do.

To learn more about the study’s findings, check out the complete 55-page report on the Tricentis web site: Forrester Research on DevOps Quality Metrics that Matter: 75 Common Metrics—Ranked by Industry Experts.


Supportscreen tag