In DevOps, we beat this mantra in our heads…

“Plan, Code, Build, Test, Release, Deploy, Operate, and Monitor.”

Now repeat the process continuously. The goal of this methodology is to release high-quality software, faster. Software testing life cycle (STLC) is a subset of this infinite methodology. STLC in DevOps aims to rid the silos between developers, operations, and testers. It also fosters better communication and collaboration with stakeholders. Testing earlier in the lifecycle and testing often saves time and money. So how do software developers achieve optimal STLC in DevOps?

Establishing Early in the Test Plan

It is vital to establish testing early on in the test plan. In order to operate faster, testing cannot exist solely towards the end of development. “QA should be a part of the whole process because iteration is what creates truly great software,” says Christen Costa, CEO of Gadget Review. Testing must begin as early as the coding staging.

Honestly, we recommend Shifting Left to include testing at the planning stage. Meaning, change the entire team’s mindset to think of how to test their work before coding. Also, pondering how to test their work before committing to the codebase. Plan for testing, building, and testing some more…

Steve Scott, CTO at Spreadsheet Planet describes the methodology, “Continuous Testing (CT) is a comparatively recent notion in software engineering that fits neatly with two of DevOps’ main tenets – process continuity and an ongoing supply of feedback. Continuous Testing is centered on test automation, as well as early and continuous testing (shift left).”

Ravi Parikh, CEO of RoverPass explains their company’s process, “We follow a test-driven development model. This means that we use testing to inform the early stages of development. Rather than build a software and test it afterward, we use test cases to dial in our code and ensure that we avoid roadblocks later on. I’ve found it to majorly boost our productivity as a team.” Developers weren’t fans of this approach initially. Parikh explains, “While our developers weren’t excited about it at first, it has become part of our standard process.”

According to Daivat Dholakia is the Director of Operations at Force by Mojio, “The days of having a single QA stage and then deeming software ‘done’ are over. The old model of QA has been recognized as staggeringly inefficient for both companies and consumers. Without the continuous testing, the single designated QA stage is at risk of being rushed (because everyone is trying to stick to the timeline) or drawn out to the detriment of your timeline (because you have to spend hours untangling issues in the code or design that could have been handled in five minutes if caught immediately).”

With older “waterfall” software development techniques, Quality Assurance would scramble to test code and features towards the end of the life cycle. This created a bottleneck in the workflow. This did not weigh much accountability on the developer if they created faulty code. Instituting unit testing brings the developer’s accountability back on themselves. When performing unit tests, it is designed to test first and code second. Small bugs are caught earlier. Even if they slip past a developer’s local machine, they are caught when trying to commit to the codebase.

This is exactly where you want to catch the majority of bugs because they get costly if found later. Research shows catching bugs earlier can save on average $7,600 if the defect made it to production. In some cases, it could cost exponentially more.

Keith Mint of Minted Empire points out a few real-world examples when continuous testing did not ensue. “Nissan recalled approximately 1 million vehicles owing to a software malfunction in the airbag sensory sensors. Two accidents have been documented as a result of this software malfunction. [Previously], Starbucks has decided to shut [down] around 60% of its locations in the United States and Canada owing to a software breakdown in their POS system. Because they couldn’t complete the purchase, the shop offered free coffee for a [short] while.”

Continuous Testing in Production and Beyond

We’ve talked a bit about testing early and often. However, STLC in DevOps should not stop once the product passes QA. Continuously testing software in production is just as vital!

Shift Right refers to actions (such as testing) later in the software development life cycle. We should continuously test on production to catch bugs. If found, it makes rolling back to previous versions easier and lessens adverse effects. Just because the software and features pass quality assurance, we still must rigorously test the production environment. Why? “Software testing in a test environment is exactly that – a test environment,” explains Sheri Byrne-Haber, an Accessibility Architect at VMware. She elaborates, “It isn’t live data, and nothing stresses a system like use by real users. You can’t copy the production environment into a test environment without lots of security (plus permission from the owners of the data). Even then, stuff like this happens.”

There should be a separation between test data and production data. This is why both datasets should be tested. Byrne-Haber explains, “You can’t mix test data in a production environment because stuff like this happens.” She refers to a weird test article that accidentally went live in the New York Times and has since been retracted. Moreover, there is the fact that users can (and will) create totally unique use cases your team may not have thought of. “There is just so much that your users might be able to do that designers, developers, and product owners never thought of as valid use. But once your users have done it, that data is there forever, and you have to figure out how to make it work.”

This is why ZOZOTOWN tests their production application hourly using our no-code test automation tool. With easy-to-use tools like Autify, anyone on the QA team can handle test automation. No more relying on highly skilled test automation engineers- they can be freed up to focus on building software.

Test Automation is Paramount

Another important aspect of DevOps is automation testing. A time-saving avenue in software testing is shifting towards automation versus manual testing. It helps drastically speed up the delivery of superior software products. Though you should not completely eliminate manual testing, as it has its benefits. Testing with automated tools should make up the majority of test cases, while manual testing should be the minor. However, test automation benefits far outpace those done manually. Those include:

  • Saving money, time, and resources
  • Avoid the redundancy of manual efforts
  • Execute more test cases at scale and improving test coverage

How, and why would you implement automation testing before the traditional QA stage? How you may ask… by making the leap and necessary investments in test automation. According to stats, those software development teams that do not make the investment in automation testing trail in comparison to competitors. That same study revealed that most companies allocate between 10%-49% of their overall QA budgets towards test automation expenditures. The aforementioned points justify the return on investment.

Test automation isn’t cheap, or easy for that matter but it’s a necessity for modern development. According to a survey, 71% of respondents search for new automation testing tools several times per year. One of the top complaints is maintenance at scale. Solving this issue solves the inflated man-hour problem in quality assurance.

Furthermore, investing in test automation leads to better software, which in turn, leads to happier customers. The study previously mentioned also revealed that faster release cycles also lead to happier customers. And happier customers retain longer thus accelerating revenue growth.

Communication & Collaboration with All Stakeholders are Key

Effective communication is the key to any successful relationship. Breaking the silos from developers, testers, and operations is vital. This includes everyone in the pipeline- from product managers to project managers, through developers and even manual testers. Each team role must be on the same page regarding testing objectives.

Feedback is important for successful software.

Darshan Somashekar, Founder & CEO of Solitaired summarizes, “The inclusion of CT in the SDLC means that the development team receives early feedback on any issues the code may cause with existing functionalities. CT delivers frequent actionable input at each step of development, assisting in the rapid deployment of software applications into production with a low defect count. This early feedback also aids in the analysis of business risk coverage and accelerates time to market.”

What Next?

The key to shifting to modern DevOps is testing. Testing early, testing often, testing through every development stage, testing production and beyond…

That stated, we recommend stakeholders implement testing in design discussions. Do not isolate testers towards the end of the development cycle. Testing should be the core part of the test plan. Hence why communication with all team members is vital. Test automation done right can help propel agile teams. If your team is in search of a next-level test automation tool, try Autify.

It ultimately saves time, money, and produces superior software products. Great software is consumed (and reviewed) by customers. Making them happy retains them- word of mouth organically grows reach. All of this is connected to revenue and relevance for software makers.