Bugs are big problems for development teams. They’re annoying. They pop up at the worst time. And they’re expensive, particularly when they’re found late in the development process or after the application has already been released. In this chapter, we‘ll discuss how unit tests can help find bugs as part of continuous integration (CI).
Why are regression bugs so important to prevent?
First, let’s get definitions straight: A bug is classified as a regression if it is a new bug that changes the behavior of the system negatively. A new feature bug exists entirely within a new feature.
By now, you might be thinking of a company or product you use that has impacted you through a bug introduced during an upgrade, eroding trust between you and the vendor. For business-to-business relationships, this can be enough to cause financial penalties in the contract, or even mean that the client looks around for a new vendor.
These are the bugs that convert to severity zero/one customer issues very quickly. When a customer can no longer do their job, they will rightly expect your development and support teams to be sweating while trying to fix their issues.
If regressions are so important, why aren’t they tested for more often?
Simple answer: It’s boring! By the third release, testing the same feature manually rarely holds much excitement for the engineer doing the testing. Management is only interested if a serious bug is found, and even then, it is an inconvenience to the timetable.
Therefore, most testers will spend the majority of their time working on testing new features. This is the fun and exciting area of working in a QA team: Being able to shape the product before anyone else.
What’s the solution?
Automation, automation and a little more automation. Finding regressions is exactly the task that automated testing does well. To stop the pain caused by these bugs, the answer is simple: ensure that your regression coverage is adequate. It’s not always reasonable to expect QA/QE to find regression bugs, so it’s time for developers to make the time to do more automated testing.
But how? And where in the pipeline?
The key to regressions and continuous integration is to do as much as possible, as early as possible. But that can be overwhelming for developers, who are usually creating code and tests during the earliest phases of the pipeline.
For organizations following DevOps practices, introducing automated regression tests in the first phases of software development shortens the time to delivery by showing the differences between builds right after new commits are made.
These tests are designed to run quickly and perform basic logical validation at the unit level. Without these tests, basic errors advance into the mainline and risk breaking it, and it takes a lot longer to find and fix those bugs later on. This is why, to have a truly continuous CI pipeline, using regression tests should be adopted as early as possible.
Check out the rest of the guide:
- Intro: What are the different types of tests?
- Chapter 1: How to write your first unit test
- Chapter 2: How to measure code coverage
- Chapter 3: How to build a complete test suite
- Chapter 4: Mocking in unit tests
- Chapter 5: Finding the time and motivation to unit test
- Chapter 6: Unit testing mistakes to avoid
- Chapter 7: How automated unit tests speed up continuous integration
- Chapter 8: How to deliver on the promises of DevOps
- Chapter 9: Why imperfect tests are better than no tests