Test cases: more of a nuisance?

The Quality Engineering (QE) profession requires foundational knowledge of test cases, including understanding what to test, how to test, identifying and logging bugs, and leveraging automation (among many other skills outlined in this earlier post).

Test cases can be a touchy subject for quality engineers and other capabilities. Traditional guidelines are to document every test case, every step of every test case, and every execution of the test case. Moreover, many developers rely on these test cases to drive their code.

In an ideal world, test cases are created before the developer works on a story. The developer then writes their code, QE tests the story by executing its tests, and the story is closed. QE would then update or maintain test cases and test plans, adding whatever is necessary for regression.

While these are standard QE processes, they often present significant challenges, raising the question: Are test cases causing more harm than good? Let’s explore some common challenges with test cases and potential solutions.

Challenges

Requirements Confusion

Often, QE creates test cases in isolation by reviewing the story's Acceptance Criteria (AC). This means they end up testing against their own interpretation of the requirements. When bugs are found, QE often fails the QA for the story, leading to back-and-forth discussions among product, design, development, and QE. The team then tries to determine how to handle scenarios that QE has uncovered but were not previously considered. Cue the classic phrase: “It’s not a bug because we never talked about it.”

Even if QE initiates these discussions and the team reaches a consensus, it happens too late in the process. Valuable time is lost clarifying requirements, with communication spread across various channels and formats—not an optimal situation.

Another consequence is that test cases may unintentionally become the source of truth for a product’s requirements. While this can work if the test cases are accessible to the entire organization, it’s not often ideal when the story is still in development as it could lead to further confusion and half-implementations.

Massive Test Case Management Effort

With every new story, test cases must be added, updated, or removed. But recall how QE often lacks time for test case management. This usually results in new test cases being added without updating or maintaining existing ones. There’s rarely time for refactoring or maintenance.

he outcome? A growing, unmanageable collection of test cases. Eventually, the team might question whether it’s worth creating test cases at all since they are seldom updated or refactored.

Solutions

Stories as the Source of Truth

One solution is to ensure that stories, not test cases, are the source of truth during a sprint. Invest time and effort upfront to detail stories as thoroughly as possible. This is easily done by running proper backlog refinement and sprint planning meetings.

When discussions about ACs lead to changes, deletions, or additions of requirements, update the story itself. This ensures nothing is missed by the time the story is closed.

Make Time for Test Cases

Another solution is to allocate adequate time for QE to create test cases. This requires more than just individual effort; it’s a team-wide responsibility. Most teams have QE working at full capacity from the start of a sprint to its end. Overloaded sprints with stories consistently carrying over exacerbate the issue.

To address this, teams should identify their story point completion trends and aim for slightly less work per sprint. This approach gives QE the bandwidth to review requirements, create test cases, update existing ones, and engage in meaningful discussions during backlog refinement or sprint planning.

Share Test Cases Early

QE should share test cases as early as possible. Ideally, this happens during backlog refinement. If not, sprint planning, the beginning of the sprint, or even while the developer is working on a story are acceptable alternatives. The worst-case scenario is not sharing test cases at all.

Don't Be a Test Case Stickler

You don’t have to follow traditional test case writing guidelines rigidly. Not every test case needs to be perfectly written or contain every detail. Overly detailed test cases are harder to manage and update. Instead, adopt a test case writing guideline that works for your team. Here's an example of one such guideline.

Summary

With all that said, test cases are absolutely helpful and necessary. They help QE and the team better think through requirements, leading to better coverage and a better product.

Beyond that, they act as the product's documentation for the entire organization. When a feature changes, the team doesn't update their stories. But the test cases get updated!

There are other benefits as well related to planning, coverage, and automation that we'll cover in an upcoming post. But there is value in creating and managing test cases and your team should invest time and energy to make sure they follow the right processes for maximum benefit.