Writing easy to follow test case titles
There are countless ways a test case title can be written. It’s very difficult to see the same format or words used across test cases. One of the things we wanted to figure out was how can we write titles that are not too wordy, convey what the test case is doing, and how can we standardize it. The way I posed the question was – “If someone was to ask whether I covered a specific test case, I should be able to give them an answer quickly and easily just by looking at the test case titles.” Quickly and easily are the key words here.
Example – Sign In Page
Let’s take a sign in page for example. A test case to cover successful sign in can be written in many ways:
- User should be signed in successfully after entering valid credentials and pressing “Sign In”
- Successfully sign in with valid credentials
- After entering valid credentials user should successfully sign in
- As a user, I should be able to sign in with valid credentials
Can we infer from this what the test case is covering? Of course we can. But it’s not clean, the test cases are too wordy, and if I wasn’t the one who wrote them I’d have a hard time finding what I’m looking for especially as the cases become more complex.
Trial 1
We thought about the key information a test case contains and looked at different approaches we were using to writing titles. Sometimes when we begin coming up with test cases, we first map them in a Google Sheet by using point form. It was easy to see at a glance what we were testing and under what conditions. We thought to come up with a format for test case titles that mimics this.
Here’s what a set of test cases using point form would look like:
- Valid username, valid password > sign in – Successful sign in, main screen shown
- Valid username, invalid password – Sign in fail, shown error “Failed to sign in, wrong username and/or password”
- Invalid username, valid password – Sign in fail, shown error “Failed to sign in, wrong username and/or password”
- ….
It’s quite easy to tell quickly what each test case is covering. So we thought to adapt this format officially in the following way by setting some standards.
[Preconditions/Parameters] : [Required actions] – [Expected Results][Parameter 1], [Parameter 2], … : [Action 1] > [Action 2] > … – [Expected Result 1], [Expected Result 2], …
So for example, the sign test case would look like:
Valid username, valid password : Sign in – Successful sign in, shown main screen
Trial 2
When we started writing test cases in this format, we found it a lot easier to map scenarios in our head and convey them. However, things got a little out of control with complex scenarios. We’d have test cases that were even harder to read because of the large amount of actions in the title. We then added a new guideline:
Only add the actions that are relevant to what the test case is trying to cover.
What we basically meant was don’t add every single action possible. We don’t need to map every step on every screen or flow. Some of us took the [Actions] part too literally. As an example, for a VPN app on iOS, you have to install a VPN profile in order to control the VPN functionality. The profile installation step was part of the sign in flow. So the sign in test case was written as:
Valid username, valid password : Sign in > Install Profile > Done – Signed in successfully, shown main screen
Trial 3
So up to now, we have a standard format for writing test cases, and have outlined guidelines for the actions. Test cases were much cleaner now and easier to read. However, the same problem began surfacing for the parameters and expected results. The solution was to adapt the same guideline as for the actions in terms of keeping only what’s necessary. We also added another guideline to shorten the test case titles and keep them clean.
Parameters can be combined when possible by rewording.
Expected results do not need to be added when the scenario is obvious. Further, if the number of expected results is higher than 2, omit the results from the title and add them within the Expected Result section of the test case.
Continuing with our sign in example, it would be re-written in the following way:
Valid credentials : Sign in
Summary
[Preconditions/Parameters] : [Required actions] – [Expected Results]
Guidelines:
- Only add the parameters, actions, and expected results relevant to what the test is covering.
- Parameters can be combined when possible.
- Expected results do not need to be added when the scenario is obvious.
- If the number of expected results is higher than 2, omit the results from the title and add them within the Expected Result section of the test case.
This format combined with our other efforts for organizing test suites and cases made for an efficient and clear process. It also helped immensely with test case reviews as we didn’t need to fumble through the wording to understand the scenario. We could see at a glance what each section was testing and the coverage.
This does not mean all problems are solved or this format is perfect for everyone. There is still a personal touch to each test case depending on the person writing them. The level of depth for which parameters, actions, and expected results to include in the title differs between each member. The idea is that we have a set of guidelines that makes certain tasks easier. We can also refine as part of the test case review. As a matter of fact, we did an exercise where we laid out a test case and asked each member to come up with the way they’d write it based on the format and guidelines. Out of the 7 members, not a single member wrote the test case exactly the same with the exact same words. The best part, that’s ok! We were all able to understand the test cases easily and could quickly point out the coverage or scenario. The other point is there is an expectation that members are familiar with the applications. New hires would need more hand-holding from the beginning but that’s not a problem. A one time investment instead of countless hours of re-writing test cases, re-wording them, and managing them.