When testing software, documentation is exceedingly important. It helps structure the process and better manage each stage. There are two particularly popular test documents that are often confused with each other: test plan and test strategy. In this article, we want to clarify what they are, how they differ, and how to create them.
Test plan, test strategy and how they are related
Both the test plan and test strategy are technical documents that facilitate and improve software testing but on different levels. And as there is confusion about these concepts, let’s dive into both of them, exploring their purposes, logic, and nature.
What is a test plan?
The test plan is a project-level document which means that it is focused on a specific software product rather than on procedures and standards adopted across the entire company. It’s created at one of the first steps in the Software Testing Life Cycle (STLC) to outline how exactly the product must be tested along with tools, test environment, schedule, resources, responsibilities, risks, and other aspects.
Software Testing Life Cycle
A test plan is designed by test managers or test leads, who take into account use case documents, software requirement specifications (SRS), and product descriptions. It breaks down the testing process into clear components, helping test engineers and software developers to be on the same page. The plan can change from release to release, given updates in project needs.
What is a test strategy?
The test strategy is an organization-level document that establishes the general test approach — what should be accomplished and how to achieve it. This document lies beyond the scope of STLC and would not specify testing requirements for an exact project. Rather, it sets up the common principles of testing for all the projects in the company. That’s why, unlike the test plan, the strategy does not change frequently if at all.
A test strategy document is usually composed by a project manager or business analyst. Yet, we recommend involving other team members, for example developers or designers. The test strategy is based on the business requirement specification (BRS) document and serves as a guidance to QA engineers, developers, and stakeholders.
How are the test plan and the test strategy related?
When it comes to the hierarchy between the test plan and test strategy, there are two ways they can relate. The main difference could depend on the enterprise's size or on the inner decision. If the test strategy is developed separately, it becomes the key file to follow for all further testing. The document in this case impacts how the test plans will be composed.
Test strategy in relation to the test plan
On the other hand, in some organizations test strategy can be just a part of the test plan document, identifying the test approach for the concrete project. For instance, little enterprises might not run a lot of projects at the same time, and so developing the test strategy as a separate document is not efficient resource-wise.
Test plan vs test strategy comparison table
Test strategy document structure
The goal of the test strategy is to set the vector of testing for all the projects in the organization and to govern the way the test plan is unfolds. Test strategy key elements later become detailed actions in the test plan.
We need the test strategy document at the early stages of the product development process, sometimes even before the outlining of specifications or requirements, as it focuses on the high-level description of the process.
There is no single standard of how the test strategy must be structured so you have quite a lot of freedom here. But commonly it highlights
- purposes, phases, and scope of testing;
- critical success factors;
- general test approach and activities;
- main roles and responsibilities;
- business concerns and key risks; and
- other aspects.
An effective test strategy may take no more than one page and one hour to create.
Test strategy one-page template
Test plan document structure
The main goal of the test plan document is to describe in detail how the testing will be done for a specific product.
Introduction/Product analysis
The testing of the software is hardly imaginable without understanding what this software is. Any test plan is better to start from the product analysis, which is focused on both business requirements and clients' needs.
Questions to answer in the product analysis section include, but are not limited to, exactly who will be the user of the software product; what the software is for; how this software will work; what the software limitations are.
Performing the analysis, a test manager can use product documentation and interview designers and developers about features and functions. Also, it would be helpful to conduct a product walkthrough from the user’s perspective. As a result, a test manager documents the product analysis which becomes the introduction part of the test plan document.
Objectives and scope
The section describes what is to be tested and why. The examples of objectives (or whys) are checking the quality of existing functionality, testing new features, or ensuring stable work throughout the product life cycle. The scope outlines items to be tested and not to be tested.
The list of the features to be tested comes with references to the requirement specifications documents which include detailed information on the functionality.
The list of features not to be tested describes reasons why a certain item is out of the scope. For example, it is low-risk, has been used before, and shows stable performance. And surely, we do not need to test the feature if it won’t be included in the next release or if it is not a functional one.
Test approach
This section details activities mentioned in the strategy (if there is a separate document) specifying the test level (when to test), its types (what to test), and methods (how to test — manually, automatically, or using a combination of both testing efforts.) The approach is dictated by the above-mentioned list of features to be tested.
Regarding test levels, there are four of them. Unit testing ensures that each software component functions well. Integration testing is responsible for checking whether the product units work correctly together. System testing verifies that software functions well as a whole and meets the technical specifications. And acceptance testing evaluates if the system works properly from a user standpoint. Usually (but not always), you must test the software product at all four levels before the launch.
As for the types of testing, there are two main groups: functional which determines if a piece of software meets functional requirements and nonfunctional which checks everything else — usability, accessibility, security, reliability, performance, etc.
In sum, there are over 100 testing types but we don’t use all of them in one project. A good plan prioritizes types and specifies activities, techniques, and participants for each test level. Say, it may define that the unit test will be executed by the software developer and approved by the test manager. The description must contain enough details to understand the major tasks and what resources you will need to handle them.
Human resources
This part is dedicated to experts required for conducting tests of a particular project. It lists team members, specifying their roles, volume of work, and responsibilities. Below we mention key specialists that drive the testing process.
A test lead or manager not only develops the test plan but also controls the testing team, designs procedures, monitors progress, tracks quality metrics, gathers updates, supervises resources, and prepares reports.
A test administrator is responsible for the creation of the test environment and its supervision.
A tester or quality assurance (QA) engineer actually performs tests at different levels, logs results, and reports bugs. They also communicate with developers to ensure that issues are fixed. The required background for those experts depends on the approach. If automated testing is used, QA engineers also write scripts and deploy automation tools for repetitive tasks.
Note that actual job titles and the scope of responsibilities differ greatly from project to project. Learn more about QA team roles from our article on QA engineering roles.
Test environment
A test environment is a combination of hardware, software, network configurations, and data required to execute a particular test. It must replicate or closely resemble the production environment to ensure that the code will behave the same way in real life.
Considering the variety of devices, versions of operating systems, and browsers, you may need multiple test environments to support several deployment scenarios. The best practice is to test features against real gadgets (smartphones, tablets, etc.) instead of using emulators and simulators. Also different environments are used for different types of testing (performance, security, durability, etc.)
Testing tools
The section contains a list of manual and automated testing tools that will support QA activities during the project. Here are some popular groups of instruments used in the testing process.
- Test management tools for test supervising, scheduling, defect logging, tracking, and analysis. Examples: PractiTest and SpiraTest.
- Bug tracking tools to help QA engineers spot, fix, and report issues. Examples: Jira Bugzilla, BugNET.
- Automated testing tools for reducing manual effort. They often operate across different platforms and browsers, allow for writing scripts in any programming language or even enable you to create and execute automated tests without coding experience. Examples: Selenium, LambdaTest.
- Performance testing tools to find and eliminate performance bottlenecks. Examples: NeoLoad and StormForge.
- Incident management tools check and control incidents in the testing process. Examples: HaloITSM and OnPage.
Of course, this is by far not the entire list of existing tools that facilitate testing operations. It’s important to select the right set of instruments covering each specific task instead of relying on one-size-fits-all solutions.
Risks
The section emphasizes possible risks associated with delays, shortage of certain resources, project changes, etc. It also offers solutions for more or less predictable cases. In the table below, we describe common risks and ways to mitigate them.
Types of risks and examples of mitigations
Test schedule
The schedule establishes test milestones, their duration, and resources and staff needed to perform tasks at each stage. The detailed schedule with start and end dates for every milestone task helps ensure that the process proceeds as planned and deadlines will be met.
Test deliverables
The deliverables section contains a list of tools, documents, and other things that should be delivered throughout the Software Testing Life Cycle — before the actual testing starts, during the process, and after it is finished.
Before testing deliverables, include a test strategy, test plan as well as test scenarios and test cases. A test scenario is high-level information about what should be tested while a test case develops the scenario, describing exact testing inputs, execution conditions, procedures, and expected results.
Examples of deliverables generated during testing are automation scripts, bug reports that are written once the error is spotted, and the requirement traceability matrix — a document detailing technical requirements for a given test scenario and verifying that they have been fulfilled.
Key deliverables created after testing completion are incident reports, test summary reports to overview all test activities, and release notes that detail changes made to the latest version of the software and sent to the clients, customers, or stakeholders.
All deliverables listed in the plan should be accompanied by due dates and names of employees responsible for them.
Test exit
By planning exit, a test lead decides when exactly is that moment the product testing is completed. For example, the process can be stopped if 100 percent of the requirements are met. Another criterion is 95 percent of test coverage, with only five percent of test cases failed, all of them of low priority.
Test exit parameters can also refer to deadlines and budget limitations. If you run out of time and money, activities will be stopped — no matter whether goals are achieved or not.
Test plan and test strategy templates and best practices
Neither the test strategy, nor the test plan document has a strict structure everybody must follow. In this article, we only mentioned information they should include to raise the chances of software working as intended. Here are some practical tips.
Make documents simple and easy to read. The test plan and test strategy are the guidelines for different team members. Keep those documents simple and straightforward, avoid long paragraphs, and use common words instead of terms wherever you can.
Use tables and lists. They help represent information in a more digestible, visual way.
Select document forms that are comfortable for the team. Mostly, the test plan and test strategy are present in the text files, but it should not limit the test manager. Both documents can be created in the form of a mind map or Wiki page.
Take advantage of test plan and test strategy templates. Fortunately, you don’t need to write your test strategy and plan from the ground up. There are a lot of templates to choose from, so you can just download and customize the one that fits your test project best. Some examples are a test plan outline based on the IEEE 829-2008 Standard for Software and System Test Documentation, and a test strategy document template composed by Loyola University Chicago. You can find a collection of test plan documents on the TemplateLAB website or use a predefined test plan template created by IBM.