Test Plan
Test Plan Definition
By ISTQB glossary the definition of a test strategy is – a document describing the scope, approach, resources and schedule of intended test activities.
Test Plan is a dynamic document. The success of a testing project depends upon a well-written Test Plan document that is current at all times. Test Plan is more or less like a blueprint of how the testing activity is going to take place in a project.
The process of writing a test strategy is more about thinking about risk factors within the project and planning to mitigate those risks, rather than ticking boxes to show that all types of testing have been included.
Test Planning At SDLC Phases
Initiate
QA team should get involved while the scope of the project is gathered from the customer/client in the form of business requirements. But in the real world, that is not the case. From a practical point of view, the involvement of the QA team is NIL. At the end of this phase, BRD is finalized and a basic Project Plan is created.
Define
SRS is created from the BRD. Test plan's initial draft is created. At this point, since the QA team is not done with the SRS review, the scope of testing is not clear. So, the TP at this phase will only contain information on when testing is going to happen, project information and the team information (if we have it).
Design
information on what to test and a good estimate of how many test cases we might get etc. A second version of the Test plan is created incorporating all this information.
Components Of A Plan Document
Scope
Test Scenarios/Test objectives that will be validated.
Out of scope
Enhanced clarity on what we are not going to cover.
Assumptions
All the conditions that need to hold true for us to be able to proceed successfully.
Schedules
- Test Scenario prep
- Test Documentation- test cases/test data/setting up environment
- Test Cycle - how many cycles
- Start and End date for cycles
Roles and Responsibilities
- Team members are listed
- Who is to do what?
- Module owners are listed and their contact info
Deliverables
- What documents (test artifacts) are going to produce at what time frames?
- What can be expected from each document?
Environment
- What kind of environment requirements exist?
- Who is going to be in charge?
- What to do in case of problems?
Tools
- For Example, JIRA for bug tracking
- Login
- How to use JIRA?
Defect Management
- Who are we going to report the defects to?
- How are we going to report?
- What is expected- do we provide screenshot?
Risks and Risk Management
- Risks are listed
- Risks are analyzed- likelihood and impact is documented
- Risk mitigation plans are drawn
Exit criteria
When to stop testing?
Test Planning
Planning a test is the core important section in the testing cycle. The outcome of the testing phase will be determined by the quality and scope of the planning that has been done for the testing.
Planning the test usually occurs during the development phase in order to save the lead time for test execution upon mutual agreement from all the parties involved.
Major Components in the Planning Phase
Given below are the major components that are involved in the planning phase.
Test Strategy: This is one of the most important sections that can explain the strategy that will be used while testing.
Test Coverage: This is essentially required and it will do conformance mapping of the business needs and the test cases so that one can ensure if the entire software has been tested or not.
Test Cycles and Durations: This can become very critical depending upon the rounds of development and their time for completing each round.
Pass/Fail Criteria: It is very much required one in which the pass and fail criteria are defined. A few times this will also be defined by the clients.
Business and Technical Requirements: Need to have the software and the purposes they serve will be clearly defined along with the low-level explanations.
Limitations
There are few things that can actually control the software testing phase especially the
planning phase.
Following are such few areas:
Features to be and not to be tested: This will clearly point out what has to be tested and what should not be.
Suspension Criteria and Resumption Requirements: This is the decision-maker on the software developed and the criteria defined in order to suspend the testing or resume the testing.
Responsibilities: A tester will have multiple responsibilities in ensuring the issues, bugs, and defects in the software under test. Additionally, the bugs have to be validated with the developers for them to fix.
Risks and Contingencies: Risks associated during the testing should be clearly mentioned and proper contingencies during the time have to be defined very clearly.
Test Execution Plan
The execution of test cases is one of the steps in the STLC phase. This will have to be performed in accordance with the plans that were worked out earlier. Hence, planning always keeps dominating the whole of the testing phase. Below is an example where the testing team gets impacted by the changes in the testing plans.
Observations:
- The testing plan will determine the test case execution.
- The execution part varies as per the plan.
- As long as the plan and the requirements are valid the test cases are valid too.
Ways to Overcome Problems while Execution
Testers will more often come across various scenarios while they perform the test execution. This is when the testers will have to understand and know the ways to resolve the problem or at least find a workaround for the issue.
Version Controlling and Management
Version Controlling and management of testing plans and the test cases are really important in order to showcase the timely deliverables. This is being more significant and is often done with the help of a version control tool.
A version control tool not only helps them to control the testing plans but also assists in defects management. When there are testing projects with multiple cycles and releases, these tools can really help a lot in bringing down the metrics for supporting the testing deliverables.
Test Plan & Test Execution Comparison
| Comparison area | Test Strategy | Test |
|---|---|---|
| Person responsible | The test manager will be preparing the Test plan and will be sharing to all the stake holders for their review. | This will be normally done by tester keeping in mind that the test cases prepared has been approved and signed off. |
| Main focus | The Test plan focus areas are how the testing should be carried out, what should be considered and what not to, environment that can be used, Test schedules etc. | The Test execution focuses mainly on the execution of the test cases provided to be tested on the software. |
| Recurring or iterative mode | This is a single time activity. Having said that it may or may not require modifications for the future releases of the software. | There are 3 parts in this area when we talk about iteration. 1. Functional testing. 2. Regression testing. 3. Re-testing. |
| Inputs | The inputs for creation of a test plan is really required and to be provided by business analysts, Architect, clients etc. | The test case document is the major input. |
| Period when it can be started | It has to be started along with the development cycle for effective outcome and to save time. But there are few models like water fall model where in the testing phase will start only after the development phase has been completed. | Execution has to be started strictly after the development of the software has been done. |
| Closure period | The test plan will have no such closure period. Generally, a sign off from all interested parties for the software will be provided. | Execution for a specific release or cycle will be considered to be closed when all of the test cases have been executed against the software. |
| Deliverable positioning | Test plan will be considered as a major deliverable for the testing activity. This will be done as the first step in testing process. | This will be coming as a last bench member in the testing phase. Post execution the defects/bugs status along with the test case execution status will be shared as one of the testing deliverables. |
| Tools usage | There will not be many tools used as the planning activity will be more of discussion and documentation. To keep track of any changes to the plan, the test managers will normally use any version control tool like VSS or something else. | It will depend on the mode of execution. In case of manual no tool will be used for execution. But for logging the defects and managing, some tools will be used. In case of automation testing, the execution will be done with the help of tools like QTP, SELENIUM etc. |
| Impacts on the deliverables | This will impact all of the testing phases in a larger manner. | This will impact the subsequent cycle or release to be tested. |
How to write a Test Plan
You already know that making a Test Plan is the most important task of Test Management Process. Follow the seven steps below to create a test plan as per IEEE 829.
- Analyze the product
- Design the Test Strategy
- Define the Test Objectives
- Define Test Criteria
- Resource Planning
- Plan Test Environment
- Schedule & Estimation
- Determine Test Deliverables
1. Analyze the product
How can you test a product without any information about it? The answer is Impossible. You must learn a product thoroughly before testing it.
You should take a look around this website and also review product documentation. Review of product documentation helps you to understand all the features of the website as well as how to use it. If you are unclear on any items, you might interview customer, developer, designer to get more information.
2. Develop Test Strategy
Test Strategy is a critical step in making a Test Plan in Software Testing. A Test Strategy document, is a high-level document, which is usually developed by Test Manager. This document defines:
- The project’s testing objectives and the means to achieve them
- Determines testing effort and costs
You can check this article to read more about test strategy.
3. Define Test Objective
Test Objective is the overall goal and achievement of the test execution. The objective of the testing is finding as many software defects as possible; ensure that the software under test is bug free before release.
To define the test objectives, you should do 2 following steps:
- List all the software features (functionality, performance, GUI…) which may need to test.
- Define the target or the goal of the test based on above features
You can choose the ‘TOP-DOWN’ method to find the website’s features which may need to test. In this method, you break down the application under test to component and sub-component.
Also, you can create a Mind-Map to find the website features:
- Check that whether website functionality (Account, Deposit…) is working as expected without any error or bugs in real business environment
- Check that the external interface of the website such as UI is working as expected and & meet the customer need
- Verify the usability of the website. Are those functionalities convenient for user or not?
4. Define Test Criteria
Test Criteria is a standard or rule on which a test procedure or test judgment can be based. There’re 2 types of test criteria as following.
Suspension Criteria
Specify the critical suspension criteria for a test. If the suspension criteria are met during testing, the active test cycle will be suspended until the criteria are resolved.
Test Plan Example: If your team members report that there are 40% of test cases failed, you should suspend testing until the development team fixes all the failed cases.
Exit Criteria
It specifies the criteria that denote a successful completion of a test phase. The exit criteria are the targeted results of the test and are necessary before proceeding to the next phase of development. Example: 95% of all critical test cases must pass.
Some methods of defining exit criteria are by specifying a targeted run rate and pass rate.
- Run rate is ratio between number test cases executed/total test cases of test specification. For example, the test specification has total 120 TCs, but the tester only executed 100 TCs, So the run rate is 100/120 = 0.83 (83%)
- Pass rate is ratio between numbers test cases passed / test cases executed. For example, in above 100 TCs executed, there’re 80 TCs that passed, so the pass rate is 80/100 = 0.8 (80%)
This data can be retrieved in Test Metric documents.
- Run rate is mandatory to be 100% unless a clear reason is given.
- Pass rate is dependent on project scope, but achieving high pass rate is a goal.
5. Resource Planning
Resource plan is a detailed summary of all types of resources required to complete project task. Resource could be human, equipment and materials needed to complete a project.
The resource planning is important factor of the test planning because helps in determining the number of resources (employee, equipment…) to be used for the project. Therefore, the Test Manager can make the correct schedule & estimation for the project.
Human Resource
The following table represents various members in your project team
| No. | Member | Tasks |
|---|---|---|
| 1 | Test Manager | Manage the whole project Define project directions Acquire appropriate resources |
| 2 | Tester | Identifying and describing appropriate test techniques/tools/automation
architecture Verify and assess the Test Approach Execute the tests, Log results, Report the defects. Tester could be in-sourced or out-sourced members, based on the project budget. For the task which required low skill ,I recommend you choose outsourced members to save project cost. |
| 3 | Developer in Test | Implement the test cases, test program, test suite etc. |
| 4 | Test Administrator | Builds up and ensures Test Environment and assets are managed and
maintained Support Tester to use the test environment for test execution |
| 5 | SQA members | Take in charge of quality assurance Check to confirm whether the testing process is meeting specified requirements |
System Resource
For testing, a web application, you should plan the resources as following tables:
| No. | Resources | Descriptions |
|---|---|---|
| 1 | Server | Install the web application under test This includes a separate web server, database server, and application server if applicable |
| 2 | Test tool | The testing tool is to automate the testing, simulate the user operation, generate
the test results There are tons of test tools you can use for this project such as Selenium, QTP…etc. |
| 3 | Network | You need a Network include LAN and Internet to simulate the real business and user environment |
| 4 | Computer | The PC which users often use to connect the web server |
6. Plan Test Environment
What is the Test Environment?
A testing environment is a setup of software and hardware on which the testing team is going to execute test cases. The test environment consists of real business and user environment, as well as physical environments, such as server, front end running environment.
How to setup the Test Environment
Back to your project, how do you set up test environment for this banking website?
To finish this task, you need a strong cooperation between Test Team and Development Team.
You should ask the developer some questions to understand the web application under test clearly. Here’re some recommended questions. Of course, you can ask the other questions if you need.
- What is the maximum user connection which this website can handle at the same time?
- What are hardware/software requirements to install this website?
- Does the user’s computer need any particular setting to browse the website?
7. Schedule & Estimation
In the Test Estimation phase, suppose you break out the whole project into small tasks and add the estimation for each task as below
| Task | Members | Estimate effort |
|---|---|---|
| Create the test specification | Test Designer | 170 man-hour |
| Perform Test Execution | Tester, Test Administrator | 80 man-hour |
| Test Report | Tester | 10 man-hour |
| Test Delivery | 20 man-hour | |
| Total | 280 man-hour |
Then you create the schedule to complete these tasks.
Making schedule is a common term in project management. By creating a solid schedule in the Test Planning, the Test Manager can use it as tool for monitoring the project progress, control the cost overruns.
To create the project schedule, the Test Manager needs several types of input as below:
Employee and project deadline: The working days, the project deadline, resource availability are the factors which affected to the schedule
Project estimation: Base on the estimation, the Test Manager knows how long it takes to complete the project. So, he can make the appropriate project schedule
Project Risk: Understanding the risk helps Test Manager add enough extra time to the project schedule to deal with the risks
8. Test Deliverables
Test Deliverables is a list of all the documents, tools and other components that has to be developed and maintained in support of the testing effort.
There are different test deliverables at every phase of the software development lifecycle.
Test deliverables are provided before testing phase.
- Test plans document.
- Test cases documents
- Test Design specifications.
Test deliverables are provided during the testing
- Test Scripts
- Simulators.
- Test Data
- Test Traceability Matrix
- Error logs and execution logs.
Test deliverables are provided after the testing cycles is over
- Test Results/reports
- Defect Report
- Installation/ Test procedures guidelines
- Release notes
Conclusion
We realize why software testing and planning are important for the optimal project workflow and how we can execute these plans efficiently. In this article, we discussed the test plan for software testing. We first discussed its definition, objectives, advantages, the method to implement it, and creating a template for it.