Test Plan

Author: Matthew Cunliffe

Date: 14th February 2013

Version 1.0

http://www.euro-point.co.uk

Contents

1. Document Control	4

1.1 Document History	4

1.2 Document Distribution	4

1.3 References	4

2. IntroductIon	6

2.1 Overview	6

2.2 Scope	6

2.3 Assumptions	7

2.4 Definitions, Acronyms and Abbreviations	7

2.5 Related Documents	7

3. Test Items	8

3.1 Introduction	8

3.2 Applications in Scope	8

3.3 Changes in Scope	8

3.4 Items out of Scope	8

4. approach	9

4.1 Introduction	9

4.2 Test Approach	9

4.3 Key Activities	9

4.4 Test Stages	9

4.5 Test Tools and Techniques	10

4.6 Metrics	10

4.7 Reporting	11

4.8 Test Entry and Exit Criteria	11

4.9 Test Pass and Fail Criteria	11

4.10 Test Execution Suspension Criteria and Resumption Requirements	11

5. Test Deliverables	13

6. Test Environments	14

6.1 Introduction	14

6.2 {Test Environment}	14

6.3 Performance and Volume Testing Environment	14

6.4 Additional Environmental Requirements	14

7. Roles and Responsibilities	15

7.1 Introduction	15

7.2 Test Manager	15

7.3 Test Analysts	15

7.4 IT Support	15

7.5 Business Support	15

7.6 Third Party Support	15

8. Skills	17

8.1 Introduction	17

8.2 Skills Required	17

9. Schedule	18

9.1 Test Milestones	18

10. Risks and Contingencies	19

Document Control

Document History

Date	Version	Details	Author
dd/mm/yyyy

Document Distribution

Title	Person	Role
Project Manager	Approver
Test Manager	Accountable & Responsible
Business Lead	Approver
Business Analyst	Consulted
Requirements Lead	Consulted
Build Manager	Consulted
Architect Lead	Consulted
Release Manager	Consulted
Project Team	BA Team Test Team Config Management Team	Informed

References

The following documents were used in creation of this document:

Ref	Document Title	Version	Location
1	Product Description for Test Plan
2	Release Management Plan
3	Defect Management Approach
4	Document & Configuration Management
5	Change Control Process
6	High Level Scope Document
7	Quality Assurance Process
8	Risk Register
10	{test phase} Approach
11	Regression Test Approach
12	Glossary

IntroductIon

Overview

This document is the {Test Phase} Test Plan for Release {xxx} of the {project} system. It defines the approach that this project will adopt to ensure that the delivered components for Release Error! Unknown document property name. are functionally tested and fit for purpose, and to confirm that there is functional integrity across the impacted {project} application. Non-functional testing will be also carried out as part of {Test Phase} Testing, and is covered by a separate Non-functional Test Plan.

The document identifies all entry and exit criteria used throughout this testing phase and defines all the activities used for ensuring software quality and how these activities interface with the testing process.

This document builds upon the details and constraints outlined in the {project} Test Strategy (document reference {nnn}-{project}-TST-STR-001) referenced in section {xxx} of the said document.

Scope

The functionality for the {project} project is to be delivered over a number of releases, some of which are to be delivered concurrently. It is subject to several milestones as defined in {project} plan. This document refers to the {Test Phase} testing of all {project} functionality which is to be delivered in Release Error! Unknown document property name.. The non-functional testing is outside the scope of this document. The following table shows the changes delivered in this release against the milestone and the functional areas to which it applies.

Change	Milestone	Functional Area

Assumptions

The following are assumed:

All Functional and Technical Specifications relating to this release have been completed and signed off;

All resources, both external and internal, are available as and when required;

Test Environments are available as and when required;

Test data is available;

Test stubs, drivers and harnesses are available.

Definitions, Acronyms and Abbreviations

The glossary contains all definitions, acronyms and abbreviations.

Related Documents

The following documents are referenced. Where a column is blank, the information is not currently available:

Document Reference	Description	Version	Issue Date

Test Items

Introduction

{Test Phase} Testing combines Functional, Interface, End to End, Sanity and Regression testing in one test phase. In this test phase, both the individual {nnn} subsystems and the internal interfaces between those subsystems are tested. The {Test Phase} will also verify and validate the external interface between {nnn} and the {project} Service Provider.

Applications in Scope

The {project} is maintained using a number of bespoke and tailored packages. The following applications are modified and enhanced by the changes in this release.

{header}

Core Text

Changes in Scope

The following functional changes incorporated into Release Error! Unknown document property name. are addressed in this document, and are considered in scope for the Release Error! Unknown document property name. test phase. A brief outline of the functionality of each change is detailed.

{identifier} – {description}

This change pertains to the ….

Items out of Scope

The following areas are considered to be out of scope for the {Test Phase} testing of {project} deliveries for this test plan:

changes not included within Release Error! Unknown document property name.;

Integration testing of Business As Usual deliveries;

Non-functional testing, such as:

Performance

Security

Capacity

Unit and Link Testing ( a prerequisite to this test phase);

User / Business Acceptance Testing;

Ready for Service Testing.

Approach

Introduction

In order to ensure test coverage and facilitate the management of test execution, a number of key activities will need to take place before and during the test execution process.

Test Approach

Test Cases will be prioritised according to risk and by change, with those cases identified as higher risk being run at the start of each test phase for each release. The objective will be to complete all planned tests for each release; however, should this not be feasible within the timescales for each release, those test cases identified as lower risk (and therefore lower priority) will be executed in a subsequent release in order to attain the target of 100% test coverage for each release. Those test cases executed in a subsequent release will not affect the delivery of software to {project} milestones.

For each release there is a requirement to carry out {project} specific regression testing on software delivered in previous releases. The intention is not to run all test cases cumulatively for each release but to select regression candidates as an iterative process thus ensuring that upon the final release there exists a comprehensive suite of regression tests available for all releases. The identification of the regression candidates will be carried out by the Test Analysts.

The test effort during test execution will steadily increase over each release to incorporate the additional regression candidates.

Key Activities

The current {project} Project Plan states that there will be xx releases of software overall. This requires that the test execution process is treated as a ‘time-boxed’ activity, in that the window of testing between each release is limited to one month. Within this period a number of activities must be carried out, as defined below in section 4.4 - Test Stages.

Test Stages

Sanity Tests

The number of sanity test cases executed for each release will remain constant as sanity testing is focused upon verifying the integration and installation of the software release on the given test environment using a predefined suite of tests.

Functional Tests

For each release there will be an identified number of test cases in order to test the functionality of each change included in that release. The total number of test cases in each release will be dependent upon the complexity of the items within that release.

Interface Tests

The interfaces between the {nnn} system and the {project} Service Provider will be tested to ensure that connectivity, the connection protocol and the data format function as specified. It is envisaged that interface testing will be required where functionality is introduced that requires a connection to an external provider.

End To End Tests

End-to-end testing of the {nnn} system from {project} Service Provider through to the contravention and enforcement systems is carried out as and when the related software components and interfaces become available. Test harnesses and test stubs will be used to simulate undelivered components. A full, non-simulated end-to-end test of the system will take place once all parts, including those provided by the {project} Service Provider, are available.

Regression Tests

The number of {project} regression test cases will increase for each release as further regression cases are identified from previous releases. In each case, regression cases will be identified according to risk based assessment: that is, where an area of functionality is utilised heavily, is modified frequently, or likely to be frequently affected by functionality changes, then the development of regression test specifications for this area is of a higher priority. Due to time and cost restraints, both in development and test time, regression will not provide 100% coverage of the {nnn} system.

Test Tools and Techniques

Testing of Release Error! Unknown document property name. will be carried out based on analysis of the functional specification: however, where necessary, reference may be made to the internal workings in order to clarify detail. Interface testing will incorporate a level of white box testing in that the format of the interface data mechanisms will need to be validated and verified.

Where required, test data will be obtained or generated using SQL, PL/SQL, Access, TOAD, or a combination of these tools. Drivers, stubs and test harnesses will be utilised when required. Test data required for the testing of interfaces between {project} functionality and third party providers, such as the {project} Service Provider, will be either created in-house using the above methods, or will be identified and requested from the third party provider.

Should boundary value analysis be required (such as in the case of date- or value-based triggers for actions), then this will be carried out to ensure a valid spread test across such boundaries. Equivalence partitioning will be carried out in these cases to further enhance the test coverage.

Cause-Effect graphing may be necessary in some cases to ensure coverage of all possible branches stated within the requirements.

Where possible, sanity tests will be developed using the {Quick Test Professional} suite. Once a release has been fully tested, that is 100% test coverage has been achieved, automation of regression for that release will commence. Where sanity or regression tests have not, or cannot be automated, manual execution will be carried out.

Test incidents will be raised and stored within {Quality Center}.

All testware will also be stored in a secure repository.

Metrics

Test metrics will be recorded daily during the {project} project duration. These metrics will encompass the estimation, scripting, and review of test documentation; the test, reiteration and results of testing; and the raising, resolution and retest of faults.

Reporting

Progress reports will be made daily to the Test Manager for each change within the release. This information will be disseminated to the relevant parties as specified in section 5.7.1 of the Test Strategy.

Test Entry and Exit Criteria

The following entry and exit criteria relate to the completion of all testing in relation to this test plan.

Entry Criteria

All release components have been successfully Unit/Link tested;

This test plan has been signed off;

All test specifications have been formally reviewed and approved;

The required test environment is available;

The release has been loaded on to test environment;

The required test data is available.

Exit Criteria

All planned testing has been completed;

There are no outstanding problems with a Business Severity of 1, 2 or 3; or

Any outstanding problems with a Business Severity of 1, 2 or 3 have been resolved in agreement with Transport for London’s representatives;

Test Execution Logs have been produced for all changes;

A Test Completion Report, signed off by the Test and Integration Manager, has been issued.

Test Pass and Fail Criteria

The following criteria must be met in order for the functionality provided by an change to be deemed to have passed:

For each test condition within a test case, the actual result must meet the expected result of the test;

The delivered functionality conforms to the specification;

The delivered functionality does not have an adverse effect on any other part of the application;

All test cases for the change have been executed.

Should any functionality fail to meet the pass criteria, a fault will be raised and the severity and impact of the failure identified.

Test Execution Suspension Criteria and Resumption Requirements

The following events will cause testing to be suspended:

Failure of any component of the environment which results in a loss of service to the Test Team;

Disabling of any component of the environment for planned maintenance or enhancement;

A fault is identified that prohibits further execution of all dependent test conditions.

Tests execution will be resumed once:

The test environment is made available;

The fault is resolved to such an extent that execution of all dependent test conditions can be continued;

Any other issues which prevented execution from continuing are resolved.

Test Deliverables

The following documentation will be delivered as part of the {Test Phase} test process for Release Error! Unknown document property name.:

Test Plan (this document);

Test Specifications for each change;

Test Execution Logs for each change, following completion of test execution;

Test Completion Report for Release Error! Unknown document property name., following completion of all test execution.

Test Environments

Introduction

There are currently {xxx} test environments, all of which will be utilised to support {Test Phase} testing of the {project} functionality. These are documented in detail in the Test Strategy document.

{Test Environment}

This environment is used to carry out sanity, functional and regression testing of {project} releases, as well as future date testing, as the system time can be advanced on the servers in this environment where specific boundary analysis around key dates is required.

Performance and Volume Testing Environment

This environment mirrors the Production environment in terms of hardware, infrastructure and security. This is the primary environment which will be utilised for performance, interface and end to end testing.

Additional Environmental Requirements

Client access to the servers will be via Windows-based desktop machines.

Roles and Responsibilities

Introduction

During the test cycle, a number of resources will be required to provide input into the test process. The resources are identified below, together with any responsibilities, whether direct or indirect, to the test cycle.

Test Manager

The Test Manager is responsible for the overall management of the test process. Metrics will be collated daily by the Test Manager for analysis and distribution.

Test Analysts

The Test Analysts will be responsible for the development and delivery of all test documentation; the execution of all test specifications; and the raising of any faults found.

Automated testing, including Sanity, Regression and Performance will also be carried out by the Test Analysts.

The Test Analysts will also be responsible for the development, where required, of all test stubs, drivers or harnesses, unless supplied by a Third Party.

IT Support

The following resource will be required to support the test process:

Technical Support Analysts: The Technical Support Analysts are responsible for the management and maintenance of all test environments and associated internal networks. They are also responsible for installation of all released software;

Database Analysts: The DBAs are responsible for the management and maintenance of all databases on the test environments.

Business Support

Business Analysts: The Business Analysts are responsible for the development of all functional specifications required for the delivery of Release Error! Unknown document property name..

They will be responsible for the management of the business acceptance and subsequent sign-off by the Customer of each change within each release.

The Business Analysts will be required to provide functional and/or technical specifications and answer solution specific questions on an ad hoc basis. They will also provide support in reviewing test specifications to ensure that coverage of business requirements has been achieved.

Third Party Support

As the application interfaces with Third Party suppliers and contractors, it will be necessary to call upon resource as and when required. It is envisaged that the following parties, at a minimum will be required:

{Third Party}: {Third Party} are responsible for the development of the {application} and will be required to provide in depth information regarding the implementation of the functional specifications. They will also provide in fault resolution, where required.

Skills

Introduction

In order for the test process to be efficacious, there is a requirement for the test team to be sufficiently skilled in a number of areas.

Skills Required

All test analysts require a solid understanding of the applications under test, together with the new functionality being introduced;

A good knowledge of SQL and PL/SQL is also required for interrogation of databases;

Understanding of UNIX shell constructs, Windows Services, Message Queues and Active Directory is required.

A good working knowledge of Test Scripting Language (TSL) and the HP suite of automated test tools (including Quick Test Professional and Loadrunner) is required.

Schedule

Test Milestones

The table below outlines the schedule required to support testing of Release Error! Unknown document property name.. NOTE: At present, estimation of the start and end dates are aligned with the estimated start and end dates of the corresponding change functional specifications outlined in the {project} project plan. These indicative Test Milestone dates are subject to change once these specifications are available:

Process	Final Specification Available	Documentation Start Date	Documentation End Date	Go Live Date
Release Error! Unknown document property name. Test Plan (this document)	N/A	01/04/2005	01/11/2005	N/A
Release Error! Unknown document property name. Test Plan Approved (this document)	N/A	01/11/2005	N/A

Risks and Contingencies

The table below identifies the known risks which may adversely affect the ability of the Service Provider to adequately test the {project} changes within the revised {nnn} application. Each risk has a proposed mitigation action which could reduce or eliminate the risk. These risks apply generally to all areas of testing.

Key	Risk	Mitigating Action	Risk Register Reference
2	{project} Service Provider external interface not available when test execution begins.	Develop external interface stub and driver.
2	Test data not available from Service Provider.	Develop test data simulator.
3	Timescales are not adhered to and therefore the testing schedule outlined is not effective.	Once cause of slippage is established either additional resource may be deployed or timescales may be increased through discussion with Client and {project} Project Management.
3	The {project} Service Provider’s test environment and / or connectivity is not sufficiently scaled or robust to cater for the required level of testing.	Create test stub.
3	Changes to requirements are not updated in functional and technical specifications and therefore not translated into testing.	All changes to functional and technical specifications once baselined are subject to change control.
3	Functionality is delivered late due to underestimated development / resource requirements.	{project} Project Management to determine additional requirements e.g. resources to prevent impact on testing and inter-stream integration testing.
3	Functionality does not work with or affects the operation of existing functionality.	Technical Design Authority/Business Analysts to investigate solution to rectify.
3	Delays in the issue of functional and technical specifications will add a delay to the Test Scripting process.	Project management ensures that functional and technical specifications are delivered on time. Early views of the specifications will also mitigate this risk.
3	Numbers of iterations of the functional and technical specifications will result in rework of the test plans and test specifications.	Test management will ensure that work is only commenced on test plans and test specifications once functionality is signed off.

Key	Risk
1	New, low-level risks identified within this document
2	Risks contained within the Risk Register for which this document provides some mitigation
3	Significant risks identified within this document, and which will be added to the Risk Register

End Of Document