ISTQB Foundation Level Notes: Chapter 5 – Test Management

In this short series, I outline the notes that I took while preparing for this foundation exam.

While the content may not have changed drastically in the time that I took the exam, there may have been small changes or additions that my notes don’t cover. So I would advise you that if you do use my notes to help you revise for the Foundation exam, that you use them as a supplement to the most recent edition of the book (this is an affiliate link) and go over your knowledge with practice exam papers.

Previous notes within this blog series:

Important Definitions

Risk a factor that could result in future negative consequences, usually expressed as impact and likelihood.

Level of risk = Probability of risk x impact

Project Risks used to manage the capability to deliver the project by the test leader.

Supplier issues e.g. contractual issues
Organisational factors e.g. skills, training, personal issues.
Technical issues e.g. problems defining the right requirements

Product Risks are managed when planning and defining tests using a risk-based testing approach by the test leader or tester.

Failure-prone software delivered
A defect in the software/hardware could cause harm to an individual or company
Poor software characteristics
Poor data integrity and quality
Software that does not perform its intended functions

Risks are used to decide where to start testing

Risks determine the test techniques to be employed and the extent to be carried out e.g. MISRA – the higher the risk, the higher the coverage required.

Risks identified prioritise testing in an attempt to find the critical defects as early as possible

Risks identified will determine any non-test activities that could be employed to reduce risk

Test Organisation and Independence

The greater the independence, the greater the likelihood of errors from unfamiliarity.

Tasks of a test leader

Writing or reviewing of:
- The test policy for the organisation
- The test strategy for the project
- Test summary reports
Writing of the test plan:
- Project and product risks
- Test objectives, test levels, cycles
- Appropriate test approaches
- Estimation of time/cost
- Test schedule
- Acquiring resources
- Configuration management
- Incidents management
- Test automation
- Liaison with stakeholders
Preparation for testing:
- Test case specification
- Test environment requirements
- Test tools required
Monitoring of tests progress
- Metrics collection
- Adopting the plan to meet project needs

Tasks of a tester

Review of, and contribution to, test plans
Analysing, review and assessment of user requirements specifications and models for testability
Test preparation
Creation of tests specifications
Set up of the test environments
Preparation of test data
Review of tests developed by others
Test execution (manual or automated)
Logging of test results
Raising defects
Use of test tools as required

Additional Resources for tasks

Development background

Component and integration testing

Business or user background

System and user acceptance

System Operators

Operational acceptance testing

Test Approaches

Test Approaches is the implementation of a test strategy.

Preventative Developed early in the life cycle.

Reactive Left until just before the start of test strategies

Types include:

Analytical approaches
Model-based approaches
Methodical approaches
Standard-compliant approaches
Process-compliant approaches
Dynamic and heuristic approaches
Consultative approaches
Regression-averse approaches

Different approaches may be combined if required

Where discretion is allowed the following factors should be considered when defining the strategy or approach:

Risk of failure of the project
Skills and experience of the people
The objective of the testing endeavour
Regulatory aspects
The nature of the product and the business

Test Planning and Estimation

Test planning

The main document produced is the project test plan
Normally produced during the early phases of the project
Will provide sufficient information to enable a test project to be established

IEEE 829 Standard For Software Test Documentation

16 sections minimum needed for an IEEE 829 compliant test plan:

Test plan identifier I

Introduction I

Test items S

Features to be tested S

Approach A

Item pass/fail criteria C

Suspension and resumption requirements C

Test deliverables D

Testing tasks T

Environmental needs E

Responsibilities P

Staffing and training needs P

Schedule T

Risks and contingencies R

Approvals P

S-P-A-C-E-D-I-R-T

S – Scope

P – People

A – Approach

C – Criteria

E – Environmental needs

D – Deliverables

I – Identifier and Intro

R – Risks and contingencies

T – Testing tasks and schedule

Test planning activities

Working with the project manager and subject matter experts to determine the scope of and the risks that need to be tested
Putting together the overall approach of testing
Liaising with the project manager and making sure that the testing activities have been included within the software life cycle activities
Working with the project to decide what needs to be tested
Building a plan the identifies when and who will undertake the test analysts and design activities
Finding and assigning resources for the different activities
Defining the management information
Ensuring that the test documentation generates repeatable
test assets

Entry Criteria

Determine when a given test activity can start

Test environment available and ready for use
Test tools installed
Testable code is available
All test data is available and correct
All test design activity has completed

Exit Criteria

Determine when a given test activity has been completed or when it should stop

All tests planned have been run
A certain level of requirements coverage has been achieved
No high-priority or severe defects are left
All high-risk areas have been fully tested
Cost – when the budget has been spent
The schedule has been achieved

Test Estimation

Two approaches:

Metrics-based approach relies upon data collected from previous or similar projects.

Expert-based approach to use the experience of owners of the relevant tasks or experts to derive an estimate.

Things that affect the level of effort required to fulfil the test requirements of a project can be:

Product characteristics

size of the test basis
complexity of the final basis
the amount of non-functional requirements
the security requirements
how much documentation is required
the availability and quality of the test basis

Development process characteristics

timescales
amount of budget available
skills of those involved in the testing and development activity
which tools are being used across the life cycle

Expected outcome of testing

the amount of errors
test cases to be written

Test Progress Monitoring and Control

Test progress monitoring to provide feedback and visibility of the progress of test activities.

Common test metrics:

% of work done in test case preparation
% of work done in test environment preparation
Test case execution
Defect information
Test coverage of requirements, risks or code
Subjective confidence of testers in the product
Dates of test milestones
Testing costs

Test reporting is the process where by test metrics are reported in a summarised format to update the reader regarding the testing tasks undertaken.

IEEE 829 includes an outline of a test summary report

Test control uses this information to decide on a course of action to ensure control of the test activities is maintained and exit criteria are met.

The test leader may recommend the following to the project manager even though it it outside of the scope of responsibility.

De-scoping of functionality
Delaying release into the production environment
Continuing testing after delivery into the production environment

Incident Management

Incident-only unplanned event occurring that requires further investigation.

Incident reports have the following objectives:

To provide developers and other parties with feedback on the problems to enable identification, isolation and correction as necessary.
To provide test leaders with a means of tracking the quality of the system
To provide ideas for test process improvement

Details included:

Date of issue
Scope, severity and priority
References, including the identity of the test case
Expected and actual results
Date the incident was discovered
Identification of the test item and environment
Software or system life-cycle process
Description of the incident
Degree of impact on stakeholder(s)
Severity of the impact on the system
Urgency/priority to fix
Status of the incident
Conclusions, recommendations and approvals
Global issues
Change history

Configuration Management

The process of managing products, facilities and processes by managing the information about them.

For testing, configuration management will involved controlling both the versions of code to be tested and the documents used during the development process.

Where’s Chapter 6 Notes?

As I mentioned in my post Continuing To Build Knowledge, I never actually finished my Foundation notes before taking the exam. I simply read over chapter 6 a few times to make sure that I had this knowledge committed enough so that I could recall it quickly for the exam. I was planning to go back to writing notes for this chapter, but life occurred and this never happened.

If you’re finding these notes useful and would want a set of notes for Chapter 6, please contact me and let me know. I’ll add them to my Projects Trello board and see where it can fit in.

Lastly, if you’re revising for the ISTQB Certified Tester Foundation Level qualification, good luck!