QA & Testing

ISTQB Foundation Level Notes: Chapter 5 – Test Management

In this short series, I outline the notes that I took while preparing for this foundation exam.

While the content may not have changed drastically in the time that I took the exam, there may have been small changes or additions that my notes don’t cover. So I would advise you that if you do use my notes to help you revise for the Foundation exam, that you use them as a supplement to the most recent edition of the book (this is an affiliate link) and go over your knowledge with practice exam papers.

Previous notes within this blog series:

Important Definitions

Risk a factor that could result in future negative consequences, usually expressed as impact and likelihood.

Level of risk = Probability of risk x impact

Project Risks used to manage the capability to deliver the project by the test leader.

  • Supplier issues e.g. contractual issues
  • Organisational factors e.g. skills, training, personal issues.
  • Technical issues e.g. problems defining the right requirements

Product Risks are managed when planning and defining tests using a risk-based testing approach by the test leader or tester.

  • Failure-prone software delivered
  • A defect in the software/hardware could cause harm to an individual or company
  • Poor software characteristics
  • Poor data integrity and quality
  • Software that does not perform its intended functions

Risks are used to decide where to start testing

Risks determine the test techniques to be employed and the extent to be carried out e.g. MISRA – the higher the risk, the higher the coverage required.

Risks identified prioritise testing in an attempt to find the critical defects as early as possible

Risks identified will determine any non-test activities that could be employed to reduce risk

Test Organisation and Independence

test organisation and independence

The greater the independence, the greater the likelihood of errors from unfamiliarity.

Tasks of a test leader

  • Writing or reviewing of:
    • The test policy for the organisation
    • The test strategy for the project
    • Test summary reports
  • Writing of the test plan:
    • Project and product risks
    • Test objectives, test levels, cycles
    • Appropriate test approaches
    • Estimation of time/cost
    • Test schedule
    • Acquiring resources
    • Configuration management
    • Incidents management
    • Test automation
    • Liaison with stakeholders
  • Preparation for testing:
    • Test case specification
    • Test environment requirements
    • Test tools required
  • Monitoring of tests progress
    • Metrics collection
    • Adopting the plan to meet project needs

Tasks of a tester

  • Review of, and contribution to, test plans
  • Analysing, review and assessment of user requirements specifications and models for testability
  • Test preparation
  • Creation of tests specifications
  • Set up of the test environments
  • Preparation of test data
  • Review of tests developed by others
  • Test execution (manual or automated)
  • Logging of test results
  • Raising defects
  • Use of test tools as required

Additional Resources for tasks

Development background

Component and integration testing

Business or user background

System and user acceptance

System Operators

Operational acceptance testing

Test Approaches

Test Approaches is the implementation of a test strategy.

Preventative Developed early in the life cycle.

Reactive Left until just before the start of test strategies

Types include:

  • Analytical approaches
  • Model-based approaches
  • Methodical approaches
  • Standard-compliant approaches
  • Process-compliant approaches
  • Dynamic and heuristic approaches
  • Consultative approaches
  • Regression-averse approaches

Different approaches may be combined if required

Where discretion is allowed the following factors should be considered when defining the strategy or approach:

  • Risk of failure of the project
  • Skills and experience of the people
  • The objective of the testing endeavour
  • Regulatory aspects
  • The nature of the product and the business

Test Planning and Estimation

Test planning

  • The main document produced is the project test plan
  • Normally produced during the early phases of the project
  • Will provide sufficient information to enable a test project to be established

IEEE 829 Standard For Software Test Documentation

16 sections minimum needed for an IEEE 829 compliant test plan:

Test plan identifier I

Introduction I

Test items S

Features to be tested S

Approach A

Item pass/fail criteria C

Suspension and resumption requirements C

Test deliverables D

Testing tasks T

Environmental needs E

Responsibilities P

Staffing and training needs P

Schedule T

Risks and contingencies R

Approvals P

S-P-A-C-E-D-I-R-T

S – Scope

P – People

A – Approach

C – Criteria

E – Environmental needs

D – Deliverables

I – Identifier and Intro

R – Risks and contingencies

T – Testing tasks and schedule

Test planning activities

  • Working with the project manager and subject matter experts to determine the scope of and the risks that need to be tested
  • Putting together the overall approach of testing
  • Liaising with the project manager and making sure that the testing activities have been included within the software life cycle activities
  • Working with the project to decide what needs to be tested
  • Building a plan the identifies when and who will undertake the test analysts and design activities
  • Finding and assigning resources for the different activities
  • Defining the management information
  • Ensuring that the test documentation generates repeatable
  • test assets

Entry Criteria

Determine when a given test activity can start

  • Test environment available and ready for use
  • Test tools installed
  • Testable code is available
  • All test data is available and correct
  • All test design activity has completed

Exit Criteria

Determine when a given test activity has been completed or when it should stop

  • All tests planned have been run
  • A certain level of requirements coverage has been achieved
  • No high-priority or severe defects are left
  • All high-risk areas have been fully tested
  • Cost – when the budget has been spent
  • The schedule has been achieved

Test Estimation

Two approaches:

Metrics-based approach relies upon data collected from previous or similar projects.

Expert-based approach to use the experience of owners of the relevant tasks or experts to derive an estimate.

Things that affect the level of effort required to fulfil the test requirements of a project can be:

Product characteristics

  • size of the test basis
  • complexity of the final basis
  • the amount of non-functional requirements
  • the security requirements
  • how much documentation is required
  • the availability and quality of the test basis

Development process characteristics

  • timescales
  • amount of budget available
  • skills of those involved in the testing and development activity
  • which tools are being used across the life cycle

Expected outcome of testing

  • the amount of errors
  • test cases to be written

Test Progress Monitoring and Control

Test progress monitoring to provide feedback and visibility of the progress of test activities.

Common test metrics:

  • % of work done in test case preparation
  • % of work done in test environment preparation
  • Test case execution
  • Defect information
  • Test coverage of requirements, risks or code
  • Subjective confidence of testers in the product
  • Dates of test milestones
  • Testing costs

Test reporting is the process where by test metrics are reported in a summarised format to update the reader regarding the testing tasks undertaken.

IEEE 829 includes an outline of a test summary report

Test control  uses this information to decide on a course of action to ensure control of the test activities is maintained and exit criteria are met.

The test leader may recommend the following to the project manager even though it it outside of the scope of responsibility.

  • De-scoping of functionality
  • Delaying release into the production environment
  • Continuing testing after delivery into the production environment

Incident Management

Incident-only unplanned event occurring that requires further investigation.

Incident reports have the following objectives:

  • To provide developers and other parties with feedback on the problems to enable identification, isolation and correction as necessary.
  • To provide test leaders with a means of tracking the quality of the system
  • To provide ideas for test process improvement

Details included:

  • Date of issue
  • Scope, severity and priority
  • References, including the identity of the test case
  • Expected and actual results
  • Date the incident was discovered
  • Identification of the test item and environment
  • Software or system life-cycle process
  • Description of the incident
  • Degree of impact on stakeholder(s)
  • Severity of the impact on the system
  • Urgency/priority to fix
  • Status of the incident
  • Conclusions, recommendations and approvals
  • Global issues
  • Change history

Configuration Management

The process of managing products, facilities and processes by managing the information about them.

For testing, configuration management will involved controlling both the versions of code to be tested and the documents used during the development process.

Where’s Chapter 6 Notes?

As I mentioned in my post Continuing To Build Knowledge, I never actually finished my Foundation notes before taking the exam. I simply read over chapter 6 a few times to make sure that I had this knowledge committed enough so that I could recall it quickly for the exam. I was planning to go back to writing notes for this chapter, but life occurred and this never happened.

If you’re finding these notes useful and would want a set of notes for Chapter 6, please contact me and let me know. I’ll add them to my Projects Trello board and see where it can fit in.

Lastly, if you’re revising for the ISTQB Certified Tester Foundation Level qualification, good luck!

You may also like...