Five months ago, I was tasked with probably my biggest challenge to date in my development career. I was asked to create and implement a test strategy for my company. The previous projects were tested before being released, but there were no dedicated members of the team whose specialty would to ensure a high level of quality. I was brought in to fill this gap.

Although I knew I was up to the challenge, I had to admit, I didn’t know where to begin. Various emotions from excitement to anxiety filled me. I was excited because I had the freedom to shape the practices and processes that will be used across the business and projects for the foreseeable future. I wasn’t answering to anyone, no one was second guessing me. Once I had researched the tools or processes I needed and came to my conclusion, that was it. I was being trusted to make the correct decisions to ensure that the level of quality across the projects internally and externally was high.

Determining the scope and challenges

During my first week I made it my main goal to get a good understanding of the systems and tech stack we were using. One thing that was helpful to me (and others that joined later), was when I spoke to the developers to get an overview of the systems that we were creating, I made easy to follow diagrams. These helped me better understand what I needed to test and which systems communicated with each other. Once I knew this I was able to highlight what tests were needed, where they were needed and if we already had any tests in place.

After a few days of taking to the developers, finding out their wants and needs in terms of testing, figuring out what parts of the projects were being unit tested, what test runners, test frameworks etc were being used, I put together a current state of testing overview.

This outlined where we were, the challenges we were or would face if we continued with that and possible solutions.

Some challenges highlighted were:

Limited testing resources

I was only one person managing all the testing and QA efforts of the company.

New technology

Another was I had moved from a completely different tech stack. Although I had some background in this new tech stack, there would be a learning curve implementing a number of things.

New types of testing to learn

The product would require load testing, which I was interested in pursuing but again had no hands on experience with to date.

Devising my test strategy

So now that I knew where we were at and where we wanted to be I began crafting my test strategy document.

Googling led me to a few very key posts which helped me create the bare bones of my test strategy.

These are the ones that I found quite useful:

I began reading through each page and picking out the key points that would become the skeleton of my strategy document.

I started by dividing my strategy into the following sections:

Scope And Overview
Test Approach
Test Environment
Testing Tools
Release Control
Risk Analysis
Review and Approvals

With the document clearly divided up. I set about trying to fill in what I knew so far within each section. Then I could clearly see what was missing, what sections needed a lot more work and if any other supporting documents needed to be made in order to create a complete guide. Where there were already documents made I chose to either update or reused them if the information was still valid. There’s no point in making more work for yourself if you don’t have to!

How long did it take to write?

I had a deadline of getting this done by 28th April so I guess writing this document took me about six weeks to complete. Although, I wasn’t working on this all day everyday as I was trying to learn the tool and tech that I needed to test. I also had to hire another tester within my third week so going through the recruitment process, interview and managing this person was now embedded into my daily activities as well as daily manual testing, Scrum meetings and researching tools and processes. So I think despite it all, I didn’t do too badly to get it all written within that time.

Once the document was complete I had produced a Test Strategy document with the following sections:

Scope And Overview
- What is Quality Assurance?
- How should this Strategy be used?
- Mission Statement
- Testing activities carried out with timelines
Test Approach
- Approaches
- User Stories
- No Acceptance/BDD Tests
- Dashboards
- Test Process
- Test Workflow
- Testing Levels
- Roles and responsibilities of each team member
- Types of Testing
- API / Service (Integration) Testing
- Acceptance Testing (BDD Tests)
- System Testing/Regression Testing/User Acceptance Testing
- Smoke Tests
- Defect Raising and Fixing process
- Internal
- External
- Automation Strategy
- Test Schedule
- Test Plans
- Test Coverage Minimum Targets
- Challenges to Consider
- How can we test a virtual reality application?
- How do we run the Unity unit tests on builds?
- Scheduling
- How will we manage other quality considerations?
Test Environment
- Environments
  - Prism
  - I-Pass
- Load Balancers – Immerse platform.
- Service Locations
- Product Domains
- Backup of Test Data
- Restore Strategy
  - Sign Off
  - Rollback Tools
Testing Tools
- Automation
- Manual
Release Control
- Build Management Process
  - From Development to Production
  - Steps
- Release Windows
- Versioning
Risk Analysis
- Likelihood
- Impact
- Potential Risks
- Monitoring and Logging
Review and Approvals
- Sign Off Process
- Release Notes
- Done Criteria

This document also spiralled off the following standalone pages:

Bug Template
Sign Off Process Template
Release Notes Template
Defect Submission Process – External
Defect Submission Process – Internal

So you can see, this was no small task!

How has this document helped?

With this document now in place, the company has a level of quality to strive towards and something concrete that everyone can refer back to if there are any queries about why we are using certain processes or tech.

What have I learned so far?

Building something from nothing is always going to be hard work. It will always take you longer than expected. So it’s important that you:

Have a clear deadline for you to work towards
Break things down into manageable chunks
Have a team or people that you can go to for help when you may need it

Once the test strategy was complete I took a short break. This was to let my mind relax so that when I went back to try and enforce the strategy I was refreshed and ready to tackle any new challenges that may arise.

Next steps

The test strategy is not a document that you write only once. It must be evaluated and reviewed to ensure that it still stays relevant and useful to your company. What I intend to do is to mark out dates on my calendar where I can sit down and update the original document to keep it in line with the developments of the business.

Another good idea that I think is working well, is to create a short overview documents that I and other members of the team can refer to regularly without having to search the entire strategy for the information.

This can list consists of basic goals for the next 3 months like code coverage levels to achieve or processes to put in place buy certain dates.

Make sure your overview document captures the short term goals and highlights where that puts you in the long term so that you can clearly see how you and your team are progressing.

This post was imported into WordPress in one click using Wordable.

Creating a Test Strategy from Scratch