How to Future-Proof Your Testing Career

Here’s my next post which appeared first on Simple Programmer..

Since I switched back into testing from a development role, I’ve been trying to build up skills that will future-proof myself. To do this, I’ve been going to more talks and conferences, and watching webinars on testing.

What do I mean by future-proofing? I want to ensure that I have the correct skills, either by focusing on one specific area or gathering a broad knowledge of a few keys areas, that may be popular or emerging in technology now or in a year or two. Technology has changed so much even in the last 10 years, so it’s one area that I think you should stay up-to-date on the trends.

Earlier this year, I watched a webinar called 2018 Test Automation Trends. I’m sure all testers have heard of or are using automation tools these days. They have probably heard the fears of some that manual testers are being made obsolete by automation (this is in no way true).

But automation isn’t everything. This isn’t the only area that could help you future-proof your testing career.

Automation is just another area of testing that’s getting more attention. It has become highly visible how important testing is, and how bad products affect end users as well as shareholders.

Although this webinar was about automation trends, at the end, the experts were asked what they thought would be things to watch out for in 2018. I believe that if you want to stay ahead in this field, you need to look ahead too.

So here are the topics that these experts recommend to focus on in 2018, and also some that I think are good areas to begin future-proofing yourself as a tester.

Blockchain

Blockchain is a data structure that exists in many locations at once. You can only add to the blockchain. No deletions or updates are permitted. The data held within a blockchain is decentralized, which means a copy of the existing blockchain is on every machine in the network.

Additions to the blockchain can be seen on every computer in that network and the transactions are cryptographically linked to the previous transaction. Therefore, carrying out fraudulent transactions is very difficult to perform. To do this, someone would have to rewrite their history to the beginning of time, which is extremely resource-heavy.

With Blockchain applications, once a contract is deployed to a blockchain, it can never be changed. So you have to be very confident the testing that’s performed is of a high level of quality and that everything that should be covered has been covered. There is no rolling back.

For more information, I wrote a blog post to introduce testers to blockchain applications. Have a read and see what you think about blockchain.

Understanding blockchain technology and how to test it will help future-proof your testing career. It will help enable you to move into different fields, like banking and the automotive industry, which are both currently using this technology. It will give you more options about where you will work.

Contracts and API Testing

Contracts and application programming interfaces (API) testing focus on testing API endpoints of services and microservices.

There are different types of services. One example is a web service, which provides a way for applications and systems like databases, mobile apps, and other web servers to communicate with the service over the internet using endpoints. Endpoints are URLs that your HTTP client interacts with in order to access the data from that URL.

When testing endpoints, you ensure that when requests are sent with the correct parameters or incorrect parameters, the system behaves in a way that you expect. For example, if you send three parameters to an endpoint and the endpoint requires four, then you would expect a response to be returned with a 400 status code.

The services architecture is widely adopted across many industries, because it allows code to be written in a more modular way. This allows the project to be easily extendible, and will also mean that systems are coupled less tightly together.

As a tester, you need to ensure that you can test endpoints and contracts thoroughly. Incorrect requests could mean that your application doesn’t respond with the correct status code or message for the user in the best case. But in the worst case, it could lead to holes in your application’s security.

As testing jobs require more knowledge about different ways to automate their testing code, learning how to implement API tests and perform contract testing will help future-proof your testing career. You’ll become an expert in a specific area of automation, and as more applications, particularly web-based ones, are built using the service architecture, your skill to competently test the integration layer will be very sought after.

Mobile Testing

Smartphones and tablet devices have become a staple product within homes all over the world, for those young and old. It’s just as important to have a well-tested app nowadays as a well-tested website.

In 2005, Google changed its search results ranking algorithm so that if a website wasn’t responsive, that site would be ranked lower in its results list. The dedication to and sophistication of mobile testing has massively increased since then.

We first began testing mobile websites and apps manually, but because of the fragmentation of Android and the yearly releases of all mobile OSs, the need to automate mobile testing has increased. Because of this need, there has been a number of mobile testing frameworks released to aid testers.

Appium is a mobile testing framework built on Webdriver technology. It was created in 2013 by Dan Cuellar, a test manager at Zoosk who found that the length of the test passes on the iOS product was getting out of hand. Appium allowed him to write automated tests for iOS as easily as Webdriver was used for automating websites.

I recently went to the first Appium Conference held in London to learn about how I could utilize Appium.

Knowing how to effectively test mobile apps and websites is now a full-time job, so tools like Appium are used to help mobile testers test thoroughly. Mobile devices are everywhere now and the development of applications will only grow, so I believe it would be a great skill to add to bolster your testing skill set.

Artificial Intelligence (AI)

AI is where scripts, machines, or anything that’s not human perform tasks that may or may not be complex.

Within AI there are a number of different areas. One that’s becoming more common now is machine learning (ML). Machine learning is when AI provides you with answers or data based on previously generated data. It takes all of the information gathered and makes predictions on future outcomes or decisions based on this historical data.

One well-known example of ML and AI is the AlphaGo Zero software from Google’s DeepMind company that taught itself to play Go. Within three days, it was able to beat the previous version of itself (and that version bested 18-time world champion Lee Se-dol).

In testing, we can use ML to predict outcomes of test cases by looking at historical test result data. It can be used to predict which test cases should be prioritized over others when certain parts of the system have been updated. It can tell you when a test case was last run so you can determine what test cases are the most important to be run.

We’re constantly investigating how to become more efficient, save time, and complete jobs faster. AI can aid in this, so it’s a good idea to know its strengths and weaknesses, and how it can be used to help you with your testing career.

We’re at the very early stages of AI and how it can be utilized within daily testing activities,helping us achieve a high level of quality of our products. Ensuring that you keep up with or are ahead of the developments and applications of AI within software testing will help solidify your expertise in the field and help future-proof your career.

Docker and Containers

Containers give us the ability to store a number of different systems virtually within the same location. This means that if one environment needs a website front end, server, and database in order to function, all of these can be stored within the same place.

For testers, containers provide a quick solution to deploy test environments that are stable, fast to set up, and a mirror-copy of staging or production environments.

Learning the strengths and weakness of the architecture that your environments are built on will mean that you know how your application can be affected and whether certain types of tests will be more suitable to ensure a higher level of quality.

For more information, I’ve written a post about containers and how we use them at my current company.

Learning about Docker and the containers technology, and how to test it, will help future-proof your testing career. Since it allows quick deployment and scaling of infrastructure, it’s being widely adopted by a number of DevOps teams in different industries. This means you, again, will give yourself more options when looking for your next job opportunity.

Some Suggestions From Me

After listening to the expert’s suggestions, I’ve come up with a short list of additional areas that I think would definitely help bolster your testing skills arsenal.

Debugging

Debugging is where you step through code in order to locate defects. This isn’t something that every developer or engineer likes to do. Having the patience to walk through code can be an overlooked skill.

Debugging can take a few minutes if you know what’s wrong and where it is, but it could sometimes take days to figure it out—particularly if the code base is made up of hundreds of files.

Debugging also makes you practice reading the code, and it will help you develop a further and deeper understanding of the code base you’re stepping through.

This skill is useful for developers and is great for testers to future-proof themselves, especially if you are developing test code in any capacity. But since it’s not one that’s actively trained, you should invest your own time into debugging code to further narrow down where in the codebase the defects can occur.

You can do this by reading log files of applications when errors occur, and tracing them through the code base to see if you can track it down.

For more information on how to improve your debugging skills, read 5 Steps to a Bullet-Proof Debugging Strategy.

Augmented Reality, Mixed Reality, and Virtual Reality

Devices that provided users with each of these experiences have been around for years. However in 2017, I believe they were accepted and adopted by a larger consumer audience.

As this new tech is adopted more and more by the public, the demand for experienced testers in these areas will increase.

Since it’s still very new, there’s not much in terms of frameworks or testing standards that can be applied yet.Still, this is the time for someone to take that first step and create something that everyone can use going forward.

Cloud Computing

Cloud computing is the on-demand delivery of IT resources like compute, database storage, and applications via cloud service platform over the internet using a pay-as-you-go format.

Platforms like Amazon Web Services (AWS), Google Cloud Platform, and Microsoft Azure are some of the options that are available presently, with AWS storming ahead of the competition.

So why should you as a tester be concerned with what tech your applications are built upon?

Well, it’s the same reason as I mentioned earlier about containers. If you know what tech your application is built upon, you should be able to understand it’s strengths and weaknesses and how that will affect your application.

Therefore, you will be able to test your application better and help close up any holes in security that may be exploited, ensure it’s performant, and that the end user has a great experience when using your product.

How Future-Proof Are You?

When you’re working within a technical sector, the ability to step back and assess the bigger picture of what’s going on in your industry, what things are emerging, what things may be going away, and what’s here to stay will tell you a lot.

It will clearly show you where you should invest your time on new skills, and which ones you may need to strengthen.

If you want to do more in order to future-proof yourself, why not try and go to a talk in one of these areas yourself, read blogs, listen to podcasts, or watch YouTube videos of experts in these fields. You can also get a free subscription to AWS and experiment with the Elastic Container Service to get a basic knowledge of containers.

I hope this post has given you a foundation of what’s to watch out for now in 2018 within the field of testing. There’s a lot of options. So don’t wait—get started.

The Challenges of Leading a Distributed Test Team

Here’s my next post which appeared first on Simple Programmer..

Being a manager of a distributed team of people can be a challenging role. When you have to communicate on a daily basis and those people are in different time zones, or if you’re at the end of the project delivery date, you may find your stress levels slowly creep up higher and higher.

I’ve not only been trying to lead a distributed (albeit small) team for the last year, but I’m also trying to balance my daily testing tasks, researching new tools that could help my team, and maintaining or improving the overall quality of projects across the company.

Let me share with you a few challenges that I’ve faced and how I have or am planning to overcome them in future.

Different Time Zones Between Team Members

It’s important in every project for team members to regularly communicate. It’s easy for details to be missed, especially during busy times in the development process.

When your team are in different time zones, it’s even more important to make sure that they communicate daily. You’ve only got a specific amount of time overlap to talk directly with your team all at once. This time must be utilized and never skipped by anyone.

Ways that your team could communicate could be via the daily stand-up meeting (if you use Agile methodologies), or making sure you are giving daily reports via other channels, like internal communication systems such as Slack or Skype. Whatever medium you use, just ensure that the communication is regular and everyone gets involved.

Lesson: Plan your meetings at times where everyone is available and projects are not busy.

Balancing the Workloads of All Team Members

Even in teams as big as 100 people, it’s important that everyone feels like they are not overwhelmed by the work we have and can also perform their job to the best of their ability.

Having a distributed team also requires you to take into account what time other team members may also be working so you can assign the correct project to the correct people. You can use tools like Trello, Asana, or even a spreadsheet to manage project assignment. Doing this will help those team members more easily manage their workload so the project is more closely aligned with their schedule.

You’ll also need to consider the skills that are needed for a project and how long the project will last. This will help you assign the right team member to the right project. For example, say you had an upcoming project where the test framework needed to be written in C# and some of your team were proficient in Java and some in Ruby. You may find that this type of project will be easier to and faster to implement by members of team know Java as they are quite similar languages.

Lesson: Try and match the right projects to team members with similar skills, interests, and time to carry out the project.

Trying To Manage Everyone and Everything

Despite being a small testing team where I work, we have a continually big workload. We are managing the quality and daily testing for two-to-four projects at once. With my job, carrying out daily testing tasks, building the automation suites, and trying to monitor the overall quality strategy in the company can be a lot to do for one person. But not for me, I’m Superwoman!

Only (and obviously) joking. Seriously, I realized very early on that there isn’t time to do everything, and that I can’t manage everyone and everything by myself. So, to make sure I can carry out all of my tasks, I prioritize my work and delegate lower-priority tasks to other team members.

I split responsibilities between team members according to project deadlines, project requirements, and team member availability. I encourage my team to do the same thing to make sure they also don’t feel overwhelmed and can get their jobs done well.

In a distributed team, this can work to your advantage. For example, if you have team members who are in a time zones hours ahead or behind you, you can leave them work when your overlap time is over. Then when you return, the task may be fully or almost complete, ready for you to focus on something else.

In order to do this, you need to know your team and trust that they will carry out their jobs so that you can focus on the big picture.

Lesson: Get good people to help you manage things.

Trying to Monitor the Progress of Your Strategy

When I first arrived at my current company, one of the first things I was asked to do was outline a Test Strategy for the company.

Every month I try to track if we’re still on our way to achieving those goals I set a year ago, if they need adjusting, or if they’re no longer relevant and some of the strategy needs revisiting.

If all of my team were in one location, it could be as simple as having a quick meeting once a month to find out where we are. But having a distributed team means that this needs to be planned in advance and at times where everyone is available. With some members of my team in time zones as far away as four hours at times, meetings need to be planned at times during the project where each member won’t be busier than usual, like the week before a product delivery.

Utilize free calendar resources, like Google Calendar, to share a team calendar booking in important dates where the team will be busy or unavailable. Use this to regularly track your progress with team meetings.

Lesson: Have a clear idea of your vision and keep track of it being executed.

It’s Easier for Quieter Members to Hide Away

Make sure the quieter members of your team are also vocal at team meetings. Find ways to get everyone engaged with their team. It’s easy for those that are quiet to sit back and let others that are more confident speak up, making themselves be heard, especially if these meetings are over Slack or Skype.

You can do this by directing specific questions to these people or asking them their opinions. This will prompt them to speak up a bit more. If English is not their first language, they may be more inclined not to speak up, but giving them a little nudge by asking a question to them here or there may be all they need to build up their confidence.

Keep in mind that there may be a reason why these people are quiet, so before you ask them questions in meetings with everyone ready and waiting, maybe talk to them privately. Learn who they are and what motivates them, then use that to regularly engage with them.

Lesson: Learn who your team are and how they like to communicate, and encourage them to participate.

It’s Hard to Keep Everyone’s Skills Up

In any technological field, like software development, virtual reality, big data, or artificial intelligence, one of the challenges is to make sure you are constantly learning. Things are always changing and it’s important to keep your skills up. It’s hard to find time to build your skills (especially within testing and QA roles) because at the end of the day, you’re employed to develop and ship a product. Improving your skill set in an area is usually a byproduct of working on something day in day out. But, I would suggest you encourage your team to carve out some dedicated time in their week to learn new skills or improve on existing ones. This not only will keep people in your team, show you care and want to invest in their personal development, but will also make them enjoy coming to work.

Utilizing resources like Pluralsight courses (aimed at those in technical professions), YouTube videos (for everyone and about everything), knowledge sharing sessions, and pair programming are all easy to access. All of these are free to low-cost ways to improve your knowledge and strengthen the skills of your team.

It’s easy to let deadlines let us work through these times, so make sure your team schedule them at times where they have little to no distractions (if at all possible).

Lesson: Tell each team member to calendarize a time slot for themselves to work on skills weekly to improve their knowledge.

The Bottom Line: Have a Great Team

Leading a distributed team as well as all of your other responsibilities as a manager is a challenging task. You shouldn’t underestimate that the amount of work done day-to-day will be the same as when you lead a team all in the same location.

Sharing tasks among your distributed team members, like responsibilities, projects, and even meetings, will help you balance the efforts of your daily and management tasks. But you need a good team to help you with this.

So, make sure you find the right people to help you. In return, invest some time into them by making sure they are not overloaded with work, get to know them by finding out what motivates them, and encourage them to build their own skills by training themselves.

My First (and the) First Appium Conference

Last year, I was selected for a scholarship to Appium Conference 2018 by White October Events. I’ve been aware of Appium for years now as I’m an app developer but I’ve not (yet) used this tool on any of my projects.

For those that don’t know, Appium is a mobile testing framework built upon the Webdriver technology. It was created in 2013 by Dan Cuellar. He was an Test Manager at Zoosk who  was finding that the length of the test passes on the iOS product was getting out of hand. Appium allowed him to write automated tests for iOS as easily as Webdriver was used for automating websites.

Five years later, Appium has a massive community building up a successful open source project that can be used on Android, iOS, Windows in a variety of coding languages.

This year was the first Appium Conference.

There was only one track which was great because I got to see everything and didn’t have to pick between two talks that were probably going to be beneficial to me.

Interesting Things to Note

  • There was a lot of variety of languages being used with Appium (part of the appeal of using the product I suspect).
  • Lots of people were using Jenkins with it as their continuous integration/continuous delivery tool. There were a couple mentions of Circle CI but none about TeamCity. Because of this, I think I’ll be looking into Jenkins more than TeamCity for my app projects.

All the talks were highly interesting and not too difficult to follow for a beginner like me. So I want to share my biggest takeaways from each of the talks.

Keynote – Appium: The Untold Story

The day began with a keynote from Dan Cuellar and Jason Huggins. They spoke about the history of Appium, where it is now and briefly touched upon where he wants it to be in the future and mentioned their vision of StarDriver.

They want to see Appium grow it’s users and the platforms it can test particularly to the internet-of-things and various hardware.

The best takeaway for me from this keynote was the phrase “Challenge everything you see”.

Appium: A Better way to Play the Game

This first talk was given by Charlene Granadosin & Charlotte Bersamin. What I found interesting within their talk was how they were integrating their release and exit reports within Jira using Xray. They used a curl command to upload their latest test results to Jira so these results are clearly visible to the Product Owners or Managers of the team.

My biggest takeaway from this talk was to investigate whether the tools we were currently using for test case management was able to integrate with Jira to give such detailed reports and to try and get the automation up and running.

Deep Hacking Appium for Fun and Profit

Daniel Puterman‘s talk explained how he had contributed to the Appium project by creating a new endpoint to gather native application screenshots.

Because the company Daniel worked with was Applitools, my biggest takeway from this was to figure out whether visual testing tools would be useful for testing virtual reality (VR) applications as much as they would be for websites of mobile applications.

Why the h# should I use Appium with ReactNative?

Wim Selles delved into a comparison talk about why they chose Appium (which was extremely useful as I’ve also been debating what automation tool to use for mobile apps).

Out of all the frameworks he mentioned, they went with Appium because it fit with a lot of his requirements for testing ReactNative apps.

There were a lot of takeways for me from this talk.

  • Consider your own project requirements when you pick your automation tools
  • What are your requirements?
  • What should your app do (now/future)?
  • Which tool supports your needs/expectations?
  • Do a proof of concept test
  • Do research into competitive tools

He also gave a couple of good ways that you can speed up app testing:

  • Remove animations on screens
  • Utilise deep-linking to get directly to screens

Layout Automation Testing (GUI)

Prachi Nagpal explained how she was using Galen to perform her UI browser based testing for mobile and desktop devices. Galen does this by measuring the distance between elements that are being tested. You can also produce heatmap results from this tool.

It was a good talk and interesting to see a tool that I had never heard about.

Can you please provide the full Appium Server logs? A Brief Tour of the Logs

Isaac Murchie next walked us through the Appium logs and the takeaway here was that he pointed out that some lines are bolded to highlight their importance.

He also noted that the following are ways to know which are requests and responses in the logs.

This is a request:

[HTTP] -> 

This is a response:

[HTTP] <-

Interaction with Native Components in Real Devices

In his talk, Telmo Cardoso told us how he tested native components of mobile operating systems. He explained the challenges that he faced (some tasks were difficult on one platform but easier on the other) and ways he and his team had got around them.

The areas he found challenging were:

  • Adding contacts
  • Pushing files to a device
  • Push notifications
  • Managing calls
  • Simulating low battery
  • Extracting logs

The biggest takeaway from this talk was that he used Cucumber for his automation framework along with with Appium successfully to test the native applications and features of smartphones and not just the applications running on them.

Using Appium for Unity Games and Apps

Because of my daily work with Unity projects, I was particularly looking forward to Ru Cindrea‘s talk on how she used Appium for her Unity games and apps.

She first explained how she used OpenCV an image recognition tool with Appium to try and test her Unity games.

The positives were:

  • Works for simple scenarios
  • No changes to game required
  • Found issues like performance issues or out of memory crashes

The negatives were:

  • Wasn’t fast enough
  • Not for games with lots of text

So she decided to create a component called AltUnityTester to help her with her issues.

AltUnityTester is created with Python bindings. It opens a socket connection and waits for a response on a specific port.

When the AltDriver is added into the Appium project it gets a list and knows everything about that Unity scene’s objects. It can then send a command to that port to get information back from the scene to perform tests e.g. checking the end position of elements or text output.

This solution is useful because it’s real-time but it does require changes to the project and it only works with Unity.

So my biggest takeaway was to investigate whether this AltUnityTester could be extended or something similar made in order to test VR applications using Unity.

It’s available as a Unity package or on Gitlab.

Docker-Android: Open-source UI Test Infrastructure for Mobile Website and Android

Application

Budi Utomo next talked about his Docker-Android image to test Android projects and websites on Android devices.

His plan for project development was to:

  1. Create UI tests for Android devices
  2. Write unit tests on Android
  3. Create UI tests on Android apps
  4. Implement Monkey/Stress tests

The biggest takeaway was the demo that showed how Appium can be used easily within Docker containers.

Application Backdoor via Appium

Rajdeep Varma explained how you can use Appium scripts to call development code methods from test code.

He used Appium in this way because he was having a number of problems when trying to write tests:

  • System pop-ups were called and were not needed when running tests
  • Driver limits e.g. mocking the device has been shaken or changing time limits
  • Tests were slow to run

This is where he is using backdoors and where he thinks they could also be used:

  • Changing backend URLs
  • Changing app locale
  • Getting session ids from the app
  • Disabling “What’s new” pop-ups
  • Disabling client side A/B tests
  • Faking SIM card for payments
  • Get analytics data from the app to validate it

The biggest takeaway from this talk was to be careful not to use backdoor for every test case and call incorrect methods in order to make tests pass.

Mobile Peer 2 Peer Communication Testing

Canberk Akduygu gave us a talk about his challenges when automating the BIP app (it’s like the Turkish Whatsapp).

He was building an extended grid solution to change to the right version of Appium and set the desired capabilities within their testing framework according to properties set in a JSON config file.

His demo was the biggest takeaway which showed two phones messaging and even calling the other. It was one scenario that was running two different steps on each device. It showed that the test steps needed to be synchronised.

From a Software Tester to an Entrepreneur: What I’ve Learned

Kristel Kruustuk next came on stage and walked us through why she founded Testilio and her struggles with the company despite being so successful and growing at a fast pace since she began.

My takeaway from this talk to was investigate and get in touch with the Testilio team and see if they had any future plans for expanding from manual and automation testing to VR testing.

Appium: The Next Five Years

Jonathan Lipps gave the final talk and began again with the history of Appium but then spoke more in depth about the vision of the product over the next five years and hopeful milestones. Some of these things were:

  • StarDriver
  • InfinityDriver
  • Extension to the W3C WebDriver protocol
  • Node js base classes and libraries for easily writing new drivers

Closing Remarks

Lastly, we were treated to a performance featuring Jonathan Lipps, Appium and Selenium. I believe four or five instruments were being played by Appium, Selenium was outputting the lyrics and Jonathan was singing and playing the ukulele.

My biggest takeway from this is that Appium can be used in a variety of ways to perform a number of impressive tasks.

After Party

The after party was held at a bar a short walk from St Paul’s Cathedral. There I managed to talk to Ru Cindrea in more detail about the project I wanted to use the AltUnityTester for and whether she thought it would work. I also, managed to talk the ears off of both Charlene and Charlotte who were the speakers from the very first talk of the day.

Overall

I had a great day, met loads of wonderful people (including attendees and speakers) and I hope that next year, I’ll have begun using Appium for something that I do so I can share my own experiences with the community.

Moving to Containers

It seems that containers are the new technology within IT that everyone is trying to incorporate into their infrastructure. And why? Containers not only benefit those in DevOps, but have positive implications for all teams involved in product delivery.

At Immerse, we’ve recently moved our infrastructure to a containerised solution. It’s early days to analyse the impact that this has had on our development teams, but I thought it would be a good opportunity to deepen my understanding of the this technology.

What Are Containers?

Containers give us the ability to store a number of different systems virtually within the same location. This means that if one environment needs a website front-end, server and database in order to function, all of these can be stored within the same place.

What sort of systems can be run within containers? Well, that’s where container images come in.

What Are Images?

A container image is a stand-alone, executable package of a software that has everything needed to run it including code, runtime, system tools, system libraries, and settings.

An image is required in order to build a container, otherwise it will be empty when created.

So where did all this new tech come from then? It seems like it’s come out of nowhere but spread fast (kinda like Bitcoin right?). Well, it all started by a small company now called Docker Inc. They created the system that containers run on, Docker.

What is Docker?

Docker is a computer program that allows you to perform operating-system-level virtualization known as containerization. This is the creation of containers.

Docker allows independent containers to run within a single Linux instance. This reduces the overhead of starting and maintaining virtual machines (VMs).

Since Docker began, there are other tools than have been developed that can perform containerization.

The World Before Containers

Before we used containers, there were virtual machines (VMs). VMs remove the need for physical hardware and allows one server to be turned into multiple servers. App of this is possible because of a hypervisor.

A hypervisor (also considered a VM monitor), is software that creates and runs VMs. It is the reason why you can run many VMs on a single machine. Each VM will have a full copy of the required operating system, one or more applications and the needed binaries and libraries. All of this can take up tens of gigabytes of space!

Some companies have made the switch to containers from VMs because:

  1. VMs can also be slow to boot, while you can spin up a container within a few minutes providing you have the right image. And if the don’t, that usually only takes a few minutes to obtain.
  2. You can pack a lot more of your companies applications into a single physical server using containers than a what you can fit in a VM.
  3. VMs take up a lot of system resources as they not only just run a full copy of an operating system, but a virtual copy of all the hardware that the operating system needs to run. All that a container needs is enough of an operating system, supporting programs and libraries, and system resources to run a specific program.
  4. With containers you can create a portable, consistent operating environment for development, testing, and deployment.

So How Do You Manage Containers?

Container Orchestration are frameworks that are used to integrate and manage containers. These are not necessary for everyone using containers. Usually enterprise level organisations are more likely to use orchestration tools as they manage a large range of containers and their images. Examples of these tools are Kubernetes, ECS and Ansible.

These tools help to simplify container management from the initial deployment to managing multiple containers, scaling for load, availability, or networking.

The Benefits and Drawbacks

Like any new piece of technology or tool, containers too have their own list of benefits and drawbacks that mean you choose to either integrate them into your development pipeline or you don’t. So, what do containers offer?

Benefits

  • The ability to spin up whole environments consisting of all the systems you need within minutes.
  • The ability to change the configuration and deploy those changes quickly.
  • Containers allow all users of the system to be self contained. This mean that rogue developers who develop new features, don’t run tests locally, push to the test environment, then leave to go home only for QA to find that test is broken is no longer a daily issue (wow, that sounded like a rant!).
  • Features can be tested in isolation easily.
  • Testers can be in control of checking out and pushing features to the test environment.
  • Differences between environments is no longer an issue because they’re all spun up from the same set of stable images

Drawbacks

  • Initially, there maybe some complexity to setting up containers.
  • There are some security issues that you med to be aware of when using containers. For example, if a user or application has superuser privileges within the container, the underlying operating system could, in theory, be cracked.
  • It’s time intensive to set up decent security measures for containers. There’s no default, out if the box solution yet
  • Everyone is making container images and it could be easy to download something malicious into your system.
  • Breaking deployments into more functional discrete parts is smart, but that means we have more parts to manage. There’s an inflection point between separation of concerns and container sprawl!
  • Containers tend to lock you into a particular operating system version

Are containers the future of development?

It seems because of the fast adoption of containers that they may eventually replace VMs once their issues have been overcome. And because it is a new technology, there will be drawbacks at this early stage so don’t let these deter you from experimenting with the tech yourself on your own projects.

However, technology has changed extremely fast over the last 30 years, so it may be that containers are superseded by a new emerging technology that solves the drawbacks of containers and gives us a while load of other benefits too.

For more information of containers especially if you’re learning the basics, please check out the Docker videos by Nigel Poulton on Pluralsight. I found these videos extremely helpful in delivering information and background about a brand new technology. The concepts were also broken down into easy to understand topics which is perfect for beginners. After watching them, I was able to understand a lot more and felt more confident when speaking to the DevOps at Immerse about how they had implemented container and why they made the decisions they did.

What’s your experience with containers? Do you love them? Are they growing on you? Or, have you not yet made the leap into using them? Whatever your experience, I hope this article has given you a better insight into the background of containers.

For more information about containers, please feel free to view the references I used:

This post was exported in one click from Google Docs into WordPress with Wordable.

A Beginner’s Guide to Testing Blockchain Applications

Here’s my next post which appeared first on Simple Programmer..

During the last few months of 2017, bitcoin and other cryptocurrencies were being talked about by some media sources on a daily basis. These currencies, which had been around for years, were suddenly experiencing major growth. For example, the price of bitcoin has grown from just under $750 in January 2017 to $5,856.10 in mid-October 2017.

This massive growth made it a major talking point. So, as these newer currencies came more into the public domain, the technology behind cryptocurrencies was also beginning to gain attention. This technology is the blockchain.

Blockchain applications are being adopted by some of the biggest industries around the world. Because of the nature of blockchain applications, it further supports how important testing and testers are, and that the field will be more highly thought of and sought after in the future. So as testers, it’s only right to wonder how this new technology will affect your day-to-day work, new opportunities, and current skill set.

What new tools will you have to learn to test blockchain applications, and what skills are needed to test them?

Here, I will outline what the blockchain technology is and how we, as testers, can prepare ourselves for testing blockchain applications.

What Is Blockchain?

Blockchain is a data structure that exists in many locations at once. You can only add to the blockchain. No deletions or updates are permitted. The data held within a blockchain is decentralized, which means a copy of the existing blockchain is on every machine in the network.

Additions to the blockchain can be seen on every computer in that network and the transactions are cryptographically linked to the previous transaction. Therefore, carrying out fraudulent transactions is very difficult to perform. To do this, someone would have to rewrite their history to the beginning of time, which is extremely resource-heavy.

In order to update every machine in the blockchain, the machines have to sync to have a common history. Although all machines will eventually have the same data because of this syncing action, only the more recent transactions are synced more often.

For a new transaction to be added, the decision is reliant on the majority of participants in the blockchain. Once the transaction’s authentication has been validated, the new block is added to the blockchain.

What Is It Used For?

Currently, blockchain is used mainly by the financial and automotive industries because of its highly secure structure. As I mentioned earlier, it’s also the technology that underpins cryptocurrencies like bitcoin and Ethereum.

Not all blockchain technology is open-source. There can be private blockchains like the ones used in banking systems.

What Types of Tests and Techniques Can You Perform on Applications Built on Blockchain?

There are many different types of tests that can be performed at the various stages of developing software projects. Below are a few types of tests that can be utilized to ensure a high level of test coverage and quality for blockchain applications.

Unit Tests
Unit tests help developers ensure their code is performing correctly at the lowest levels and smallest parts of functionality. This should always be the first line of defense to ensure an application catches the majority of bugs early during development.

Integration Tests
Integration tests help developers and test engineers ensure the communication of their code between different components, and possibly between internal and external systems like databases.

User Interface
User interface (UI) testing uncovers how the application works from the end user’s perspective. It’s important to ensure that you carry out UI testing to make sure their experience is positive or that they at least get the correct feedback from the application when it doesn’t perform well.

Application Programming Interface
Application programming interface (API) testing gives you confidence that you’ve validated the responses that your application receives from external APIs and makes sure the formats of your API requests are correct and being handled correctly.

With blockchain applications, there is also a similar type of technology to APIs that allows you to adopt the same testing practices for APIs. These are called smart contracts.

What Are Smart Contracts?

Smart contracts are a big part of the validation technology within a blockchain. A smart contract is a “set of rules in the form of programmable constructs that are capable of automatically enforcing themselves when predefined conditions are met.” For example, a precondition could be transactions trying to append to a specific ledger will undergo additional validations or go through a different set of validations that are more robust.

Although a smart contract is very similar to an API, whereby it has public functions that can be called by anyone registered on that blockchain network, it cannot call external web APIs.

So why do I think that testers are extremely important when testing blockchain applications over other types of systems? Simply because once a contract is deployed to a blockchain, it can never be changed. So you have to be very confident the testing that’s performed is of a high level of quality and that everything that should be covered has been covered.

If a defect is found in production, then a new version of the contract has to be created and deployed. New versions of existing contacts can’t simply get the existing data transferred in; you have to manually initialize the previous data with the new contract.

Updating a contract and rolling back an update is also not a viable option; this increases the complexity of development and means that the importance of implementing and running unit and integration tests on your application before it reaches production could save you major time and money in rectifying defects.

What Skills Do Testers Need for Blockchain Applications?

Although blockchain applications are relatively new in software development, I don’t think testers need to adopt new skills in order to test this type of technology.

Some of the skills I’ve highlighted below are natural skills of good testers or simply skills that you learn early on in your testing career, which grows with your experience in the field.

Critical Thinking
The ability to critically analyze and think about and around a problem is a timeless skill for testers and will be even more sought after for testing blockchain applications.

Testers think about problems such as: Will transactions still execute if x, y, and z are not done? What happens if the network has lots of transactions waiting to confirm? What feedback is given to the user in these cases? Should this be the feedback given to the user? Or is this feedback exposing any security risks?

Another thing to consider if embarking on a new project is to question whether blockchain is the best technology for your use. It’s the new shiny toy, so everyone will want to adopt it, but it may not be suitable for what you want to achieve.

Things to keep in mind are compliance issues; for example, you shouldn’t store health or criminal records, as there are no deletions allowed. So, when criminal records for minors can be wiped, you won’t be able to do so with this technology.

Test Design Techniques
During the ISTQB-BCS Foundation Software Testing syllabus, you are introduced to test design techniques. Knowing even the basics of test design techniques, like boundary value analysis and equivalence partitioning, will make sure you are constantly thinking about and checking the inputs and outputs of the application.

Things to consider could be: How will the application act when you input values that are within, on the edge, and outside of the boundaries of acceptable values? Will the transaction complete? If not, what type of error will be returned? Is this error code correct for the type returned? Should it return anything at all?

Automation
Strong automation skills in all languages, either for lower-level unit, mid-level integration, or API or high-level UI tests, are good skills that can be transferred to testing blockchain applications. Having a solid foundation of automated tests will be needed to ensure that the majority of issues are found early in development.

Manual UI Testing
If a solid foundation of automated tests is put in place, testers can focus on the outlier issues that can more easily be found via exploratory testing performed manually.

Being able to work independently, investigating different areas of the application, trying to find weak areas, and being able to successfully reproduce these are always skills that great testers will need. Despite the world looking to automation to perform a lot of the repetitive and arduous tasks, manual testing skills are still something to hone and improve.

Learn New Tools Quickly
As new technology comes to light, the list of tools to test this technology will also grow. You’ll need to be able to learn how to use these new tools quickly and judge which is the best for what you’re trying to accomplish.

Future-Proof Yourself and Prepare for Blockchain

Hopefully by now, you have a better understanding of the blockchain technology and no longer think it’s as scary as your first thought.

I’m sure if you’re a tester, you already have the foundations of the skills I’ve outlined above. My advice is to push further into an area that you have an interest in and possibly try to improve in the areas where you are weakest to give you the best chance to grow your skills when testing blockchain applications.

Rebasing from Master onto your local branch

 

Now that my QA team are adding to the test coverage by writing integration tests directly into the project code base, it’s finally time that I start to embrace rebasing (I sort of feel that lightning should crack when you read rebasing)!

As I survive with the basics of Git and because Irina and I only usually submit to our own repos, apart from branching and merging back to master, there’s not much activity going on. But once you work on a repo where at least two people are active on it, merging their code into master and creating branches weekly, your local changes can fall behind quite quickly. This is where rebasing comes in.

Rebasing essentially rewinds back your branch to when you created it from master (if that’s where you branched from) and applies all the commits that have happened from master branch to your own. If there are conflicts or differences between the commits to the master branch and yours, you can either:

fix them at each commit, then you

[sourcecode language=”css”]git rebase –continue[/sourcecode]

, or you can accept the changes on the other branch and

[sourcecode language=”css”]git rebase –skip[/sourcecode]

The concept is quite clear to me (now), but what I struggled with was doing this all on the commandline. I know I could use a tool like Source Tree to do all the heavy lifting for me, but in fact, despite me being quite a visual person, I like using the commandline for Git. So I vowed that this will be the time to learn and to do it more often so that it’s solidified in my brain.

So after being walked through the process, I learned that these are the steps that you need in order to rebase from master on to your own branch.

So these are the steps, pretty simple and after a couple of times, I reckon I’ll be able to do this from memory…providing the conflicts are few and far between!

[sourcecode language=”css”]

git checkout master
git pull
git checkout MyBranchName
git rebase master
git rebase –continue (until you fix the conflicts) or git rebase –skip (to skip these changes)
git pull
git push
git status

[/sourcecode]

The steps here are what I found works, but if you spot anything wrong, don’t be afraid to let me know.

Building a Test Pyramid from the Bottom Up

I was brought into my current role at Immerse.io to uphold a high level of quality for the company’s awesome products. But where to start?

After I put together my test strategy I needed to know how much test coverage was currently in place.

Plenty of manual testing was taking place but it needed refining as not everything needed to be tested manually. Also, there were no test cases for the new system documented or being run so there was no way to know how much had been tested, what test was last run let alone what results those tests gained.

So with no test cases to look at, I turned my attention to the developers and their unit tests. The good news is that they had unit tests, the bad news is that they didn’t know how much. There were no code coverage tools integrated into their builds so they couldn’t tell.

So I first turned my attention to increasing unit tests in the projects and adding code coverage tools. Once we had the code coverage levels, we could work to increase the code coverage of the project codebases. If our code coverage level was high, it would give us more confidence about the quality of the system at a low level. But how many test cases should we create?

With this question in mind, I started thinking how I could use the Test Pyramid concept and apply it across our products.

What is the Test Pyramid?

The Test Pyramid is a concept that is used to determine what type of tests and how many of each type should be used at different levels within your product’s development. There are usually three different levels to the pyramid (although some depictions may contain between three to five). The image below is referenced from Martin Fowler’s blog post regarding the Test Pyramid.

As you go up in the levels there should be less tests. Also, as you progress up the pyramid, the tests become more complex because they involve connecting to more than one component or system. Therefore tests will take longer to run as you progress higher.

The Different Levels

The bottom level contains the majority of your tests. Unit tests are used for this level as run fast, have a small focus, are easy to add to and require little maintenance. Overall, they provide fast feedback on changes to the system at a low level.

The middle level contains your integration tests and the top level is for UI tests. The tests in the middle and top levels tend to be more susceptible to breaking because they connect with a number of components and systems.

UI tests are particularly brittle because they rely on web page elements which can be changed easily. Because the automation code is so tightly coupled to the page elements, if one is updated without the other, the tests will break. And, as these two codebases are usually maintained by different teams, sometimes the teams won’t remember to update one another. This could lead to one codebase being updated without the other one. These tests usually require more maintenance than unit and integration tests.

Fix issues as early as possible

The Test Pyramid supports the idea that a lot of thorough testing at the low levels and early phases of development helps to prevent bugs reaching the production environment.

Catching issues within your product during the later stages when the project is completed and ready for consumers is costly and can be more time consuming to fix. Attempting to apprehend issues, before they become defects i.e. production bugs, during the early phases of development is preferred because:

  • It involves less people to fix so can be presumed to be less time consuming
  • There is no impact to the customers so it’s less costly to the business

In your project, try and do the following when focusing on building your base level:

  • You should try to do the majority of your testing during the earlier phases of development with tests that are the cheapest to run.
  • It’s important to make sure that the amount of unit tests in your project is large enough to ensure that issues are caught and fixed before shipping.
  • Make sure you monitor the level of code coverage you have in your project to make sure you have the right amount to reach your target level of quality that will provide you with confidence in your product.

Follow the Test Pyramid and see what results you get.

Integrating Code Coverage

In order to follow the Test Pyramid concept, we first needed to build up the amount of tests in the base level.

Now that the tests were being added to, we needed them to be integrated within the CI tools we were using. So for one team, I set about getting these running within TeamCity. Using the Unity command-line test runner I managed to get this working when we were using Unity 5.4. Unfortunately, we found that this set up broke when we upgraded to Unity 5.6. Luckily, this has now be rectified in version of Unity 2017.2 and we have unit tests running in TeamCity on every build once again.

Unfortunately, there’s no way to measure code coverage on Unity projects so we’re just going have to be content with increasing the number of tests and making sure that these tests are meaningful. I’ll continue to monitor this though.

I next looked at getting a code coverage tool installed for the other team. Because of the tech stack here and how simple it was to setup on a simple project, I chose to use Istanbul. This has now been set up into a CI tool that runs on every push so that all code is checked and a code coverage report is produced. This gives us an idea of where we need to focus our efforts to increase the amount of unit tests.

Our Next Steps

As the production of unit tests are continuing and the developers are actively doing this themselves, I turn my attention to the next level of the Test Pyramid, the Integration tests.

This post was exported in one click from Google Docs into WordPress with Wordable.

Building positive habits to progress

After reading The Power Of Habit by Charles Duhigg, it made me realise that people’s habits can be used to change behaviours for the good and yes, sometimes the bad. These changes in behaviours can either positively or negatively impact your life.

So, in 2018, I’m going to try and do this for myself. Why not try and improve my development skills by building a new positive habit into my daily or weekly routine?

But building a habit is time consuming. It can also be difficult if what you’re trying to learn is brand new and interrupts other, more longstanding habits whether they’re good or bad.

Slow progression

Building a new positive habit into your routine is like when you first start going to the gym. It will take a conscious decision to keep it up in the beginning (it may hurt a bit too if what you’re doing is physical). The trick is to try and work the new behaviour into your routine little by little. Try and fit your new behaviour into times where it’s easy for you to implement.

Making sure your new behaviour fits into the SMARTER acronym will help you to get onto the right path building a new behaviour into a routine that will eventually become a habit.

Be SMARTER

If you’re unfamiliar of this well known guideline for target and goal limiting, each of the letters stands for a particular limit that you need to make sure the goal you set meets.

The traditional acronym has always just been “SMART”, however I recently came across this extended version. The addition of the “ER” is to make sure that you learn from the targets you set and make changes depending on the results. The practice of retrospectives are used a lot more within development teams now as it’s a vital meeting within the agile process of SCRUM which is widely used across many organisations now. So if you’re a part of a new organisation you’re probably well acquainted with these meetings. You analyse what you did well, what you didn’t do so well and how to improve next time. This essentially are what these last two letters add to your targets making them grow with you.

Definitions

S is for Specific

Make sure you are clear and concise about your behaviour.

M is for Measurable

Your progress should be tangible. You should be able to clearly see how much you have progressed with quantifiable results.

A is for Achievable

You are only human with so much time, so ensure you can actually reach your target of the behaviour you want to incorporate.

R is for Realistic

Again, that silly little human aspect means that we can’t state that we’ll be able to learn a new programming language in a day because the scope is too broad and there’s just not enough time. Make sure you set yourself goals that are realistic.

T is for Time-bound

In order to measure your progress, it’s good to set a date to work towards. That way you can tell how much you have learned between a set amount of time.

E is for Evaluate

After you have measured your progress, look closer at your results and think why you’ve achieved those specific results. Questions like these will help you think more deeply:

  • What was the approach you took to learning your new behaviour?
  • How much time did you dedicate?
  • Was it the time of day that you chose?

How have these particular decisions affected your behavior being adopted? Could small changes to these make it easier to adopt other behaviours or even commit them to being a habit faster? Give this stage a bit of time to come to useful conclusions.

R is for Re-evaluate

The last stage is Re-evaluate.  After analysing how your “behaviour to habit” building exercise has worked you must apply anything you have learned to your next behaviour to try and achieve the best results possible.

Start learning your new behaviour

As well as working to the SMARTER guide above, I would recommend you try and keep in mind the points below. This should help you turn your behaviour into a habit.

Limit your chances to be distracted

Set yourself up in environments where you’re less likely to procrastinate or get distracted. Unfortunately, due to the invention of the smartphone, we have a device capable of distracting or keeping us entertained most of the day. But at the times where you should be doing something productive, actively put your device on silent, hide it away or turn it off for the duration that you need to ensure you stay focused.

Make the sessions short

When you’re trying something new you should first try and introduce it in small increments. Depending on what you’re trying to achieve, think of a sensible and small timescale start with. By starting with small increments it means that it’s not so intimidating and will seem easier to accomplish. You are more likely to perform new activities especially if they’re in small chunks.

Ensure it can be done daily (or at least the majority of the week)

In order to build up your new tasks into a habit, you must perform the action daily. It’s been said that it takes about 21 days for new behaviors to become habits so performing the task everyday will make sure you build this new habit as fast as possible.

Building a new positive habit is a challenging goal, but one well worth it. Good luck setting your 2018 goals!

This post was imported into WordPress in one click using Wordable.

Quiz Scramble Template on Sale!

Earlier this year, I redeveloped my app Werdz Movies from UnityScript into C#.

This was just a first iteration. I redesigned the interface so that the hint is always on screen and I still need to add in the power-ups somewhere. These will eventually become in app purchases, something I never got around to doing in the released versions on the App Store and Google Play.

I designed this new version as a template with basic graphics so that anyone can use it and make it their own. After doing this, I packaged it all up and uploaded it to the Unity Asset Store.

This has been available since the 23rd October but from November 27th to December 8th PST it’ll be ON SALE!

You’ll be able to buy this with a massive discount of 30% off.

So if you’re interested in creating your own version of Werdz Movies maybe with games, historical figures or TV shows trivia, grab this now from the Asset Store. It’s only on sale for a short time so make the most of it.

 

Testing Your App Using TestFlight

Here’s my next post which appeared first on Simple Programmer..

There are many types of testing that you can add at different phases of your project. Your last line of defense is testing within the final stages of the project—where you can catch those last-minute defects that can potentially harm your product’s use, damage its value to the user, and tarnish your company’s reputation as a manufacturer of high-quality items.

Various tools exist to assist you during these final stages. One of the most well-known tools for testing iOS apps is TestFlight.

If you’ve never used TestFlight, this post will be an introduction to the tool and will describe how you can best use it during your project to ensure few to no defects affect your final product.

What is TestFlight?

TestFlight is an online tool that allows you to install and test mobile apps “over-the-air.” Historically, app developers tested their projects on actual devices that had to be connected to the development machine using a wire to get a new build. Over-the-air allows developers to distribute their latest builds to testers without the need to connect to the phones physically. Because of this, new builds can effectively be sent to testers in any part of the world.

Who should use it?

TestFlight is perfect for small businesses that don’t have access to a large in-house testing team. And since it’s free, it allows for more businesses other than just revenue-generating companies to use it.

But it can also be good for enterprise companies with remote testers. The access and ease of distribution to up to 10,000 external users and the ability to gain their feedback is extremely useful to companies of all sizes.

When was TestFlight released?

On December 23, 2010, TestFlight was founded by Benjamin Satterfield and Trystan Kosmynka. At the time of its initial release, TestFlight was a single platform designed with the intention of testing mobile apps for both Android and iOS devices.

In 2012, TestFlight was acquired by Burstly, which raised a considerable amount of capital from venture capital firms in order to develop further features and launch TestFlight Live (a now discontinued service providing the user with real-time analytics and engagement metrics).

In February 2014, Apple acquired Burstly. By March of that year, they terminated support for Android devices.

How has TestFlight changed since being acquired?

Today, TestFlight is one of the many options for you to use to test your iOS app. Although there are alternative systems, a few of these still utilize TestFlight to do their app build distribution, especially since the limit for external testers was increased to 10,000 users. With that number of users, you’ll be sure to receive adequate feedback and get enough testing done to ensure a high-quality level of your app.

Currently, you can use TestFlight only for apps developed for Apple devices, as it’s available only to developers within the iOS Developer Program.

Once signed up for this service, you can distribute your app builds to internal or external beta testers who can provide you with feedback about the app. Along with this service, the TestFlight software development kit also allows developers to receive remote logs and crash reports from testers. Being able to gather data from users when they have experienced an issue can be invaluable, as some defects can be difficult to reproduce. The data from these crash reports could provide you with information to help you track down the issue and fix it before it affects anyone else.

At what stage should you use TestFlight?

Although TestFlight would be great during the earlier phases of development to test your initial builds, you need to upload a release build in order to use TestFlight. Using release builds to perform initial testing on isn’t typical, because they contain less debug information. In the event of a crash, you won’t have sufficient details in order to step through the code, find out what happened, and try to fix it.

Release builds are created when the developer is satisfied that the major bugs have been caught and it’s ready to be a beta candidate build. This is a close representation of the final version that will go to customers. Generally, the build is stable with few to no crashes. Any issues encountered at this phase usually require only minor tweaks, so release builds are better for beta candidate builds.

You can also use TestFlight after release to get live information about your app’s performance and feedback from users. Gaining ratings and reviews once an app is live is challenging, so giving users a way to easily submit feedback may help combat this issue.

What are the benefits and drawbacks of TestFlight?

While the history behind the product’s evolution gives you an idea of its features, there are many reasons why you may, or may not, want to choose TestFlight for your testing phase.

Benefits

Wide range of users
Being able to distribute your app to up to 10,000 external users over-the-air gives you the opportunity to get feedback for your app from a large number of users that you wouldn’t have previously been able to reach.

Having a beta testing group at this maximum amount is most common with game companies, as they have prospective users registering ahead of time to test the latest versions of a game iteration.

For normal apps, this number of testers is very large. I would typically expect a maximum of a couple hundred beta testers. Because remember, the more testers you have, the more test results you have to analyze and feedback you’ll need to review.

Keeping the number of beta testers to something sensible is recommended, no matter what maximum limit TestFlight provides.

Testing on multiple apps at once
You can test up to 100 apps at once with either internal or external users. You can also upload different iOS app builds—watchOS apps, tvOS apps, and iMessage apps—at the same time.

Internal users
Up to 25 members assigned to your team in iTunes Connect can also test your app on up to 30 devices.

Unlike external testers who will have access only to the build that their assigned link points to, internal testers will have access to all builds of your app. If you have multiple versions of your app—for example, a free and paid-for version—the testers could have access to each build of these different versions so you can get one person to test multiple apps.

Quick distribution
The tool allows you to quickly distribute your app to a specific set of beta testers over-the-air immediately. You may choose to build different versions to perform A/B testing of your app with your testers. Being able to send a specific build to a number of users at once will help you gain more results faster.

Easily accessible
TestFlight is integrated within the Apple Developer Dashboard that you need to use in order to distribute your app within the App Store. You only need to switch tabs to gain access; no separate login is required.

Drawbacks

iOS only
If you’re developing a cross-platform tool, this will allow you to test and receive feedback easily only on your iOS build. You’ll have to do additional work if you want to test the Android build.

Extra review needed to distribute via TestFlight
Apps that use TestFlight will require a Beta App Review and must comply with the full App Store Review Guidelines before testing can begin.

This is because your app is effectively going out to a select group of the public, so the Apple team wants to ensure that the build that is distributed is of the same quality as the current releases in the App Store.

Just like with the normal app submission process, if your app has significant changes between versions, you’re required to resubmit it for review. This will add extra time to the last phases of your app’s development before it reaches the public.

After you have completed beta testing, you will need to submit your app for review through the usual iTunes Connect screens.

Limited time for beta testing
Submitted builds are accessible from TestFlight for only 90 days after invitations to the testers have been sent. The app will stop writing after these 90 days, so you’ll have to either update your beta submission or get all of your testing completed within those 90 days. This restriction doesn’t affect your internal testers, only those external to the project.

Restricted to specific builds
Only release builds that are signed with the correct provisioning profile and distribution certificate are allowed to be uploaded for beta review. So the build needs to be signed appropriately in order for it to be built and accepted.

Only for devices running iOS 8 and above
A new iOS version is released every year, but not everyone upgrades and not everyone can upgrade, because Apple chooses to drop support for “old” versions of iOS hardware within about three years. This does slightly limit the reach to your audience, as you have no idea what devices they are using and whether their devices support iOS 8.

No application programming interface for continuous integration/continuous deployment
The upload process has to be carried out manually, as there is no way to integrate continuous integration, delivery, or deployment tools into iTunes Connect. So this will slightly slow your progress during testing.

Is TestFlight for you?

Testing your app is a fundamental step in the process of developing one. Without testing, you’re blindly throwing your creation into the wild with no assurance that it will even work for your potential customers. TestFlight is a robust tool for iOS testing that’ll help you manage your beta testing stage effectively and get it out to potentially thousands of users.

I can completely understand the excitement to develop your idea into something tangible, but you must keep in mind that the goal is to create a great app, not simply to distribute anything that resembles your idea into the app stores.

To do this well, you need to remember that integrating thorough testing (even if you’re a team of one) into your development is essential for building a robust product and successful project. And gaining feedback from beta testers via TestFlight before your product is opened up to everyone could reveal some key insights.

This testing could change some big features and could possibly put your app on the course to becoming extraordinary. So while there are several drawbacks to using TestFlight, I think the positives ultimately outweigh the negatives and make it worth considering TestFlight as a resource for beta testing your next app.