A Beginner’s Guide to Testing Blockchain Applications

Here’s my next post which appeared first on Simple Programmer..

During the last few months of 2017, bitcoin and other cryptocurrencies were being talked about by some media sources on a daily basis. These currencies, which had been around for years, were suddenly experiencing major growth. For example, the price of bitcoin has grown from just under $750 in January 2017 to $5,856.10 in mid-October 2017.

This massive growth made it a major talking point. So, as these newer currencies came more into the public domain, the technology behind cryptocurrencies was also beginning to gain attention. This technology is the blockchain.

Blockchain applications are being adopted by some of the biggest industries around the world. Because of the nature of blockchain applications, it further supports how important testing and testers are, and that the field will be more highly thought of and sought after in the future. So as testers, it’s only right to wonder how this new technology will affect your day-to-day work, new opportunities, and current skill set.

What new tools will you have to learn to test blockchain applications, and what skills are needed to test them?

Here, I will outline what the blockchain technology is and how we, as testers, can prepare ourselves for testing blockchain applications.

What Is Blockchain?

Blockchain is a data structure that exists in many locations at once. You can only add to the blockchain. No deletions or updates are permitted. The data held within a blockchain is decentralized, which means a copy of the existing blockchain is on every machine in the network.

Additions to the blockchain can be seen on every computer in that network and the transactions are cryptographically linked to the previous transaction. Therefore, carrying out fraudulent transactions is very difficult to perform. To do this, someone would have to rewrite their history to the beginning of time, which is extremely resource-heavy.

In order to update every machine in the blockchain, the machines have to sync to have a common history. Although all machines will eventually have the same data because of this syncing action, only the more recent transactions are synced more often.

For a new transaction to be added, the decision is reliant on the majority of participants in the blockchain. Once the transaction’s authentication has been validated, the new block is added to the blockchain.

What Is It Used For?

Currently, blockchain is used mainly by the financial and automotive industries because of its highly secure structure. As I mentioned earlier, it’s also the technology that underpins cryptocurrencies like bitcoin and Ethereum.

Not all blockchain technology is open-source. There can be private blockchains like the ones used in banking systems.

What Types of Tests and Techniques Can You Perform on Applications Built on Blockchain?

There are many different types of tests that can be performed at the various stages of developing software projects. Below are a few types of tests that can be utilized to ensure a high level of test coverage and quality for blockchain applications.

Unit Tests
Unit tests help developers ensure their code is performing correctly at the lowest levels and smallest parts of functionality. This should always be the first line of defense to ensure an application catches the majority of bugs early during development.

Integration Tests
Integration tests help developers and test engineers ensure the communication of their code between different components, and possibly between internal and external systems like databases.

User Interface
User interface (UI) testing uncovers how the application works from the end user’s perspective. It’s important to ensure that you carry out UI testing to make sure their experience is positive or that they at least get the correct feedback from the application when it doesn’t perform well.

Application Programming Interface
Application programming interface (API) testing gives you confidence that you’ve validated the responses that your application receives from external APIs and makes sure the formats of your API requests are correct and being handled correctly.

With blockchain applications, there is also a similar type of technology to APIs that allows you to adopt the same testing practices for APIs. These are called smart contracts.

What Are Smart Contracts?

Smart contracts are a big part of the validation technology within a blockchain. A smart contract is a “set of rules in the form of programmable constructs that are capable of automatically enforcing themselves when predefined conditions are met.” For example, a precondition could be transactions trying to append to a specific ledger will undergo additional validations or go through a different set of validations that are more robust.

Although a smart contract is very similar to an API, whereby it has public functions that can be called by anyone registered on that blockchain network, it cannot call external web APIs.

So why do I think that testers are extremely important when testing blockchain applications over other types of systems? Simply because once a contract is deployed to a blockchain, it can never be changed. So you have to be very confident the testing that’s performed is of a high level of quality and that everything that should be covered has been covered.

If a defect is found in production, then a new version of the contract has to be created and deployed. New versions of existing contacts can’t simply get the existing data transferred in; you have to manually initialize the previous data with the new contract.

Updating a contract and rolling back an update is also not a viable option; this increases the complexity of development and means that the importance of implementing and running unit and integration tests on your application before it reaches production could save you major time and money in rectifying defects.

What Skills Do Testers Need for Blockchain Applications?

Although blockchain applications are relatively new in software development, I don’t think testers need to adopt new skills in order to test this type of technology.

Some of the skills I’ve highlighted below are natural skills of good testers or simply skills that you learn early on in your testing career, which grows with your experience in the field.

Critical Thinking
The ability to critically analyze and think about and around a problem is a timeless skill for testers and will be even more sought after for testing blockchain applications.

Testers think about problems such as: Will transactions still execute if x, y, and z are not done? What happens if the network has lots of transactions waiting to confirm? What feedback is given to the user in these cases? Should this be the feedback given to the user? Or is this feedback exposing any security risks?

Another thing to consider if embarking on a new project is to question whether blockchain is the best technology for your use. It’s the new shiny toy, so everyone will want to adopt it, but it may not be suitable for what you want to achieve.

Things to keep in mind are compliance issues; for example, you shouldn’t store health or criminal records, as there are no deletions allowed. So, when criminal records for minors can be wiped, you won’t be able to do so with this technology.

Test Design Techniques
During the ISTQB-BCS Foundation Software Testing syllabus, you are introduced to test design techniques. Knowing even the basics of test design techniques, like boundary value analysis and equivalence partitioning, will make sure you are constantly thinking about and checking the inputs and outputs of the application.

Things to consider could be: How will the application act when you input values that are within, on the edge, and outside of the boundaries of acceptable values? Will the transaction complete? If not, what type of error will be returned? Is this error code correct for the type returned? Should it return anything at all?

Automation
Strong automation skills in all languages, either for lower-level unit, mid-level integration, or API or high-level UI tests, are good skills that can be transferred to testing blockchain applications. Having a solid foundation of automated tests will be needed to ensure that the majority of issues are found early in development.

Manual UI Testing
If a solid foundation of automated tests is put in place, testers can focus on the outlier issues that can more easily be found via exploratory testing performed manually.

Being able to work independently, investigating different areas of the application, trying to find weak areas, and being able to successfully reproduce these are always skills that great testers will need. Despite the world looking to automation to perform a lot of the repetitive and arduous tasks, manual testing skills are still something to hone and improve.

Learn New Tools Quickly
As new technology comes to light, the list of tools to test this technology will also grow. You’ll need to be able to learn how to use these new tools quickly and judge which is the best for what you’re trying to accomplish.

Future-Proof Yourself and Prepare for Blockchain

Hopefully by now, you have a better understanding of the blockchain technology and no longer think it’s as scary as your first thought.

I’m sure if you’re a tester, you already have the foundations of the skills I’ve outlined above. My advice is to push further into an area that you have an interest in and possibly try to improve in the areas where you are weakest to give you the best chance to grow your skills when testing blockchain applications.

Building a Test Pyramid from the Bottom Up

I was brought into my current role at Immerse.io to uphold a high level of quality for the company’s awesome products. But where to start?

After I put together my test strategy I needed to know how much test coverage was currently in place.

Plenty of manual testing was taking place but it needed refining as not everything needed to be tested manually. Also, there were no test cases for the new system documented or being run so there was no way to know how much had been tested, what test was last run let alone what results those tests gained.

So with no test cases to look at, I turned my attention to the developers and their unit tests. The good news is that they had unit tests, the bad news is that they didn’t know how much. There were no code coverage tools integrated into their builds so they couldn’t tell.

So I first turned my attention to increasing unit tests in the projects and adding code coverage tools. Once we had the code coverage levels, we could work to increase the code coverage of the project codebases. If our code coverage level was high, it would give us more confidence about the quality of the system at a low level. But how many test cases should we create?

With this question in mind, I started thinking how I could use the Test Pyramid concept and apply it across our products.

What is the Test Pyramid?

The Test Pyramid is a concept that is used to determine what type of tests and how many of each type should be used at different levels within your product’s development. There are usually three different levels to the pyramid (although some depictions may contain between three to five). The image below is referenced from Martin Fowler’s blog post regarding the Test Pyramid.

As you go up in the levels there should be less tests. Also, as you progress up the pyramid, the tests become more complex because they involve connecting to more than one component or system. Therefore tests will take longer to run as you progress higher.

The Different Levels

The bottom level contains the majority of your tests. Unit tests are used for this level as run fast, have a small focus, are easy to add to and require little maintenance. Overall, they provide fast feedback on changes to the system at a low level.

The middle level contains your integration tests and the top level is for UI tests. The tests in the middle and top levels tend to be more susceptible to breaking because they connect with a number of components and systems.

UI tests are particularly brittle because they rely on web page elements which can be changed easily. Because the automation code is so tightly coupled to the page elements, if one is updated without the other, the tests will break. And, as these two codebases are usually maintained by different teams, sometimes the teams won’t remember to update one another. This could lead to one codebase being updated without the other one. These tests usually require more maintenance than unit and integration tests.

Fix issues as early as possible

The Test Pyramid supports the idea that a lot of thorough testing at the low levels and early phases of development helps to prevent bugs reaching the production environment.

Catching issues within your product during the later stages when the project is completed and ready for consumers is costly and can be more time consuming to fix. Attempting to apprehend issues, before they become defects i.e. production bugs, during the early phases of development is preferred because:

  • It involves less people to fix so can be presumed to be less time consuming
  • There is no impact to the customers so it’s less costly to the business

In your project, try and do the following when focusing on building your base level:

  • You should try to do the majority of your testing during the earlier phases of development with tests that are the cheapest to run.
  • It’s important to make sure that the amount of unit tests in your project is large enough to ensure that issues are caught and fixed before shipping.
  • Make sure you monitor the level of code coverage you have in your project to make sure you have the right amount to reach your target level of quality that will provide you with confidence in your product.

Follow the Test Pyramid and see what results you get.

Integrating Code Coverage

In order to follow the Test Pyramid concept, we first needed to build up the amount of tests in the base level.

Now that the tests were being added to, we needed them to be integrated within the CI tools we were using. So for one team, I set about getting these running within TeamCity. Using the Unity command-line test runner I managed to get this working when we were using Unity 5.4. Unfortunately, we found that this set up broke when we upgraded to Unity 5.6. Luckily, this has now be rectified in version of Unity 2017.2 and we have unit tests running in TeamCity on every build once again.

Unfortunately, there’s no way to measure code coverage on Unity projects so we’re just going have to be content with increasing the number of tests and making sure that these tests are meaningful. I’ll continue to monitor this though.

I next looked at getting a code coverage tool installed for the other team. Because of the tech stack here and how simple it was to setup on a simple project, I chose to use Istanbul. This has now been set up into a CI tool that runs on every push so that all code is checked and a code coverage report is produced. This gives us an idea of where we need to focus our efforts to increase the amount of unit tests.

Our Next Steps

As the production of unit tests are continuing and the developers are actively doing this themselves, I turn my attention to the next level of the Test Pyramid, the Integration tests.

This post was exported in one click from Google Docs into WordPress with Wordable.