Element is not clickable at point (x,y). Other element would receive the click…

I’ve been extending an automation suite that will be used to run UI tests on the platform in several environments to help us verify the build and highlight any breaking changes early.

Running these tests will also free us up to concentrate on tests that weren’t covered by automation and exploratory tests.

This particular suite is written in Ruby with Capybara and RSpec. This was a the first time that I had used Ruby to write an automation framework so it took a bit of time to get used to. Some of the issues that I was coming up against were annoying as I knew they were easily solvable in Selenium using C# but I persisted and eventually overcame them. 

I had most of the high priority tests passing apart from one. When Googling the answer, the only thing that I was coming up with was that the window size wasn’t big enough. This was annoying me because I had passed the Chrome Options parameter of “start-maximized” and, when not running in headless mode, the Chrome window was indeed maximised.

After a long day of searching and testing alternatives, I finally came across this post.  So in fact, the issue was only occurring in headless mode because when running tests in this mode, despite setting the browser to start maximized using the Chrome Options, it defaults to 800×600. So, I then set the Chrome Options window-size parameter to “1980,1080” to override this. And voila! The test passed in headless mode.

So, I would suggest if you want to run your tests in headless mode that you also pass in a specific size for the window so you don’t get caught by this too.

 

 

 

Career Mentors (Bye Jay)

Today my manager Jay is leaving. It was a massive shock when I found out. As he’s the CTO I had two thoughts:

  1. “Oh no, I’m sad now”
  2. “Is the company in trouble?”

So in fact, he’s making the decision to go purely because he wants a new challenge which is completely understandable and I wish him the best of luck.

He has been the best manager I have had so far because he’s been supportive, encouraging, and an ear when I need to moan (which can happen often as I am very picky particular dedicated to delivering a great product). He also doesn’t just tell me things I want to hear (which is pretty rare in people). I get honest answers and opinions.

Being a woman in tech, I didn’t have many people to look up to in order to steer me in the right direction, getting me to where I am now. I have done it all by myself (which makes me proud now that I think about it) but it’s been exhausting.

I haven’t had someone else to give me their wisdom which would’ve undoubtedly meant that I probably would’ve made less mistakes along the way. But from the beginning Jay has given advice, let me feel like I can research everything that will help improve the team and the area that I work in for the company, has given me autonomy, let me attend the events that I want (through which I have met some great people), sends me Twitter links to awesome women in tech (like @TheAmyCode) for further encouragement, and in general has made me feel more confident about being in this role and that I deserve this role because I am awesome.

Find A Career Mentor

It’s important to find people like this to guide you through your career because no matter whether you’re male or female, careers are hard and a little guidance goes a long long way.

If you do find these people, make sure that it’s not just a one-sided relationship. Make sure that there’s something that you can give them too. Whether it’s small things like being self-sufficient so that they don’t need to worry about you or giving back something tangible that could help make their lives easier or better.

So, bye Jay. See you soon, it’s been great working with you and remember BBQ!

 

How to Future-Proof Your Testing Career

Here’s my next post which appeared first on Simple Programmer..

Since I switched back into testing from a development role, I’ve been trying to build up skills that will future-proof myself. To do this, I’ve been going to more talks and conferences, and watching webinars on testing.

What do I mean by future-proofing? I want to ensure that I have the correct skills, either by focusing on one specific area or gathering a broad knowledge of a few keys areas, that may be popular or emerging in technology now or in a year or two. Technology has changed so much even in the last 10 years, so it’s one area that I think you should stay up-to-date on the trends.

Earlier this year, I watched a webinar called 2018 Test Automation Trends. I’m sure all testers have heard of or are using automation tools these days. They have probably heard the fears of some that manual testers are being made obsolete by automation (this is in no way true).

But automation isn’t everything. This isn’t the only area that could help you future-proof your testing career.

Automation is just another area of testing that’s getting more attention. It has become highly visible how important testing is, and how bad products affect end users as well as shareholders.

Although this webinar was about automation trends, at the end, the experts were asked what they thought would be things to watch out for in 2018. I believe that if you want to stay ahead in this field, you need to look ahead too.

So here are the topics that these experts recommend to focus on in 2018, and also some that I think are good areas to begin future-proofing yourself as a tester.

Blockchain

Blockchain is a data structure that exists in many locations at once. You can only add to the blockchain. No deletions or updates are permitted. The data held within a blockchain is decentralized, which means a copy of the existing blockchain is on every machine in the network.

Additions to the blockchain can be seen on every computer in that network and the transactions are cryptographically linked to the previous transaction. Therefore, carrying out fraudulent transactions is very difficult to perform. To do this, someone would have to rewrite their history to the beginning of time, which is extremely resource-heavy.

With Blockchain applications, once a contract is deployed to a blockchain, it can never be changed. So you have to be very confident the testing that’s performed is of a high level of quality and that everything that should be covered has been covered. There is no rolling back.

For more information, I wrote a blog post to introduce testers to blockchain applications. Have a read and see what you think about blockchain.

Understanding blockchain technology and how to test it will help future-proof your testing career. It will help enable you to move into different fields, like banking and the automotive industry, which are both currently using this technology. It will give you more options about where you will work.

Contracts and API Testing

Contracts and application programming interfaces (API) testing focus on testing API endpoints of services and microservices.

There are different types of services. One example is a web service, which provides a way for applications and systems like databases, mobile apps, and other web servers to communicate with the service over the internet using endpoints. Endpoints are URLs that your HTTP client interacts with in order to access the data from that URL.

When testing endpoints, you ensure that when requests are sent with the correct parameters or incorrect parameters, the system behaves in a way that you expect. For example, if you send three parameters to an endpoint and the endpoint requires four, then you would expect a response to be returned with a 400 status code.

The services architecture is widely adopted across many industries, because it allows code to be written in a more modular way. This allows the project to be easily extendible, and will also mean that systems are coupled less tightly together.

As a tester, you need to ensure that you can test endpoints and contracts thoroughly. Incorrect requests could mean that your application doesn’t respond with the correct status code or message for the user in the best case. But in the worst case, it could lead to holes in your application’s security.

As testing jobs require more knowledge about different ways to automate their testing code, learning how to implement API tests and perform contract testing will help future-proof your testing career. You’ll become an expert in a specific area of automation, and as more applications, particularly web-based ones, are built using the service architecture, your skill to competently test the integration layer will be very sought after.

Mobile Testing

Smartphones and tablet devices have become a staple product within homes all over the world, for those young and old. It’s just as important to have a well-tested app nowadays as a well-tested website.

In 2005, Google changed its search results ranking algorithm so that if a website wasn’t responsive, that site would be ranked lower in its results list. The dedication to and sophistication of mobile testing has massively increased since then.

We first began testing mobile websites and apps manually, but because of the fragmentation of Android and the yearly releases of all mobile OSs, the need to automate mobile testing has increased. Because of this need, there has been a number of mobile testing frameworks released to aid testers.

Appium is a mobile testing framework built on Webdriver technology. It was created in 2013 by Dan Cuellar, a test manager at Zoosk who found that the length of the test passes on the iOS product was getting out of hand. Appium allowed him to write automated tests for iOS as easily as Webdriver was used for automating websites.

I recently went to the first Appium Conference held in London to learn about how I could utilize Appium.

Knowing how to effectively test mobile apps and websites is now a full-time job, so tools like Appium are used to help mobile testers test thoroughly. Mobile devices are everywhere now and the development of applications will only grow, so I believe it would be a great skill to add to bolster your testing skill set.

Artificial Intelligence (AI)

AI is where scripts, machines, or anything that’s not human perform tasks that may or may not be complex.

Within AI there are a number of different areas. One that’s becoming more common now is machine learning (ML). Machine learning is when AI provides you with answers or data based on previously generated data. It takes all of the information gathered and makes predictions on future outcomes or decisions based on this historical data.

One well-known example of ML and AI is the AlphaGo Zero software from Google’s DeepMind company that taught itself to play Go. Within three days, it was able to beat the previous version of itself (and that version bested 18-time world champion Lee Se-dol).

In testing, we can use ML to predict outcomes of test cases by looking at historical test result data. It can be used to predict which test cases should be prioritized over others when certain parts of the system have been updated. It can tell you when a test case was last run so you can determine what test cases are the most important to be run.

We’re constantly investigating how to become more efficient, save time, and complete jobs faster. AI can aid in this, so it’s a good idea to know its strengths and weaknesses, and how it can be used to help you with your testing career.

We’re at the very early stages of AI and how it can be utilized within daily testing activities,helping us achieve a high level of quality of our products. Ensuring that you keep up with or are ahead of the developments and applications of AI within software testing will help solidify your expertise in the field and help future-proof your career.

Docker and Containers

Containers give us the ability to store a number of different systems virtually within the same location. This means that if one environment needs a website front end, server, and database in order to function, all of these can be stored within the same place.

For testers, containers provide a quick solution to deploy test environments that are stable, fast to set up, and a mirror-copy of staging or production environments.

Learning the strengths and weakness of the architecture that your environments are built on will mean that you know how your application can be affected and whether certain types of tests will be more suitable to ensure a higher level of quality.

For more information, I’ve written a post about containers and how we use them at my current company.

Learning about Docker and the containers technology, and how to test it, will help future-proof your testing career. Since it allows quick deployment and scaling of infrastructure, it’s being widely adopted by a number of DevOps teams in different industries. This means you, again, will give yourself more options when looking for your next job opportunity.

Some Suggestions From Me

After listening to the expert’s suggestions, I’ve come up with a short list of additional areas that I think would definitely help bolster your testing skills arsenal.

Debugging

Debugging is where you step through code in order to locate defects. This isn’t something that every developer or engineer likes to do. Having the patience to walk through code can be an overlooked skill.

Debugging can take a few minutes if you know what’s wrong and where it is, but it could sometimes take days to figure it out—particularly if the code base is made up of hundreds of files.

Debugging also makes you practice reading the code, and it will help you develop a further and deeper understanding of the code base you’re stepping through.

This skill is useful for developers and is great for testers to future-proof themselves, especially if you are developing test code in any capacity. But since it’s not one that’s actively trained, you should invest your own time into debugging code to further narrow down where in the codebase the defects can occur.

You can do this by reading log files of applications when errors occur, and tracing them through the code base to see if you can track it down.

For more information on how to improve your debugging skills, read 5 Steps to a Bullet-Proof Debugging Strategy.

Augmented Reality, Mixed Reality, and Virtual Reality

Devices that provided users with each of these experiences have been around for years. However in 2017, I believe they were accepted and adopted by a larger consumer audience.

As this new tech is adopted more and more by the public, the demand for experienced testers in these areas will increase.

Since it’s still very new, there’s not much in terms of frameworks or testing standards that can be applied yet.Still, this is the time for someone to take that first step and create something that everyone can use going forward.

Cloud Computing

Cloud computing is the on-demand delivery of IT resources like compute, database storage, and applications via cloud service platform over the internet using a pay-as-you-go format.

Platforms like Amazon Web Services (AWS), Google Cloud Platform, and Microsoft Azure are some of the options that are available presently, with AWS storming ahead of the competition.

So why should you as a tester be concerned with what tech your applications are built upon?

Well, it’s the same reason as I mentioned earlier about containers. If you know what tech your application is built upon, you should be able to understand it’s strengths and weaknesses and how that will affect your application.

Therefore, you will be able to test your application better and help close up any holes in security that may be exploited, ensure it’s performant, and that the end user has a great experience when using your product.

How Future-Proof Are You?

When you’re working within a technical sector, the ability to step back and assess the bigger picture of what’s going on in your industry, what things are emerging, what things may be going away, and what’s here to stay will tell you a lot.

It will clearly show you where you should invest your time on new skills, and which ones you may need to strengthen.

If you want to do more in order to future-proof yourself, why not try and go to a talk in one of these areas yourself, read blogs, listen to podcasts, or watch YouTube videos of experts in these fields. You can also get a free subscription to AWS and experiment with the Elastic Container Service to get a basic knowledge of containers.

I hope this post has given you a foundation of what’s to watch out for now in 2018 within the field of testing. There’s a lot of options. So don’t wait—get started.

The Challenges of Leading a Distributed Test Team

Here’s my next post which appeared first on Simple Programmer..

Being a manager of a distributed team of people can be a challenging role. When you have to communicate on a daily basis and those people are in different time zones, or if you’re at the end of the project delivery date, you may find your stress levels slowly creep up higher and higher.

I’ve not only been trying to lead a distributed (albeit small) team for the last year, but I’m also trying to balance my daily testing tasks, researching new tools that could help my team, and maintaining or improving the overall quality of projects across the company.

Let me share with you a few challenges that I’ve faced and how I have or am planning to overcome them in future.

Different Time Zones Between Team Members

It’s important in every project for team members to regularly communicate. It’s easy for details to be missed, especially during busy times in the development process.

When your team are in different time zones, it’s even more important to make sure that they communicate daily. You’ve only got a specific amount of time overlap to talk directly with your team all at once. This time must be utilized and never skipped by anyone.

Ways that your team could communicate could be via the daily stand-up meeting (if you use Agile methodologies), or making sure you are giving daily reports via other channels, like internal communication systems such as Slack or Skype. Whatever medium you use, just ensure that the communication is regular and everyone gets involved.

Lesson: Plan your meetings at times where everyone is available and projects are not busy.

Balancing the Workloads of All Team Members

Even in teams as big as 100 people, it’s important that everyone feels like they are not overwhelmed by the work we have and can also perform their job to the best of their ability.

Having a distributed team also requires you to take into account what time other team members may also be working so you can assign the correct project to the correct people. You can use tools like Trello, Asana, or even a spreadsheet to manage project assignment. Doing this will help those team members more easily manage their workload so the project is more closely aligned with their schedule.

You’ll also need to consider the skills that are needed for a project and how long the project will last. This will help you assign the right team member to the right project. For example, say you had an upcoming project where the test framework needed to be written in C# and some of your team were proficient in Java and some in Ruby. You may find that this type of project will be easier to and faster to implement by members of team know Java as they are quite similar languages.

Lesson: Try and match the right projects to team members with similar skills, interests, and time to carry out the project.

Trying To Manage Everyone and Everything

Despite being a small testing team where I work, we have a continually big workload. We are managing the quality and daily testing for two-to-four projects at once. With my job, carrying out daily testing tasks, building the automation suites, and trying to monitor the overall quality strategy in the company can be a lot to do for one person. But not for me, I’m Superwoman!

Only (and obviously) joking. Seriously, I realized very early on that there isn’t time to do everything, and that I can’t manage everyone and everything by myself. So, to make sure I can carry out all of my tasks, I prioritize my work and delegate lower-priority tasks to other team members.

I split responsibilities between team members according to project deadlines, project requirements, and team member availability. I encourage my team to do the same thing to make sure they also don’t feel overwhelmed and can get their jobs done well.

In a distributed team, this can work to your advantage. For example, if you have team members who are in a time zones hours ahead or behind you, you can leave them work when your overlap time is over. Then when you return, the task may be fully or almost complete, ready for you to focus on something else.

In order to do this, you need to know your team and trust that they will carry out their jobs so that you can focus on the big picture.

Lesson: Get good people to help you manage things.

Trying to Monitor the Progress of Your Strategy

When I first arrived at my current company, one of the first things I was asked to do was outline a Test Strategy for the company.

Every month I try to track if we’re still on our way to achieving those goals I set a year ago, if they need adjusting, or if they’re no longer relevant and some of the strategy needs revisiting.

If all of my team were in one location, it could be as simple as having a quick meeting once a month to find out where we are. But having a distributed team means that this needs to be planned in advance and at times where everyone is available. With some members of my team in time zones as far away as four hours at times, meetings need to be planned at times during the project where each member won’t be busier than usual, like the week before a product delivery.

Utilize free calendar resources, like Google Calendar, to share a team calendar booking in important dates where the team will be busy or unavailable. Use this to regularly track your progress with team meetings.

Lesson: Have a clear idea of your vision and keep track of it being executed.

It’s Easier for Quieter Members to Hide Away

Make sure the quieter members of your team are also vocal at team meetings. Find ways to get everyone engaged with their team. It’s easy for those that are quiet to sit back and let others that are more confident speak up, making themselves be heard, especially if these meetings are over Slack or Skype.

You can do this by directing specific questions to these people or asking them their opinions. This will prompt them to speak up a bit more. If English is not their first language, they may be more inclined not to speak up, but giving them a little nudge by asking a question to them here or there may be all they need to build up their confidence.

Keep in mind that there may be a reason why these people are quiet, so before you ask them questions in meetings with everyone ready and waiting, maybe talk to them privately. Learn who they are and what motivates them, then use that to regularly engage with them.

Lesson: Learn who your team are and how they like to communicate, and encourage them to participate.

It’s Hard to Keep Everyone’s Skills Up

In any technological field, like software development, virtual reality, big data, or artificial intelligence, one of the challenges is to make sure you are constantly learning. Things are always changing and it’s important to keep your skills up. It’s hard to find time to build your skills (especially within testing and QA roles) because at the end of the day, you’re employed to develop and ship a product. Improving your skill set in an area is usually a byproduct of working on something day in day out. But, I would suggest you encourage your team to carve out some dedicated time in their week to learn new skills or improve on existing ones. This not only will keep people in your team, show you care and want to invest in their personal development, but will also make them enjoy coming to work.

Utilizing resources like Pluralsight courses (aimed at those in technical professions), YouTube videos (for everyone and about everything), knowledge sharing sessions, and pair programming are all easy to access. All of these are free to low-cost ways to improve your knowledge and strengthen the skills of your team.

It’s easy to let deadlines let us work through these times, so make sure your team schedule them at times where they have little to no distractions (if at all possible).

Lesson: Tell each team member to calendarize a time slot for themselves to work on skills weekly to improve their knowledge.

The Bottom Line: Have a Great Team

Leading a distributed team as well as all of your other responsibilities as a manager is a challenging task. You shouldn’t underestimate that the amount of work done day-to-day will be the same as when you lead a team all in the same location.

Sharing tasks among your distributed team members, like responsibilities, projects, and even meetings, will help you balance the efforts of your daily and management tasks. But you need a good team to help you with this.

So, make sure you find the right people to help you. In return, invest some time into them by making sure they are not overloaded with work, get to know them by finding out what motivates them, and encourage them to build their own skills by training themselves.

My First (and the) First Appium Conference

Last year, I was selected for a scholarship to Appium Conference 2018 by White October Events. I’ve been aware of Appium for years now as I’m an app developer but I’ve not (yet) used this tool on any of my projects.

For those that don’t know, Appium is a mobile testing framework built upon the Webdriver technology. It was created in 2013 by Dan Cuellar. He was an Test Manager at Zoosk who  was finding that the length of the test passes on the iOS product was getting out of hand. Appium allowed him to write automated tests for iOS as easily as Webdriver was used for automating websites.

Five years later, Appium has a massive community building up a successful open source project that can be used on Android, iOS, Windows in a variety of coding languages.

This year was the first Appium Conference.

There was only one track which was great because I got to see everything and didn’t have to pick between two talks that were probably going to be beneficial to me.

Interesting Things to Note

  • There was a lot of variety of languages being used with Appium (part of the appeal of using the product I suspect).
  • Lots of people were using Jenkins with it as their continuous integration/continuous delivery tool. There were a couple mentions of Circle CI but none about TeamCity. Because of this, I think I’ll be looking into Jenkins more than TeamCity for my app projects.

All the talks were highly interesting and not too difficult to follow for a beginner like me. So I want to share my biggest takeaways from each of the talks.

Keynote – Appium: The Untold Story

The day began with a keynote from Dan Cuellar and Jason Huggins. They spoke about the history of Appium, where it is now and briefly touched upon where he wants it to be in the future and mentioned their vision of StarDriver.

They want to see Appium grow it’s users and the platforms it can test particularly to the internet-of-things and various hardware.

The best takeaway for me from this keynote was the phrase “Challenge everything you see”.

Appium: A Better way to Play the Game

This first talk was given by Charlene Granadosin & Charlotte Bersamin. What I found interesting within their talk was how they were integrating their release and exit reports within Jira using Xray. They used a curl command to upload their latest test results to Jira so these results are clearly visible to the Product Owners or Managers of the team.

My biggest takeaway from this talk was to investigate whether the tools we were currently using for test case management was able to integrate with Jira to give such detailed reports and to try and get the automation up and running.

Deep Hacking Appium for Fun and Profit

Daniel Puterman‘s talk explained how he had contributed to the Appium project by creating a new endpoint to gather native application screenshots.

Because the company Daniel worked with was Applitools, my biggest takeway from this was to figure out whether visual testing tools would be useful for testing virtual reality (VR) applications as much as they would be for websites of mobile applications.

Why the h# should I use Appium with ReactNative?

Wim Selles delved into a comparison talk about why they chose Appium (which was extremely useful as I’ve also been debating what automation tool to use for mobile apps).

Out of all the frameworks he mentioned, they went with Appium because it fit with a lot of his requirements for testing ReactNative apps.

There were a lot of takeways for me from this talk.

  • Consider your own project requirements when you pick your automation tools
  • What are your requirements?
  • What should your app do (now/future)?
  • Which tool supports your needs/expectations?
  • Do a proof of concept test
  • Do research into competitive tools

He also gave a couple of good ways that you can speed up app testing:

  • Remove animations on screens
  • Utilise deep-linking to get directly to screens

Layout Automation Testing (GUI)

Prachi Nagpal explained how she was using Galen to perform her UI browser based testing for mobile and desktop devices. Galen does this by measuring the distance between elements that are being tested. You can also produce heatmap results from this tool.

It was a good talk and interesting to see a tool that I had never heard about.

Can you please provide the full Appium Server logs? A Brief Tour of the Logs

Isaac Murchie next walked us through the Appium logs and the takeaway here was that he pointed out that some lines are bolded to highlight their importance.

He also noted that the following are ways to know which are requests and responses in the logs.

This is a request:

[HTTP] -> 

This is a response:

[HTTP] <-

Interaction with Native Components in Real Devices

In his talk, Telmo Cardoso told us how he tested native components of mobile operating systems. He explained the challenges that he faced (some tasks were difficult on one platform but easier on the other) and ways he and his team had got around them.

The areas he found challenging were:

  • Adding contacts
  • Pushing files to a device
  • Push notifications
  • Managing calls
  • Simulating low battery
  • Extracting logs

The biggest takeaway from this talk was that he used Cucumber for his automation framework along with with Appium successfully to test the native applications and features of smartphones and not just the applications running on them.

Using Appium for Unity Games and Apps

Because of my daily work with Unity projects, I was particularly looking forward to Ru Cindrea‘s talk on how she used Appium for her Unity games and apps.

She first explained how she used OpenCV an image recognition tool with Appium to try and test her Unity games.

The positives were:

  • Works for simple scenarios
  • No changes to game required
  • Found issues like performance issues or out of memory crashes

The negatives were:

  • Wasn’t fast enough
  • Not for games with lots of text

So she decided to create a component called AltUnityTester to help her with her issues.

AltUnityTester is created with Python bindings. It opens a socket connection and waits for a response on a specific port.

When the AltDriver is added into the Appium project it gets a list and knows everything about that Unity scene’s objects. It can then send a command to that port to get information back from the scene to perform tests e.g. checking the end position of elements or text output.

This solution is useful because it’s real-time but it does require changes to the project and it only works with Unity.

So my biggest takeaway was to investigate whether this AltUnityTester could be extended or something similar made in order to test VR applications using Unity.

It’s available as a Unity package or on Gitlab.

Docker-Android: Open-source UI Test Infrastructure for Mobile Website and Android

Application

Budi Utomo next talked about his Docker-Android image to test Android projects and websites on Android devices.

His plan for project development was to:

  1. Create UI tests for Android devices
  2. Write unit tests on Android
  3. Create UI tests on Android apps
  4. Implement Monkey/Stress tests

The biggest takeaway was the demo that showed how Appium can be used easily within Docker containers.

Application Backdoor via Appium

Rajdeep Varma explained how you can use Appium scripts to call development code methods from test code.

He used Appium in this way because he was having a number of problems when trying to write tests:

  • System pop-ups were called and were not needed when running tests
  • Driver limits e.g. mocking the device has been shaken or changing time limits
  • Tests were slow to run

This is where he is using backdoors and where he thinks they could also be used:

  • Changing backend URLs
  • Changing app locale
  • Getting session ids from the app
  • Disabling “What’s new” pop-ups
  • Disabling client side A/B tests
  • Faking SIM card for payments
  • Get analytics data from the app to validate it

The biggest takeaway from this talk was to be careful not to use backdoor for every test case and call incorrect methods in order to make tests pass.

Mobile Peer 2 Peer Communication Testing

Canberk Akduygu gave us a talk about his challenges when automating the BIP app (it’s like the Turkish Whatsapp).

He was building an extended grid solution to change to the right version of Appium and set the desired capabilities within their testing framework according to properties set in a JSON config file.

His demo was the biggest takeaway which showed two phones messaging and even calling the other. It was one scenario that was running two different steps on each device. It showed that the test steps needed to be synchronised.

From a Software Tester to an Entrepreneur: What I’ve Learned

Kristel Kruustuk next came on stage and walked us through why she founded Testilio and her struggles with the company despite being so successful and growing at a fast pace since she began.

My takeaway from this talk to was investigate and get in touch with the Testilio team and see if they had any future plans for expanding from manual and automation testing to VR testing.

Appium: The Next Five Years

Jonathan Lipps gave the final talk and began again with the history of Appium but then spoke more in depth about the vision of the product over the next five years and hopeful milestones. Some of these things were:

  • StarDriver
  • InfinityDriver
  • Extension to the W3C WebDriver protocol
  • Node js base classes and libraries for easily writing new drivers

Closing Remarks

Lastly, we were treated to a performance featuring Jonathan Lipps, Appium and Selenium. I believe four or five instruments were being played by Appium, Selenium was outputting the lyrics and Jonathan was singing and playing the ukulele.

My biggest takeway from this is that Appium can be used in a variety of ways to perform a number of impressive tasks.

After Party

The after party was held at a bar a short walk from St Paul’s Cathedral. There I managed to talk to Ru Cindrea in more detail about the project I wanted to use the AltUnityTester for and whether she thought it would work. I also, managed to talk the ears off of both Charlene and Charlotte who were the speakers from the very first talk of the day.

Overall

I had a great day, met loads of wonderful people (including attendees and speakers) and I hope that next year, I’ll have begun using Appium for something that I do so I can share my own experiences with the community.

A Beginner’s Guide to Testing Blockchain Applications

Here’s my next post which appeared first on Simple Programmer..

During the last few months of 2017, bitcoin and other cryptocurrencies were being talked about by some media sources on a daily basis. These currencies, which had been around for years, were suddenly experiencing major growth. For example, the price of bitcoin has grown from just under $750 in January 2017 to $5,856.10 in mid-October 2017.

This massive growth made it a major talking point. So, as these newer currencies came more into the public domain, the technology behind cryptocurrencies was also beginning to gain attention. This technology is the blockchain.

Blockchain applications are being adopted by some of the biggest industries around the world. Because of the nature of blockchain applications, it further supports how important testing and testers are, and that the field will be more highly thought of and sought after in the future. So as testers, it’s only right to wonder how this new technology will affect your day-to-day work, new opportunities, and current skill set.

What new tools will you have to learn to test blockchain applications, and what skills are needed to test them?

Here, I will outline what the blockchain technology is and how we, as testers, can prepare ourselves for testing blockchain applications.

What Is Blockchain?

Blockchain is a data structure that exists in many locations at once. You can only add to the blockchain. No deletions or updates are permitted. The data held within a blockchain is decentralized, which means a copy of the existing blockchain is on every machine in the network.

Additions to the blockchain can be seen on every computer in that network and the transactions are cryptographically linked to the previous transaction. Therefore, carrying out fraudulent transactions is very difficult to perform. To do this, someone would have to rewrite their history to the beginning of time, which is extremely resource-heavy.

In order to update every machine in the blockchain, the machines have to sync to have a common history. Although all machines will eventually have the same data because of this syncing action, only the more recent transactions are synced more often.

For a new transaction to be added, the decision is reliant on the majority of participants in the blockchain. Once the transaction’s authentication has been validated, the new block is added to the blockchain.

What Is It Used For?

Currently, blockchain is used mainly by the financial and automotive industries because of its highly secure structure. As I mentioned earlier, it’s also the technology that underpins cryptocurrencies like bitcoin and Ethereum.

Not all blockchain technology is open-source. There can be private blockchains like the ones used in banking systems.

What Types of Tests and Techniques Can You Perform on Applications Built on Blockchain?

There are many different types of tests that can be performed at the various stages of developing software projects. Below are a few types of tests that can be utilized to ensure a high level of test coverage and quality for blockchain applications.

Unit Tests
Unit tests help developers ensure their code is performing correctly at the lowest levels and smallest parts of functionality. This should always be the first line of defense to ensure an application catches the majority of bugs early during development.

Integration Tests
Integration tests help developers and test engineers ensure the communication of their code between different components, and possibly between internal and external systems like databases.

User Interface
User interface (UI) testing uncovers how the application works from the end user’s perspective. It’s important to ensure that you carry out UI testing to make sure their experience is positive or that they at least get the correct feedback from the application when it doesn’t perform well.

Application Programming Interface
Application programming interface (API) testing gives you confidence that you’ve validated the responses that your application receives from external APIs and makes sure the formats of your API requests are correct and being handled correctly.

With blockchain applications, there is also a similar type of technology to APIs that allows you to adopt the same testing practices for APIs. These are called smart contracts.

What Are Smart Contracts?

Smart contracts are a big part of the validation technology within a blockchain. A smart contract is a “set of rules in the form of programmable constructs that are capable of automatically enforcing themselves when predefined conditions are met.” For example, a precondition could be transactions trying to append to a specific ledger will undergo additional validations or go through a different set of validations that are more robust.

Although a smart contract is very similar to an API, whereby it has public functions that can be called by anyone registered on that blockchain network, it cannot call external web APIs.

So why do I think that testers are extremely important when testing blockchain applications over other types of systems? Simply because once a contract is deployed to a blockchain, it can never be changed. So you have to be very confident the testing that’s performed is of a high level of quality and that everything that should be covered has been covered.

If a defect is found in production, then a new version of the contract has to be created and deployed. New versions of existing contacts can’t simply get the existing data transferred in; you have to manually initialize the previous data with the new contract.

Updating a contract and rolling back an update is also not a viable option; this increases the complexity of development and means that the importance of implementing and running unit and integration tests on your application before it reaches production could save you major time and money in rectifying defects.

What Skills Do Testers Need for Blockchain Applications?

Although blockchain applications are relatively new in software development, I don’t think testers need to adopt new skills in order to test this type of technology.

Some of the skills I’ve highlighted below are natural skills of good testers or simply skills that you learn early on in your testing career, which grows with your experience in the field.

Critical Thinking
The ability to critically analyze and think about and around a problem is a timeless skill for testers and will be even more sought after for testing blockchain applications.

Testers think about problems such as: Will transactions still execute if x, y, and z are not done? What happens if the network has lots of transactions waiting to confirm? What feedback is given to the user in these cases? Should this be the feedback given to the user? Or is this feedback exposing any security risks?

Another thing to consider if embarking on a new project is to question whether blockchain is the best technology for your use. It’s the new shiny toy, so everyone will want to adopt it, but it may not be suitable for what you want to achieve.

Things to keep in mind are compliance issues; for example, you shouldn’t store health or criminal records, as there are no deletions allowed. So, when criminal records for minors can be wiped, you won’t be able to do so with this technology.

Test Design Techniques
During the ISTQB-BCS Foundation Software Testing syllabus, you are introduced to test design techniques. Knowing even the basics of test design techniques, like boundary value analysis and equivalence partitioning, will make sure you are constantly thinking about and checking the inputs and outputs of the application.

Things to consider could be: How will the application act when you input values that are within, on the edge, and outside of the boundaries of acceptable values? Will the transaction complete? If not, what type of error will be returned? Is this error code correct for the type returned? Should it return anything at all?

Automation
Strong automation skills in all languages, either for lower-level unit, mid-level integration, or API or high-level UI tests, are good skills that can be transferred to testing blockchain applications. Having a solid foundation of automated tests will be needed to ensure that the majority of issues are found early in development.

Manual UI Testing
If a solid foundation of automated tests is put in place, testers can focus on the outlier issues that can more easily be found via exploratory testing performed manually.

Being able to work independently, investigating different areas of the application, trying to find weak areas, and being able to successfully reproduce these are always skills that great testers will need. Despite the world looking to automation to perform a lot of the repetitive and arduous tasks, manual testing skills are still something to hone and improve.

Learn New Tools Quickly
As new technology comes to light, the list of tools to test this technology will also grow. You’ll need to be able to learn how to use these new tools quickly and judge which is the best for what you’re trying to accomplish.

Future-Proof Yourself and Prepare for Blockchain

Hopefully by now, you have a better understanding of the blockchain technology and no longer think it’s as scary as your first thought.

I’m sure if you’re a tester, you already have the foundations of the skills I’ve outlined above. My advice is to push further into an area that you have an interest in and possibly try to improve in the areas where you are weakest to give you the best chance to grow your skills when testing blockchain applications.

Building a Test Pyramid from the Bottom Up

I was brought into my current role at Immerse.io to uphold a high level of quality for the company’s awesome products. But where to start?

After I put together my test strategy I needed to know how much test coverage was currently in place.

Plenty of manual testing was taking place but it needed refining as not everything needed to be tested manually. Also, there were no test cases for the new system documented or being run so there was no way to know how much had been tested, what test was last run let alone what results those tests gained.

So with no test cases to look at, I turned my attention to the developers and their unit tests. The good news is that they had unit tests, the bad news is that they didn’t know how much. There were no code coverage tools integrated into their builds so they couldn’t tell.

So I first turned my attention to increasing unit tests in the projects and adding code coverage tools. Once we had the code coverage levels, we could work to increase the code coverage of the project codebases. If our code coverage level was high, it would give us more confidence about the quality of the system at a low level. But how many test cases should we create?

With this question in mind, I started thinking how I could use the Test Pyramid concept and apply it across our products.

What is the Test Pyramid?

The Test Pyramid is a concept that is used to determine what type of tests and how many of each type should be used at different levels within your product’s development. There are usually three different levels to the pyramid (although some depictions may contain between three to five). The image below is referenced from Martin Fowler’s blog post regarding the Test Pyramid.

As you go up in the levels there should be less tests. Also, as you progress up the pyramid, the tests become more complex because they involve connecting to more than one component or system. Therefore tests will take longer to run as you progress higher.

The Different Levels

The bottom level contains the majority of your tests. Unit tests are used for this level as run fast, have a small focus, are easy to add to and require little maintenance. Overall, they provide fast feedback on changes to the system at a low level.

The middle level contains your integration tests and the top level is for UI tests. The tests in the middle and top levels tend to be more susceptible to breaking because they connect with a number of components and systems.

UI tests are particularly brittle because they rely on web page elements which can be changed easily. Because the automation code is so tightly coupled to the page elements, if one is updated without the other, the tests will break. And, as these two codebases are usually maintained by different teams, sometimes the teams won’t remember to update one another. This could lead to one codebase being updated without the other one. These tests usually require more maintenance than unit and integration tests.

Fix issues as early as possible

The Test Pyramid supports the idea that a lot of thorough testing at the low levels and early phases of development helps to prevent bugs reaching the production environment.

Catching issues within your product during the later stages when the project is completed and ready for consumers is costly and can be more time consuming to fix. Attempting to apprehend issues, before they become defects i.e. production bugs, during the early phases of development is preferred because:

  • It involves less people to fix so can be presumed to be less time consuming
  • There is no impact to the customers so it’s less costly to the business

In your project, try and do the following when focusing on building your base level:

  • You should try to do the majority of your testing during the earlier phases of development with tests that are the cheapest to run.
  • It’s important to make sure that the amount of unit tests in your project is large enough to ensure that issues are caught and fixed before shipping.
  • Make sure you monitor the level of code coverage you have in your project to make sure you have the right amount to reach your target level of quality that will provide you with confidence in your product.

Follow the Test Pyramid and see what results you get.

Integrating Code Coverage

In order to follow the Test Pyramid concept, we first needed to build up the amount of tests in the base level.

Now that the tests were being added to, we needed them to be integrated within the CI tools we were using. So for one team, I set about getting these running within TeamCity. Using the Unity command-line test runner I managed to get this working when we were using Unity 5.4. Unfortunately, we found that this set up broke when we upgraded to Unity 5.6. Luckily, this has now be rectified in version of Unity 2017.2 and we have unit tests running in TeamCity on every build once again.

Unfortunately, there’s no way to measure code coverage on Unity projects so we’re just going have to be content with increasing the number of tests and making sure that these tests are meaningful. I’ll continue to monitor this though.

I next looked at getting a code coverage tool installed for the other team. Because of the tech stack here and how simple it was to setup on a simple project, I chose to use Istanbul. This has now been set up into a CI tool that runs on every push so that all code is checked and a code coverage report is produced. This gives us an idea of where we need to focus our efforts to increase the amount of unit tests.

Our Next Steps

As the production of unit tests are continuing and the developers are actively doing this themselves, I turn my attention to the next level of the Test Pyramid, the Integration tests.

This post was exported in one click from Google Docs into WordPress with Wordable.

Test Expo 2017

Last Tuesday I attended TestExpo 2017.

Since I switched back into testing, I have tried to read more about the changes in this field so that I’m more informed and in a better position to make decisions about the direction we take for QA within my day job and my personal projects.

Why is going to conferences and events good in general?

As I mentioned in my previous post, Ways to keep up your development skills as a Test Engineer, going to events like conferences or  meet ups are great for improving your technical knowledge. But, they are also great environments to flex your social muscles and converse with your peers.

Learning to approach and make conversation with complete strangers is a soft skill that everyone should look to be improving. But more than that, it gets you out of your comfort zone of interacting with the same circle of people that you deal with everyday.

Putting yourself in uncomfortable situations makes you grow by building skills you wouldn’t normally use.

Being in a place where the majority of people are dedicated to improving the quality processes and products was definitely an unusual but brilliant feeling. As I tester, particularly in agile teams, you’re outnumbered by “sole” developers (those whose main role is development). This may lead to situations where you have to champion your opinions in order to get things changed for the better. So being in an environment so different to my day to day life instantly lifted some of my nerves.

Event Vendors

At these types of events there are always opportunities for sponsored partners or vendors promoting their products.

My advice for dealing with vendors is to find out exactly what they do and only give your contact details to those who have products that you know may be useful in the future to your goals. Otherwise you could open yourself up to a lot of LinkedIn requests. By all means take as much information away, but be clear that you’ll be assessing what they have and whether it’s inline with what you want to achieve. You’ll contact them, not the other way around. It sounds a bit harsh, but in all fairness, you don’t want to waste anyone’s time, yours or theirs if their product isn’t what you’re looking for.

The vendors at this event were very honest and helpful and I got some great information in the form of the World Quality Report 2017  from Sogeti.

TestExpo talks

The day was arranged into series of talks and a round table (group discussion) session.

There were three specialty tracks being covered during the day at the event: Testing, Agile and DevOps. After the keynotes were finished, you could decide to pick and choose between talks from other tracks which I found was great as your interests may not solely be about testing.

The programme list for the day can be found on their website.

I chose to attend the talks within the Testing track. In total I attended all but one of the day’s talks (I did manage to get hold of slides from the talk that I missed so I wasn’t too disappointed).

Best talks

My favourite talks of the day were:

  • Keynote: Becoming a Unicorn
  • The core competencies of a good tester

Both of these talks made me think the most and gave me a list of actionable tasks that I believe will make me a better test engineer.

Top takeaways

  • Make UI automation tests cover key user journeys and smoke tests  and ensure a good ROI
  • The values and benefits gained from good QA should be:
    • Intelligable
    • Trackable
    • Testable
  • You should aim to deliver quantifiable results that support the business values and it’s qualities.
  • Build dashboards to share throughout the company’s teams to share quantifiable results
  • The soft skills of QA engineer are just as important as their technical and analytical skills.

Overall

There was lots of actionable information for me to think of ways to improve the testing practices that I’m currently implementing. The content was a nice mix of beginner/refresher and new/intriguing tech. The attendees were very approachable and friendly and the venue, although a bit far out (near Heathrow!), was a lovely building.

I’d definitely recommend anyone with an interest in testing going next year!

How Important Is CI and CD For An Appreneur?

Here’s my next post which appeared first on Simple Programmer..

During the day, I work as a test engineer. But I also consider myself a developer—more specifically, an appreneur.

What is an appreneur? Why, it’s any entrepreneur trying to build a business via mobile applications. You could be producing templates for others to use, creating apps to sell directly on the app stores, or providing a number of other services reliant on mobile applications.

I work by myself, so I’m running a business solo. Most appreneurs work in this way, though a few have a small team of trusted contractors handling specialized jobs.

It’s important to find tools and processes to help you continue to maintain the quality of your app once it is launched. Especially if it’s just you; you want to spend more of your time and resources creating and launching new products than maintaining older ones.

You want to reduce the time you spend on individual tasks, improve the quality of your end product, and reduce the cost of your overall efforts, and you need to carefully consider what might be the right tools to help you do all that.

Continuous integration and continuous development have been used by project development teams for many years to make the development process more efficient and get better end results. Can we use the same processes as appreneurs?

As an appreneur, I’m the one responsible for the design, development, testing, and business activities involved in creating and releasing a product to market. Make no mistake, completing one of these projects is no small feat.

If you are an appreneur, you are probably trying to increase or update your range of apps being sold in the App Store, or expand your catalog into a new App Store ecosystem, or create a new sustainable revenue stream related to app development. It all depends on where you are in your business.

Regardless of what specific paths you are pursuing, you are certainly hard at work, juggling multiple activities and milestones and products for your business. Even if you have a small team of contractors, you’re still overseeing the whole thing. Add to that all the pressures and distractions of life, and you’re managing a lot.

Continuous integration and continuous deployment were designed to let teams function better and create a smoother workflow. Generally, they’re used when you’ve got a lot of people working on a product or project to decrease confusion, potential bottlenecks, and the possibility of releasing products with issues. With appreneurs, it’s just you or a small team—will these processes actually help?

Let’s look at what exactly continuous integration and continuous development are, discuss some of the advantages and drawbacks, and then talk about if they are right for you as a solo or small team appreneur.

What is Continuous Integration?

Continuous integration (CI) is the practice of merging development branches that have been verified into a single, shared repository several times a day. CI is used by established and professional businesses across various sectors.

Developers use CI to avoid integration hell, the problem where conflict resolving battles are waged against merging multiple code branches into one shared location.

The concept of CI was adopted into the practice of extreme programming (XP) (which was designed to improve software quality and responsiveness) to be used in combination with automated unit tests that would automatically run in the developer’s local environment before commits to the mainline branch would occur.

A commit is the act of moving your development code from your local repository to the remote repository. Once the code has all passed these automated tests (which are run on build servers), commits to mainline (which is the shared repository) are accepted.

These automated tests are run on build servers (usually set up by people with experience with CI and CD tools like DevOps or test engineers) after a build is successful.

This process helps avoid one developer’s work-in-progress breaking another developer’s features. It also means that developers can be more confident that if the tests fail, then their work won’t make it into the production environment or, worse, into their customers’ hands with bugs.

XP is not usually adopted by solo developers or small teams; it is generally used by larger, professional, established product teams. CI isn’t a process specifically for appreneurs developing apps, but it is a general practice that can be adopted by anyone developing software.

Common Tools for CI

At the time of writing, the most common CI tools are:

  • Jenkins
  • TeamCity
  • Travis CI
  • Go CD
  • Bamboo
  • GitLab CI
  • CircleCI
  • Codeship

For a more detailed explanation about all of the above tools, please see this excellent article on Code Maze.

What Is Continuous Deployment?

Continuous deployment (CD) continually releases updates to the production environment after changes to the product are verified by automated tests at various stages through the development process. It is also used by established and professional businesses across various sectors.

CD tools are used in a chain of events, so when one step in the workflow is successful, the next is kicked off. When that is successful, the next begins. This practice means that no manual intervention is necessary to deploy to production.

The process works step by step until it gets to the production environment. If at any point a step fails, the appropriate people are alerted and fixes are made to ensure the process starts and runs again successfully.

Using CD in conjunction with CI means that development teams should have less chance of releasing products with issues to customers.

Continuous deployment should be the goal of most companies that are not constrained by regulatory or other requirements. Because it automates the process, it won’t be held up by errors or a forgotten step, and it also means that a failed step can’t be missed.

Again, CD isn’t specifically for appreneurs developing apps but can make you more self-sufficient in your deployment process to production.

Common Tools for CD

There are some useful tools that help with automated deployment. These tools are generally used by larger teams, but you can use them as a solo appreneur. Here are some of the most common CD tools:

  • Jenkins
  • ElectricFlow
  • Microsoft Visual Studio
  • Chef
  • Octopus Deploy
  • IBM UrbanCode
  • AWS CodeDeploy
  • DeployBot
  • Shippable
  • TeamCity
  • Codar
  • Distelli
  • XL Deploy
  • GoCD
  • Capistrano

For a more detailed explanation about these tools and others, please see this great list of CD and CI tools by DZone.

Fastlane

Automating other processes can be a big help, too. A tool that that I have found for mobile app developers which automates the app submission process is called Fastlane. Fastlane is open source, so it’s free to use. This tool can be used in conjunction with CI and CD to help push your apps through the submission process to production faster.

As the app submission process is so tiresome, even if you’re only submitting for one platform, it’s worth looking into Fastlane to save you time.

The Benefits and Drawbacks of CI and CD

Generally speaking, CI and CD are considered practices that most companies should use. Since that’s the case, should appreneurs try and implement these concepts, too?

Let’s look at some of the benefits and drawbacks of these practices and see whether it can work for us small timers.

The Benefits

Peace of mind
No need to worry about broken builds getting deployed because, providing you have the right tests in place, the build will fail and it won’t be released.

But if you have tests in place, how might the build still fail? Building software is always an iterative process. We are only human, so minor and even major bugs can slip through to the production environment if the majority of your code is not covered by tests.

The way to prevent the same bugs from reappearing is to write a new test to help identify when this situation will occur. So if the test fails, you will know that this bug is likely to occur again. You can then fix your code to make sure the test passes and the bug never again sees production.

New tests are automatically run
Any tests that you add during your development process are automatically picked up and run. You can also pass different parameters to run version tests in certain conditions.

Allows you to work with the minimum team size
You don’t need a full-time DevOps, release manager, and several testers to ensure your product’s quality level is maintained across environments. Once you put in place the relevant build chains, you can make sure your development and production releases are monitored by the minimum amount of people from different disciplines.

Flexible and trustworthy
If your team expands, you don’t need to worry that they won’t remember to run tests before they perform a commit to the master branch. If their commit fails, it doesn’t break everything for everyone else.

Easy to extend with templates
After you have come up with the perfect configuration for your project, you can create a template to reuse. As your project grows, you can use the template to create new build configurations.

The Drawbacks

Time to set CI/CD up
Once your CI and CD builds are done, you just have to maintain them. But until then, you’ll have to dedicate a bit of time to setting up the initial build configuration templates.

How much time? Well, that’ll all depend on how quickly you learn, how much experience you have with CI and CD tools, and what tools you select. So it’s pretty subjective, based on you.

There will be a learning curve
How steep your learning curve to overcome is will depend on how much experience you have with CI and CD tools.

You have to learn how to set up build configurations, link these to your code repository, write unit tests (if you haven’t been doing this already), integrate the tests into the builds, and possibly also add in a tool to output your level of code coverage after every build. Then once the build is created and passed, you have to learn to link the build’s output (probably the app executable) to the CD tool to push to your next step.

Maintenance
Those new shiny tools will get updated as well, so you’ll need to keep on top of the software updates and licenses. It may not be essential to upgrade to the latest version of everything, but ensure your tools are compatible with each other and that you keep an eye on the new features appearing in new versions.

Can be costly for the more popular tools
Let’s not forget that some of these tools aren’t free and some of them that do offer a free version are capped. So if you want the more costly option, make sure you have enough budgeted for these development costs.

Although the pricier options are sometimes preferred in larger companies because they come with a certain level of support that you don’t get when using free licenses, I find you can get quite far using a free license and having Google at your disposal. I would recommend that you try a free tool before spending money on a paid-for service. After all, using CI and CD tools may not be suitable for you or your project, so keep yourself and your budget in mind.

Builds can be fickle
During a previous role for my day job, when my team was new to TeamCity, we found that our builds were constantly failing. Again, this failure is part of the learning curve; it should also be considered in the time budget you need to put together a robust template. So if you find your builds are failing often, you’ll have to put more time aside to figure out why.

So: Should Appreneurs Use CI and CD?

There are some good points about using CI and CD—mainly that these processes enable an appreneur to remain self-sufficient for longer and makes you confident that your builds are stable and high quality all the way through to production.

But it does take a lot of work up front. And if you don’t have any previous experience with CI and CD tools, it will probably take you longer.

If you’re working with apps as a hobby and don’t have any previous experience with these tools, I wouldn’t recommend setting up a CI or CD process. The time you’ll take to learn the tools and maintain it all may not give you a great return.

But if this is the beginning of building a solid development base and practices for a long-term project, you really don’t have anything to lose.

Start with one process first, preferably CI. You can then build up your test suite to ensure a high level of code coverage (code covered by unit tests) and still control the release process to ensure nothing gets released accidentally while you’re still learning the tools. After you’re confident with your CI builds, pick a complimentary CD tool to help you build your solid foundation.

Anything you can do to make building your app business more efficient and your products more robust and higher quality can only be a good thing.

Setting up Istanbul on a Node project for Mac and Windows

When I took on this new role, one of my tasks was to ensure that we deliver high quality products.  In order to get an estimate of where we were currently at, I decided to look at implementing code coverage tools into the build processes so that we can gauge the level of quality we have on each product after each  successful build.

After watching the Testing JavaScript for Node.js with Mocha Pluralsight course (which I recommend for any .Net C# newbie to Mocha), I felt like I gained a good understanding of unit testing within node applications. And as an added bonus, the course even bolted on a short demo of a tool that I had been meaning to investigate. This was Istanbul.

Code Coverage with Istanbul

Istanbul is a Javascript code coverage tool that outputs the amount of code covered by unit tests in your project. It is an open source tool which is available on GitHub.

Setting Up

Whilst watching the video, I installed Istanbul into my small node project that I had been using throughout the course using npm install -g istanbul. This installed Istanbul globally to my Mac.

Next, I entered the command:

istanbul cover node_modules/.bin/_mocha -- tests/**/* (on Mac)

This is the command when trying to execute on a Windows machine:

istanbul cover node_modules/mocha/bin/_mocha -- tests/**/* (on Windows)

The Reports

After doing this, Mocha should run your tests. Appended onto the end, you’ll see a short output of the level of code coverage in the command line window that has been helpfully coloured in green or red (if other colours exist I haven’t seen them yet).

You will also now have a new directory within your project named “coverage”. In this directory, you’ll find a coverage report directory that contains a detailed HTML version of your application’s code coverage.

And that’s it!

I was very surprised how easy this was to set up and get running. The next step is to get this running on Team City! So that we can read the code coverage levels after every build and hopefully enforce a minimum level of code coverage so that whenever new features are written, if the amount of unit tests covering those new lines of code don’t increase, then we can fail the build to encourage developers to write more tests and increase the quality of their code.
<!–
Setting up for TC

  1. add to the artifacts path under general  settings the path where the html reports are output.

–>