Blog

Experiment 1: Asset Store Graphics Package

From now, I’m going to be undertaking conscious and a more structured approach to my appreneur journey. Instead of just building small apps that I’d like to see on the app stores, I have decided to also target the supporting connected services of app development to broaden my revenues streams and increase my skill set from just programming.

Experiment 1

I’ve found that the graphics package that I created is selling better than my project template on the Unity Asset store. Graphics packages are easier and faster to produce rather than full templates or code samples. And buyers can easily see what they will get for their money.

In this first experiment, I’m going to create a set of my own menu icons. Menu icons are something that I use across all my app projects, so by creating a set of them to reuse in every app, it will mean more consistency across my apps for users. It will also mean that I can tidy my Dropbox folder of icons so only use a few variations.

Another reason why I choose to create graphics is that the skills I gain from this experiment will be create a good base to build upon and will be useful for years to come.

Goal

My objectives with this experiment are to:

  • Improve my design skills from MVP to mid range

This will allow me to become more self sufficient at developing apps up to a higher graphical level.

  • Create at least one set of menu icons suitable for sale in the Unity Asset Store.

This will give me another product that I can begin to sell to hopefully increase my revenue from the Asset Store.

Contents of Icons Package

The menu icons I’ve identified that I consistently use are listed below. So this (at a minimum) will be my base set of menu icons.

  • Button background variations – slightly rounded corners, rounded corners, square corners
  • Menu
  • Pause
  • Play
  • Rewind
  • Fast Forward
  • Sound playing
  • Sound muted
  • No sound
  • Arrow left
  • Arrow right
  • Arrow up
  • Arrow down
  • Reset
  • Skip
  • No internet
  • Saving
  • Save
  • Load
  • Unable to save
  • Settings
  • Cancel
  • Close
  • Back

Project Length and Deadlines

I’ll be doing this experiment in between my contract work and my other project, but I’m hoping the progress will be good.

As it’s July, I thought that a suitable length of time for this experiment would be until the end of the October. I want to get these icons completed by the end of September and into the Asset Store by the end of October. It may sound like a long time, but I’m going on holiday in August and currently wrapping up a client project. This project could lead to a second stage which would be completed by December so I didn’t want this experiment and that project to overlap too much especially during the later stages of the project. Overall, I think this is realistic time frame.

Follow Me To Learn More

I’ll probably post about the results of this experiment on the blog at certain points and then do a post about my results of this experiment. So, if you’d like to hear more about my progress within this experiment and other projects I’m working on, I’ll be sharing skills learned and assets created to my mailing list every couple of weeks so please sign up to get updated.

Element is not clickable at point (x,y). Other element would receive the click…

I’ve been extending an automation suite that will be used to run UI tests on the platform in several environments to help us verify the build and highlight any breaking changes early.

Running these tests will also free us up to concentrate on tests that weren’t covered by automation and exploratory tests.

This particular suite is written in Ruby with Capybara and RSpec. This was a the first time that I had used Ruby to write an automation framework so it took a bit of time to get used to. Some of the issues that I was coming up against were annoying as I knew they were easily solvable in Selenium using C# but I persisted and eventually overcame them. 

I had most of the high priority tests passing apart from one. When Googling the answer, the only thing that I was coming up with was that the window size wasn’t big enough. This was annoying me because I had passed the Chrome Options parameter of “start-maximized” and, when not running in headless mode, the Chrome window was indeed maximised.

After a long day of searching and testing alternatives, I finally came across this post.  So in fact, the issue was only occurring in headless mode because when running tests in this mode, despite setting the browser to start maximized using the Chrome Options, it defaults to 800×600. So, I then set the Chrome Options window-size parameter to “1980,1080” to override this. And voila! The test passed in headless mode.

So, I would suggest if you want to run your tests in headless mode that you also pass in a specific size for the window so you don’t get caught by this too.

 

 

 

Career Mentors (Bye Jay)

Today my manager Jay is leaving. It was a massive shock when I found out. As he’s the CTO I had two thoughts:

  1. “Oh no, I’m sad now”
  2. “Is the company in trouble?”

So in fact, he’s making the decision to go purely because he wants a new challenge which is completely understandable and I wish him the best of luck.

He has been the best manager I have had so far because he’s been supportive, encouraging, and an ear when I need to moan (which can happen often as I am very picky particular dedicated to delivering a great product). He also doesn’t just tell me things I want to hear (which is pretty rare in people). I get honest answers and opinions.

Being a woman in tech, I didn’t have many people to look up to in order to steer me in the right direction, getting me to where I am now. I have done it all by myself (which makes me proud now that I think about it) but it’s been exhausting.

I haven’t had someone else to give me their wisdom which would’ve undoubtedly meant that I probably would’ve made less mistakes along the way. But from the beginning Jay has given advice, let me feel like I can research everything that will help improve the team and the area that I work in for the company, has given me autonomy, let me attend the events that I want (through which I have met some great people), sends me Twitter links to awesome women in tech (like @TheAmyCode) for further encouragement, and in general has made me feel more confident about being in this role and that I deserve this role because I am awesome.

Find A Career Mentor

It’s important to find people like this to guide you through your career because no matter whether you’re male or female, careers are hard and a little guidance goes a long long way.

If you do find these people, make sure that it’s not just a one-sided relationship. Make sure that there’s something that you can give them too. Whether it’s small things like being self-sufficient so that they don’t need to worry about you or giving back something tangible that could help make their lives easier or better.

So, bye Jay. See you soon, it’s been great working with you and remember BBQ!

 

How to Future-Proof Your Testing Career

Here’s my next post which appeared first on Simple Programmer..

Since I switched back into testing from a development role, I’ve been trying to build up skills that will future-proof myself. To do this, I’ve been going to more talks and conferences, and watching webinars on testing.

What do I mean by future-proofing? I want to ensure that I have the correct skills, either by focusing on one specific area or gathering a broad knowledge of a few keys areas, that may be popular or emerging in technology now or in a year or two. Technology has changed so much even in the last 10 years, so it’s one area that I think you should stay up-to-date on the trends.

Earlier this year, I watched a webinar called 2018 Test Automation Trends. I’m sure all testers have heard of or are using automation tools these days. They have probably heard the fears of some that manual testers are being made obsolete by automation (this is in no way true).

But automation isn’t everything. This isn’t the only area that could help you future-proof your testing career.

Automation is just another area of testing that’s getting more attention. It has become highly visible how important testing is, and how bad products affect end users as well as shareholders.

Although this webinar was about automation trends, at the end, the experts were asked what they thought would be things to watch out for in 2018. I believe that if you want to stay ahead in this field, you need to look ahead too.

So here are the topics that these experts recommend to focus on in 2018, and also some that I think are good areas to begin future-proofing yourself as a tester.

Blockchain

Blockchain is a data structure that exists in many locations at once. You can only add to the blockchain. No deletions or updates are permitted. The data held within a blockchain is decentralized, which means a copy of the existing blockchain is on every machine in the network.

Additions to the blockchain can be seen on every computer in that network and the transactions are cryptographically linked to the previous transaction. Therefore, carrying out fraudulent transactions is very difficult to perform. To do this, someone would have to rewrite their history to the beginning of time, which is extremely resource-heavy.

With Blockchain applications, once a contract is deployed to a blockchain, it can never be changed. So you have to be very confident the testing that’s performed is of a high level of quality and that everything that should be covered has been covered. There is no rolling back.

For more information, I wrote a blog post to introduce testers to blockchain applications. Have a read and see what you think about blockchain.

Understanding blockchain technology and how to test it will help future-proof your testing career. It will help enable you to move into different fields, like banking and the automotive industry, which are both currently using this technology. It will give you more options about where you will work.

Contracts and API Testing

Contracts and application programming interfaces (API) testing focus on testing API endpoints of services and microservices.

There are different types of services. One example is a web service, which provides a way for applications and systems like databases, mobile apps, and other web servers to communicate with the service over the internet using endpoints. Endpoints are URLs that your HTTP client interacts with in order to access the data from that URL.

When testing endpoints, you ensure that when requests are sent with the correct parameters or incorrect parameters, the system behaves in a way that you expect. For example, if you send three parameters to an endpoint and the endpoint requires four, then you would expect a response to be returned with a 400 status code.

The services architecture is widely adopted across many industries, because it allows code to be written in a more modular way. This allows the project to be easily extendible, and will also mean that systems are coupled less tightly together.

As a tester, you need to ensure that you can test endpoints and contracts thoroughly. Incorrect requests could mean that your application doesn’t respond with the correct status code or message for the user in the best case. But in the worst case, it could lead to holes in your application’s security.

As testing jobs require more knowledge about different ways to automate their testing code, learning how to implement API tests and perform contract testing will help future-proof your testing career. You’ll become an expert in a specific area of automation, and as more applications, particularly web-based ones, are built using the service architecture, your skill to competently test the integration layer will be very sought after.

Mobile Testing

Smartphones and tablet devices have become a staple product within homes all over the world, for those young and old. It’s just as important to have a well-tested app nowadays as a well-tested website.

In 2005, Google changed its search results ranking algorithm so that if a website wasn’t responsive, that site would be ranked lower in its results list. The dedication to and sophistication of mobile testing has massively increased since then.

We first began testing mobile websites and apps manually, but because of the fragmentation of Android and the yearly releases of all mobile OSs, the need to automate mobile testing has increased. Because of this need, there has been a number of mobile testing frameworks released to aid testers.

Appium is a mobile testing framework built on Webdriver technology. It was created in 2013 by Dan Cuellar, a test manager at Zoosk who found that the length of the test passes on the iOS product was getting out of hand. Appium allowed him to write automated tests for iOS as easily as Webdriver was used for automating websites.

I recently went to the first Appium Conference held in London to learn about how I could utilize Appium.

Knowing how to effectively test mobile apps and websites is now a full-time job, so tools like Appium are used to help mobile testers test thoroughly. Mobile devices are everywhere now and the development of applications will only grow, so I believe it would be a great skill to add to bolster your testing skill set.

Artificial Intelligence (AI)

AI is where scripts, machines, or anything that’s not human perform tasks that may or may not be complex.

Within AI there are a number of different areas. One that’s becoming more common now is machine learning (ML). Machine learning is when AI provides you with answers or data based on previously generated data. It takes all of the information gathered and makes predictions on future outcomes or decisions based on this historical data.

One well-known example of ML and AI is the AlphaGo Zero software from Google’s DeepMind company that taught itself to play Go. Within three days, it was able to beat the previous version of itself (and that version bested 18-time world champion Lee Se-dol).

In testing, we can use ML to predict outcomes of test cases by looking at historical test result data. It can be used to predict which test cases should be prioritized over others when certain parts of the system have been updated. It can tell you when a test case was last run so you can determine what test cases are the most important to be run.

We’re constantly investigating how to become more efficient, save time, and complete jobs faster. AI can aid in this, so it’s a good idea to know its strengths and weaknesses, and how it can be used to help you with your testing career.

We’re at the very early stages of AI and how it can be utilized within daily testing activities,helping us achieve a high level of quality of our products. Ensuring that you keep up with or are ahead of the developments and applications of AI within software testing will help solidify your expertise in the field and help future-proof your career.

Docker and Containers

Containers give us the ability to store a number of different systems virtually within the same location. This means that if one environment needs a website front end, server, and database in order to function, all of these can be stored within the same place.

For testers, containers provide a quick solution to deploy test environments that are stable, fast to set up, and a mirror-copy of staging or production environments.

Learning the strengths and weakness of the architecture that your environments are built on will mean that you know how your application can be affected and whether certain types of tests will be more suitable to ensure a higher level of quality.

For more information, I’ve written a post about containers and how we use them at my current company.

Learning about Docker and the containers technology, and how to test it, will help future-proof your testing career. Since it allows quick deployment and scaling of infrastructure, it’s being widely adopted by a number of DevOps teams in different industries. This means you, again, will give yourself more options when looking for your next job opportunity.

Some Suggestions From Me

After listening to the expert’s suggestions, I’ve come up with a short list of additional areas that I think would definitely help bolster your testing skills arsenal.

Debugging

Debugging is where you step through code in order to locate defects. This isn’t something that every developer or engineer likes to do. Having the patience to walk through code can be an overlooked skill.

Debugging can take a few minutes if you know what’s wrong and where it is, but it could sometimes take days to figure it out—particularly if the code base is made up of hundreds of files.

Debugging also makes you practice reading the code, and it will help you develop a further and deeper understanding of the code base you’re stepping through.

This skill is useful for developers and is great for testers to future-proof themselves, especially if you are developing test code in any capacity. But since it’s not one that’s actively trained, you should invest your own time into debugging code to further narrow down where in the codebase the defects can occur.

You can do this by reading log files of applications when errors occur, and tracing them through the code base to see if you can track it down.

For more information on how to improve your debugging skills, read 5 Steps to a Bullet-Proof Debugging Strategy.

Augmented Reality, Mixed Reality, and Virtual Reality

Devices that provided users with each of these experiences have been around for years. However in 2017, I believe they were accepted and adopted by a larger consumer audience.

As this new tech is adopted more and more by the public, the demand for experienced testers in these areas will increase.

Since it’s still very new, there’s not much in terms of frameworks or testing standards that can be applied yet.Still, this is the time for someone to take that first step and create something that everyone can use going forward.

Cloud Computing

Cloud computing is the on-demand delivery of IT resources like compute, database storage, and applications via cloud service platform over the internet using a pay-as-you-go format.

Platforms like Amazon Web Services (AWS), Google Cloud Platform, and Microsoft Azure are some of the options that are available presently, with AWS storming ahead of the competition.

So why should you as a tester be concerned with what tech your applications are built upon?

Well, it’s the same reason as I mentioned earlier about containers. If you know what tech your application is built upon, you should be able to understand it’s strengths and weaknesses and how that will affect your application.

Therefore, you will be able to test your application better and help close up any holes in security that may be exploited, ensure it’s performant, and that the end user has a great experience when using your product.

How Future-Proof Are You?

When you’re working within a technical sector, the ability to step back and assess the bigger picture of what’s going on in your industry, what things are emerging, what things may be going away, and what’s here to stay will tell you a lot.

It will clearly show you where you should invest your time on new skills, and which ones you may need to strengthen.

If you want to do more in order to future-proof yourself, why not try and go to a talk in one of these areas yourself, read blogs, listen to podcasts, or watch YouTube videos of experts in these fields. You can also get a free subscription to AWS and experiment with the Elastic Container Service to get a basic knowledge of containers.

I hope this post has given you a foundation of what’s to watch out for now in 2018 within the field of testing. There’s a lot of options. So don’t wait—get started.

The Challenges of Leading a Distributed Test Team

Here’s my next post which appeared first on Simple Programmer..

Being a manager of a distributed team of people can be a challenging role. When you have to communicate on a daily basis and those people are in different time zones, or if you’re at the end of the project delivery date, you may find your stress levels slowly creep up higher and higher.

I’ve not only been trying to lead a distributed (albeit small) team for the last year, but I’m also trying to balance my daily testing tasks, researching new tools that could help my team, and maintaining or improving the overall quality of projects across the company.

Let me share with you a few challenges that I’ve faced and how I have or am planning to overcome them in future.

Different Time Zones Between Team Members

It’s important in every project for team members to regularly communicate. It’s easy for details to be missed, especially during busy times in the development process.

When your team are in different time zones, it’s even more important to make sure that they communicate daily. You’ve only got a specific amount of time overlap to talk directly with your team all at once. This time must be utilized and never skipped by anyone.

Ways that your team could communicate could be via the daily stand-up meeting (if you use Agile methodologies), or making sure you are giving daily reports via other channels, like internal communication systems such as Slack or Skype. Whatever medium you use, just ensure that the communication is regular and everyone gets involved.

Lesson: Plan your meetings at times where everyone is available and projects are not busy.

Balancing the Workloads of All Team Members

Even in teams as big as 100 people, it’s important that everyone feels like they are not overwhelmed by the work we have and can also perform their job to the best of their ability.

Having a distributed team also requires you to take into account what time other team members may also be working so you can assign the correct project to the correct people. You can use tools like Trello, Asana, or even a spreadsheet to manage project assignment. Doing this will help those team members more easily manage their workload so the project is more closely aligned with their schedule.

You’ll also need to consider the skills that are needed for a project and how long the project will last. This will help you assign the right team member to the right project. For example, say you had an upcoming project where the test framework needed to be written in C# and some of your team were proficient in Java and some in Ruby. You may find that this type of project will be easier to and faster to implement by members of team know Java as they are quite similar languages.

Lesson: Try and match the right projects to team members with similar skills, interests, and time to carry out the project.

Trying To Manage Everyone and Everything

Despite being a small testing team where I work, we have a continually big workload. We are managing the quality and daily testing for two-to-four projects at once. With my job, carrying out daily testing tasks, building the automation suites, and trying to monitor the overall quality strategy in the company can be a lot to do for one person. But not for me, I’m Superwoman!

Only (and obviously) joking. Seriously, I realized very early on that there isn’t time to do everything, and that I can’t manage everyone and everything by myself. So, to make sure I can carry out all of my tasks, I prioritize my work and delegate lower-priority tasks to other team members.

I split responsibilities between team members according to project deadlines, project requirements, and team member availability. I encourage my team to do the same thing to make sure they also don’t feel overwhelmed and can get their jobs done well.

In a distributed team, this can work to your advantage. For example, if you have team members who are in a time zones hours ahead or behind you, you can leave them work when your overlap time is over. Then when you return, the task may be fully or almost complete, ready for you to focus on something else.

In order to do this, you need to know your team and trust that they will carry out their jobs so that you can focus on the big picture.

Lesson: Get good people to help you manage things.

Trying to Monitor the Progress of Your Strategy

When I first arrived at my current company, one of the first things I was asked to do was outline a Test Strategy for the company.

Every month I try to track if we’re still on our way to achieving those goals I set a year ago, if they need adjusting, or if they’re no longer relevant and some of the strategy needs revisiting.

If all of my team were in one location, it could be as simple as having a quick meeting once a month to find out where we are. But having a distributed team means that this needs to be planned in advance and at times where everyone is available. With some members of my team in time zones as far away as four hours at times, meetings need to be planned at times during the project where each member won’t be busier than usual, like the week before a product delivery.

Utilize free calendar resources, like Google Calendar, to share a team calendar booking in important dates where the team will be busy or unavailable. Use this to regularly track your progress with team meetings.

Lesson: Have a clear idea of your vision and keep track of it being executed.

It’s Easier for Quieter Members to Hide Away

Make sure the quieter members of your team are also vocal at team meetings. Find ways to get everyone engaged with their team. It’s easy for those that are quiet to sit back and let others that are more confident speak up, making themselves be heard, especially if these meetings are over Slack or Skype.

You can do this by directing specific questions to these people or asking them their opinions. This will prompt them to speak up a bit more. If English is not their first language, they may be more inclined not to speak up, but giving them a little nudge by asking a question to them here or there may be all they need to build up their confidence.

Keep in mind that there may be a reason why these people are quiet, so before you ask them questions in meetings with everyone ready and waiting, maybe talk to them privately. Learn who they are and what motivates them, then use that to regularly engage with them.

Lesson: Learn who your team are and how they like to communicate, and encourage them to participate.

It’s Hard to Keep Everyone’s Skills Up

In any technological field, like software development, virtual reality, big data, or artificial intelligence, one of the challenges is to make sure you are constantly learning. Things are always changing and it’s important to keep your skills up. It’s hard to find time to build your skills (especially within testing and QA roles) because at the end of the day, you’re employed to develop and ship a product. Improving your skill set in an area is usually a byproduct of working on something day in day out. But, I would suggest you encourage your team to carve out some dedicated time in their week to learn new skills or improve on existing ones. This not only will keep people in your team, show you care and want to invest in their personal development, but will also make them enjoy coming to work.

Utilizing resources like Pluralsight courses (aimed at those in technical professions), YouTube videos (for everyone and about everything), knowledge sharing sessions, and pair programming are all easy to access. All of these are free to low-cost ways to improve your knowledge and strengthen the skills of your team.

It’s easy to let deadlines let us work through these times, so make sure your team schedule them at times where they have little to no distractions (if at all possible).

Lesson: Tell each team member to calendarize a time slot for themselves to work on skills weekly to improve their knowledge.

The Bottom Line: Have a Great Team

Leading a distributed team as well as all of your other responsibilities as a manager is a challenging task. You shouldn’t underestimate that the amount of work done day-to-day will be the same as when you lead a team all in the same location.

Sharing tasks among your distributed team members, like responsibilities, projects, and even meetings, will help you balance the efforts of your daily and management tasks. But you need a good team to help you with this.

So, make sure you find the right people to help you. In return, invest some time into them by making sure they are not overloaded with work, get to know them by finding out what motivates them, and encourage them to build their own skills by training themselves.

My First (and the) First Appium Conference

Last year, I was selected for a scholarship to Appium Conference 2018 by White October Events. I’ve been aware of Appium for years now as I’m an app developer but I’ve not (yet) used this tool on any of my projects.

For those that don’t know, Appium is a mobile testing framework built upon the Webdriver technology. It was created in 2013 by Dan Cuellar. He was an Test Manager at Zoosk who  was finding that the length of the test passes on the iOS product was getting out of hand. Appium allowed him to write automated tests for iOS as easily as Webdriver was used for automating websites.

Five years later, Appium has a massive community building up a successful open source project that can be used on Android, iOS, Windows in a variety of coding languages.

This year was the first Appium Conference.

There was only one track which was great because I got to see everything and didn’t have to pick between two talks that were probably going to be beneficial to me.

Interesting Things to Note

  • There was a lot of variety of languages being used with Appium (part of the appeal of using the product I suspect).
  • Lots of people were using Jenkins with it as their continuous integration/continuous delivery tool. There were a couple mentions of Circle CI but none about TeamCity. Because of this, I think I’ll be looking into Jenkins more than TeamCity for my app projects.

All the talks were highly interesting and not too difficult to follow for a beginner like me. So I want to share my biggest takeaways from each of the talks.

Keynote – Appium: The Untold Story

The day began with a keynote from Dan Cuellar and Jason Huggins. They spoke about the history of Appium, where it is now and briefly touched upon where he wants it to be in the future and mentioned their vision of StarDriver.

They want to see Appium grow it’s users and the platforms it can test particularly to the internet-of-things and various hardware.

The best takeaway for me from this keynote was the phrase “Challenge everything you see”.

Appium: A Better way to Play the Game

This first talk was given by Charlene Granadosin & Charlotte Bersamin. What I found interesting within their talk was how they were integrating their release and exit reports within Jira using Xray. They used a curl command to upload their latest test results to Jira so these results are clearly visible to the Product Owners or Managers of the team.

My biggest takeaway from this talk was to investigate whether the tools we were currently using for test case management was able to integrate with Jira to give such detailed reports and to try and get the automation up and running.

Deep Hacking Appium for Fun and Profit

Daniel Puterman‘s talk explained how he had contributed to the Appium project by creating a new endpoint to gather native application screenshots.

Because the company Daniel worked with was Applitools, my biggest takeway from this was to figure out whether visual testing tools would be useful for testing virtual reality (VR) applications as much as they would be for websites of mobile applications.

Why the h# should I use Appium with ReactNative?

Wim Selles delved into a comparison talk about why they chose Appium (which was extremely useful as I’ve also been debating what automation tool to use for mobile apps).

Out of all the frameworks he mentioned, they went with Appium because it fit with a lot of his requirements for testing ReactNative apps.

There were a lot of takeways for me from this talk.

  • Consider your own project requirements when you pick your automation tools
  • What are your requirements?
  • What should your app do (now/future)?
  • Which tool supports your needs/expectations?
  • Do a proof of concept test
  • Do research into competitive tools

He also gave a couple of good ways that you can speed up app testing:

  • Remove animations on screens
  • Utilise deep-linking to get directly to screens

Layout Automation Testing (GUI)

Prachi Nagpal explained how she was using Galen to perform her UI browser based testing for mobile and desktop devices. Galen does this by measuring the distance between elements that are being tested. You can also produce heatmap results from this tool.

It was a good talk and interesting to see a tool that I had never heard about.

Can you please provide the full Appium Server logs? A Brief Tour of the Logs

Isaac Murchie next walked us through the Appium logs and the takeaway here was that he pointed out that some lines are bolded to highlight their importance.

He also noted that the following are ways to know which are requests and responses in the logs.

This is a request:

[HTTP] -> 

This is a response:

[HTTP] <-

Interaction with Native Components in Real Devices

In his talk, Telmo Cardoso told us how he tested native components of mobile operating systems. He explained the challenges that he faced (some tasks were difficult on one platform but easier on the other) and ways he and his team had got around them.

The areas he found challenging were:

  • Adding contacts
  • Pushing files to a device
  • Push notifications
  • Managing calls
  • Simulating low battery
  • Extracting logs

The biggest takeaway from this talk was that he used Cucumber for his automation framework along with with Appium successfully to test the native applications and features of smartphones and not just the applications running on them.

Using Appium for Unity Games and Apps

Because of my daily work with Unity projects, I was particularly looking forward to Ru Cindrea‘s talk on how she used Appium for her Unity games and apps.

She first explained how she used OpenCV an image recognition tool with Appium to try and test her Unity games.

The positives were:

  • Works for simple scenarios
  • No changes to game required
  • Found issues like performance issues or out of memory crashes

The negatives were:

  • Wasn’t fast enough
  • Not for games with lots of text

So she decided to create a component called AltUnityTester to help her with her issues.

AltUnityTester is created with Python bindings. It opens a socket connection and waits for a response on a specific port.

When the AltDriver is added into the Appium project it gets a list and knows everything about that Unity scene’s objects. It can then send a command to that port to get information back from the scene to perform tests e.g. checking the end position of elements or text output.

This solution is useful because it’s real-time but it does require changes to the project and it only works with Unity.

So my biggest takeaway was to investigate whether this AltUnityTester could be extended or something similar made in order to test VR applications using Unity.

It’s available as a Unity package or on Gitlab.

Docker-Android: Open-source UI Test Infrastructure for Mobile Website and Android

Application

Budi Utomo next talked about his Docker-Android image to test Android projects and websites on Android devices.

His plan for project development was to:

  1. Create UI tests for Android devices
  2. Write unit tests on Android
  3. Create UI tests on Android apps
  4. Implement Monkey/Stress tests

The biggest takeaway was the demo that showed how Appium can be used easily within Docker containers.

Application Backdoor via Appium

Rajdeep Varma explained how you can use Appium scripts to call development code methods from test code.

He used Appium in this way because he was having a number of problems when trying to write tests:

  • System pop-ups were called and were not needed when running tests
  • Driver limits e.g. mocking the device has been shaken or changing time limits
  • Tests were slow to run

This is where he is using backdoors and where he thinks they could also be used:

  • Changing backend URLs
  • Changing app locale
  • Getting session ids from the app
  • Disabling “What’s new” pop-ups
  • Disabling client side A/B tests
  • Faking SIM card for payments
  • Get analytics data from the app to validate it

The biggest takeaway from this talk was to be careful not to use backdoor for every test case and call incorrect methods in order to make tests pass.

Mobile Peer 2 Peer Communication Testing

Canberk Akduygu gave us a talk about his challenges when automating the BIP app (it’s like the Turkish Whatsapp).

He was building an extended grid solution to change to the right version of Appium and set the desired capabilities within their testing framework according to properties set in a JSON config file.

His demo was the biggest takeaway which showed two phones messaging and even calling the other. It was one scenario that was running two different steps on each device. It showed that the test steps needed to be synchronised.

From a Software Tester to an Entrepreneur: What I’ve Learned

Kristel Kruustuk next came on stage and walked us through why she founded Testilio and her struggles with the company despite being so successful and growing at a fast pace since she began.

My takeaway from this talk to was investigate and get in touch with the Testilio team and see if they had any future plans for expanding from manual and automation testing to VR testing.

Appium: The Next Five Years

Jonathan Lipps gave the final talk and began again with the history of Appium but then spoke more in depth about the vision of the product over the next five years and hopeful milestones. Some of these things were:

  • StarDriver
  • InfinityDriver
  • Extension to the W3C WebDriver protocol
  • Node js base classes and libraries for easily writing new drivers

Closing Remarks

Lastly, we were treated to a performance featuring Jonathan Lipps, Appium and Selenium. I believe four or five instruments were being played by Appium, Selenium was outputting the lyrics and Jonathan was singing and playing the ukulele.

My biggest takeway from this is that Appium can be used in a variety of ways to perform a number of impressive tasks.

After Party

The after party was held at a bar a short walk from St Paul’s Cathedral. There I managed to talk to Ru Cindrea in more detail about the project I wanted to use the AltUnityTester for and whether she thought it would work. I also, managed to talk the ears off of both Charlene and Charlotte who were the speakers from the very first talk of the day.

Overall

I had a great day, met loads of wonderful people (including attendees and speakers) and I hope that next year, I’ll have begun using Appium for something that I do so I can share my own experiences with the community.

Moving to Containers

It seems that containers are the new technology within IT that everyone is trying to incorporate into their infrastructure. And why? Containers not only benefit those in DevOps, but have positive implications for all teams involved in product delivery.

At Immerse, we’ve recently moved our infrastructure to a containerised solution. It’s early days to analyse the impact that this has had on our development teams, but I thought it would be a good opportunity to deepen my understanding of the this technology.

What Are Containers?

Containers give us the ability to store a number of different systems virtually within the same location. This means that if one environment needs a website front-end, server and database in order to function, all of these can be stored within the same place.

What sort of systems can be run within containers? Well, that’s where container images come in.

What Are Images?

A container image is a stand-alone, executable package of a software that has everything needed to run it including code, runtime, system tools, system libraries, and settings.

An image is required in order to build a container, otherwise it will be empty when created.

So where did all this new tech come from then? It seems like it’s come out of nowhere but spread fast (kinda like Bitcoin right?). Well, it all started by a small company now called Docker Inc. They created the system that containers run on, Docker.

What is Docker?

Docker is a computer program that allows you to perform operating-system-level virtualization known as containerization. This is the creation of containers.

Docker allows independent containers to run within a single Linux instance. This reduces the overhead of starting and maintaining virtual machines (VMs).

Since Docker began, there are other tools than have been developed that can perform containerization.

The World Before Containers

Before we used containers, there were virtual machines (VMs). VMs remove the need for physical hardware and allows one server to be turned into multiple servers. App of this is possible because of a hypervisor.

A hypervisor (also considered a VM monitor), is software that creates and runs VMs. It is the reason why you can run many VMs on a single machine. Each VM will have a full copy of the required operating system, one or more applications and the needed binaries and libraries. All of this can take up tens of gigabytes of space!

Some companies have made the switch to containers from VMs because:

  1. VMs can also be slow to boot, while you can spin up a container within a few minutes providing you have the right image. And if the don’t, that usually only takes a few minutes to obtain.
  2. You can pack a lot more of your companies applications into a single physical server using containers than a what you can fit in a VM.
  3. VMs take up a lot of system resources as they not only just run a full copy of an operating system, but a virtual copy of all the hardware that the operating system needs to run. All that a container needs is enough of an operating system, supporting programs and libraries, and system resources to run a specific program.
  4. With containers you can create a portable, consistent operating environment for development, testing, and deployment.

So How Do You Manage Containers?

Container Orchestration are frameworks that are used to integrate and manage containers. These are not necessary for everyone using containers. Usually enterprise level organisations are more likely to use orchestration tools as they manage a large range of containers and their images. Examples of these tools are Kubernetes, ECS and Ansible.

These tools help to simplify container management from the initial deployment to managing multiple containers, scaling for load, availability, or networking.

The Benefits and Drawbacks

Like any new piece of technology or tool, containers too have their own list of benefits and drawbacks that mean you choose to either integrate them into your development pipeline or you don’t. So, what do containers offer?

Benefits

  • The ability to spin up whole environments consisting of all the systems you need within minutes.
  • The ability to change the configuration and deploy those changes quickly.
  • Containers allow all users of the system to be self contained. This mean that rogue developers who develop new features, don’t run tests locally, push to the test environment, then leave to go home only for QA to find that test is broken is no longer a daily issue (wow, that sounded like a rant!).
  • Features can be tested in isolation easily.
  • Testers can be in control of checking out and pushing features to the test environment.
  • Differences between environments is no longer an issue because they’re all spun up from the same set of stable images

Drawbacks

  • Initially, there maybe some complexity to setting up containers.
  • There are some security issues that you med to be aware of when using containers. For example, if a user or application has superuser privileges within the container, the underlying operating system could, in theory, be cracked.
  • It’s time intensive to set up decent security measures for containers. There’s no default, out if the box solution yet
  • Everyone is making container images and it could be easy to download something malicious into your system.
  • Breaking deployments into more functional discrete parts is smart, but that means we have more parts to manage. There’s an inflection point between separation of concerns and container sprawl!
  • Containers tend to lock you into a particular operating system version

Are containers the future of development?

It seems because of the fast adoption of containers that they may eventually replace VMs once their issues have been overcome. And because it is a new technology, there will be drawbacks at this early stage so don’t let these deter you from experimenting with the tech yourself on your own projects.

However, technology has changed extremely fast over the last 30 years, so it may be that containers are superseded by a new emerging technology that solves the drawbacks of containers and gives us a while load of other benefits too.

For more information of containers especially if you’re learning the basics, please check out the Docker videos by Nigel Poulton on Pluralsight. I found these videos extremely helpful in delivering information and background about a brand new technology. The concepts were also broken down into easy to understand topics which is perfect for beginners. After watching them, I was able to understand a lot more and felt more confident when speaking to the DevOps at Immerse about how they had implemented container and why they made the decisions they did.

What’s your experience with containers? Do you love them? Are they growing on you? Or, have you not yet made the leap into using them? Whatever your experience, I hope this article has given you a better insight into the background of containers.

For more information about containers, please feel free to view the references I used:

This post was exported in one click from Google Docs into WordPress with Wordable.

A Beginner’s Guide to Testing Blockchain Applications

Here’s my next post which appeared first on Simple Programmer..

During the last few months of 2017, bitcoin and other cryptocurrencies were being talked about by some media sources on a daily basis. These currencies, which had been around for years, were suddenly experiencing major growth. For example, the price of bitcoin has grown from just under $750 in January 2017 to $5,856.10 in mid-October 2017.

This massive growth made it a major talking point. So, as these newer currencies came more into the public domain, the technology behind cryptocurrencies was also beginning to gain attention. This technology is the blockchain.

Blockchain applications are being adopted by some of the biggest industries around the world. Because of the nature of blockchain applications, it further supports how important testing and testers are, and that the field will be more highly thought of and sought after in the future. So as testers, it’s only right to wonder how this new technology will affect your day-to-day work, new opportunities, and current skill set.

What new tools will you have to learn to test blockchain applications, and what skills are needed to test them?

Here, I will outline what the blockchain technology is and how we, as testers, can prepare ourselves for testing blockchain applications.

What Is Blockchain?

Blockchain is a data structure that exists in many locations at once. You can only add to the blockchain. No deletions or updates are permitted. The data held within a blockchain is decentralized, which means a copy of the existing blockchain is on every machine in the network.

Additions to the blockchain can be seen on every computer in that network and the transactions are cryptographically linked to the previous transaction. Therefore, carrying out fraudulent transactions is very difficult to perform. To do this, someone would have to rewrite their history to the beginning of time, which is extremely resource-heavy.

In order to update every machine in the blockchain, the machines have to sync to have a common history. Although all machines will eventually have the same data because of this syncing action, only the more recent transactions are synced more often.

For a new transaction to be added, the decision is reliant on the majority of participants in the blockchain. Once the transaction’s authentication has been validated, the new block is added to the blockchain.

What Is It Used For?

Currently, blockchain is used mainly by the financial and automotive industries because of its highly secure structure. As I mentioned earlier, it’s also the technology that underpins cryptocurrencies like bitcoin and Ethereum.

Not all blockchain technology is open-source. There can be private blockchains like the ones used in banking systems.

What Types of Tests and Techniques Can You Perform on Applications Built on Blockchain?

There are many different types of tests that can be performed at the various stages of developing software projects. Below are a few types of tests that can be utilized to ensure a high level of test coverage and quality for blockchain applications.

Unit Tests
Unit tests help developers ensure their code is performing correctly at the lowest levels and smallest parts of functionality. This should always be the first line of defense to ensure an application catches the majority of bugs early during development.

Integration Tests
Integration tests help developers and test engineers ensure the communication of their code between different components, and possibly between internal and external systems like databases.

User Interface
User interface (UI) testing uncovers how the application works from the end user’s perspective. It’s important to ensure that you carry out UI testing to make sure their experience is positive or that they at least get the correct feedback from the application when it doesn’t perform well.

Application Programming Interface
Application programming interface (API) testing gives you confidence that you’ve validated the responses that your application receives from external APIs and makes sure the formats of your API requests are correct and being handled correctly.

With blockchain applications, there is also a similar type of technology to APIs that allows you to adopt the same testing practices for APIs. These are called smart contracts.

What Are Smart Contracts?

Smart contracts are a big part of the validation technology within a blockchain. A smart contract is a “set of rules in the form of programmable constructs that are capable of automatically enforcing themselves when predefined conditions are met.” For example, a precondition could be transactions trying to append to a specific ledger will undergo additional validations or go through a different set of validations that are more robust.

Although a smart contract is very similar to an API, whereby it has public functions that can be called by anyone registered on that blockchain network, it cannot call external web APIs.

So why do I think that testers are extremely important when testing blockchain applications over other types of systems? Simply because once a contract is deployed to a blockchain, it can never be changed. So you have to be very confident the testing that’s performed is of a high level of quality and that everything that should be covered has been covered.

If a defect is found in production, then a new version of the contract has to be created and deployed. New versions of existing contacts can’t simply get the existing data transferred in; you have to manually initialize the previous data with the new contract.

Updating a contract and rolling back an update is also not a viable option; this increases the complexity of development and means that the importance of implementing and running unit and integration tests on your application before it reaches production could save you major time and money in rectifying defects.

What Skills Do Testers Need for Blockchain Applications?

Although blockchain applications are relatively new in software development, I don’t think testers need to adopt new skills in order to test this type of technology.

Some of the skills I’ve highlighted below are natural skills of good testers or simply skills that you learn early on in your testing career, which grows with your experience in the field.

Critical Thinking
The ability to critically analyze and think about and around a problem is a timeless skill for testers and will be even more sought after for testing blockchain applications.

Testers think about problems such as: Will transactions still execute if x, y, and z are not done? What happens if the network has lots of transactions waiting to confirm? What feedback is given to the user in these cases? Should this be the feedback given to the user? Or is this feedback exposing any security risks?

Another thing to consider if embarking on a new project is to question whether blockchain is the best technology for your use. It’s the new shiny toy, so everyone will want to adopt it, but it may not be suitable for what you want to achieve.

Things to keep in mind are compliance issues; for example, you shouldn’t store health or criminal records, as there are no deletions allowed. So, when criminal records for minors can be wiped, you won’t be able to do so with this technology.

Test Design Techniques
During the ISTQB-BCS Foundation Software Testing syllabus, you are introduced to test design techniques. Knowing even the basics of test design techniques, like boundary value analysis and equivalence partitioning, will make sure you are constantly thinking about and checking the inputs and outputs of the application.

Things to consider could be: How will the application act when you input values that are within, on the edge, and outside of the boundaries of acceptable values? Will the transaction complete? If not, what type of error will be returned? Is this error code correct for the type returned? Should it return anything at all?

Automation
Strong automation skills in all languages, either for lower-level unit, mid-level integration, or API or high-level UI tests, are good skills that can be transferred to testing blockchain applications. Having a solid foundation of automated tests will be needed to ensure that the majority of issues are found early in development.

Manual UI Testing
If a solid foundation of automated tests is put in place, testers can focus on the outlier issues that can more easily be found via exploratory testing performed manually.

Being able to work independently, investigating different areas of the application, trying to find weak areas, and being able to successfully reproduce these are always skills that great testers will need. Despite the world looking to automation to perform a lot of the repetitive and arduous tasks, manual testing skills are still something to hone and improve.

Learn New Tools Quickly
As new technology comes to light, the list of tools to test this technology will also grow. You’ll need to be able to learn how to use these new tools quickly and judge which is the best for what you’re trying to accomplish.

Future-Proof Yourself and Prepare for Blockchain

Hopefully by now, you have a better understanding of the blockchain technology and no longer think it’s as scary as your first thought.

I’m sure if you’re a tester, you already have the foundations of the skills I’ve outlined above. My advice is to push further into an area that you have an interest in and possibly try to improve in the areas where you are weakest to give you the best chance to grow your skills when testing blockchain applications.

Rebasing from Master onto your local branch

 

Now that my QA team are adding to the test coverage by writing integration tests directly into the project code base, it’s finally time that I start to embrace rebasing (I sort of feel that lightning should crack when you read rebasing)!

As I survive with the basics of Git and because Irina and I only usually submit to our own repos, apart from branching and merging back to master, there’s not much activity going on. But once you work on a repo where at least two people are active on it, merging their code into master and creating branches weekly, your local changes can fall behind quite quickly. This is where rebasing comes in.

Rebasing essentially rewinds back your branch to when you created it from master (if that’s where you branched from) and applies all the commits that have happened from master branch to your own. If there are conflicts or differences between the commits to the master branch and yours, you can either:

fix them at each commit, then you

[sourcecode language=”css”]git rebase –continue[/sourcecode]

, or you can accept the changes on the other branch and

[sourcecode language=”css”]git rebase –skip[/sourcecode]

The concept is quite clear to me (now), but what I struggled with was doing this all on the commandline. I know I could use a tool like Source Tree to do all the heavy lifting for me, but in fact, despite me being quite a visual person, I like using the commandline for Git. So I vowed that this will be the time to learn and to do it more often so that it’s solidified in my brain.

So after being walked through the process, I learned that these are the steps that you need in order to rebase from master on to your own branch.

So these are the steps, pretty simple and after a couple of times, I reckon I’ll be able to do this from memory…providing the conflicts are few and far between!

[sourcecode language=”css”]

git checkout master
git pull
git checkout MyBranchName
git rebase master
git rebase –continue (until you fix the conflicts) or git rebase –skip (to skip these changes)
git pull
git push
git status

[/sourcecode]

The steps here are what I found works, but if you spot anything wrong, don’t be afraid to let me know.

Building a Test Pyramid from the Bottom Up

I was brought into my current role at Immerse.io to uphold a high level of quality for the company’s awesome products. But where to start?

After I put together my test strategy I needed to know how much test coverage was currently in place.

Plenty of manual testing was taking place but it needed refining as not everything needed to be tested manually. Also, there were no test cases for the new system documented or being run so there was no way to know how much had been tested, what test was last run let alone what results those tests gained.

So with no test cases to look at, I turned my attention to the developers and their unit tests. The good news is that they had unit tests, the bad news is that they didn’t know how much. There were no code coverage tools integrated into their builds so they couldn’t tell.

So I first turned my attention to increasing unit tests in the projects and adding code coverage tools. Once we had the code coverage levels, we could work to increase the code coverage of the project codebases. If our code coverage level was high, it would give us more confidence about the quality of the system at a low level. But how many test cases should we create?

With this question in mind, I started thinking how I could use the Test Pyramid concept and apply it across our products.

What is the Test Pyramid?

The Test Pyramid is a concept that is used to determine what type of tests and how many of each type should be used at different levels within your product’s development. There are usually three different levels to the pyramid (although some depictions may contain between three to five). The image below is referenced from Martin Fowler’s blog post regarding the Test Pyramid.

As you go up in the levels there should be less tests. Also, as you progress up the pyramid, the tests become more complex because they involve connecting to more than one component or system. Therefore tests will take longer to run as you progress higher.

The Different Levels

The bottom level contains the majority of your tests. Unit tests are used for this level as run fast, have a small focus, are easy to add to and require little maintenance. Overall, they provide fast feedback on changes to the system at a low level.

The middle level contains your integration tests and the top level is for UI tests. The tests in the middle and top levels tend to be more susceptible to breaking because they connect with a number of components and systems.

UI tests are particularly brittle because they rely on web page elements which can be changed easily. Because the automation code is so tightly coupled to the page elements, if one is updated without the other, the tests will break. And, as these two codebases are usually maintained by different teams, sometimes the teams won’t remember to update one another. This could lead to one codebase being updated without the other one. These tests usually require more maintenance than unit and integration tests.

Fix issues as early as possible

The Test Pyramid supports the idea that a lot of thorough testing at the low levels and early phases of development helps to prevent bugs reaching the production environment.

Catching issues within your product during the later stages when the project is completed and ready for consumers is costly and can be more time consuming to fix. Attempting to apprehend issues, before they become defects i.e. production bugs, during the early phases of development is preferred because:

  • It involves less people to fix so can be presumed to be less time consuming
  • There is no impact to the customers so it’s less costly to the business

In your project, try and do the following when focusing on building your base level:

  • You should try to do the majority of your testing during the earlier phases of development with tests that are the cheapest to run.
  • It’s important to make sure that the amount of unit tests in your project is large enough to ensure that issues are caught and fixed before shipping.
  • Make sure you monitor the level of code coverage you have in your project to make sure you have the right amount to reach your target level of quality that will provide you with confidence in your product.

Follow the Test Pyramid and see what results you get.

Integrating Code Coverage

In order to follow the Test Pyramid concept, we first needed to build up the amount of tests in the base level.

Now that the tests were being added to, we needed them to be integrated within the CI tools we were using. So for one team, I set about getting these running within TeamCity. Using the Unity command-line test runner I managed to get this working when we were using Unity 5.4. Unfortunately, we found that this set up broke when we upgraded to Unity 5.6. Luckily, this has now be rectified in version of Unity 2017.2 and we have unit tests running in TeamCity on every build once again.

Unfortunately, there’s no way to measure code coverage on Unity projects so we’re just going have to be content with increasing the number of tests and making sure that these tests are meaningful. I’ll continue to monitor this though.

I next looked at getting a code coverage tool installed for the other team. Because of the tech stack here and how simple it was to setup on a simple project, I chose to use Istanbul. This has now been set up into a CI tool that runs on every push so that all code is checked and a code coverage report is produced. This gives us an idea of where we need to focus our efforts to increase the amount of unit tests.

Our Next Steps

As the production of unit tests are continuing and the developers are actively doing this themselves, I turn my attention to the next level of the Test Pyramid, the Integration tests.

This post was exported in one click from Google Docs into WordPress with Wordable.