When developing new software or maintaining an existing application, often we want to Automate the QA Testing process to make the testing process faster, more efficient, and more accurate.

But what does QA Test Automation mean?

All of the following are a form of QA Test Automation:

• Unit Testing
• User Interface Testing
• API or Integration Testing
• Regression Testing
• Load or Stress Testing
• Performance Monitoring or Synthetic Testing

‍

Unit Testing

As a Software Developer, typically you will create a small script with example data to automatically run it against the Portion or “Unit” of code that you are working on.  Once the script is built, this Unit Test can be used numerous times, in order to help the Developer fix any errors in their code.

This is an extremely important process, as the earlier we can catch errors, the cheaper and easier it is to fix.

‍

Graphical User Interface (GUI or UI) Testing

When automating Testing for a Graphical User Interface (GUI), a QA Engineer will use a separate Application known as a Testing Framework, to create a series of scripts that mimic what a specific Persona or User Role would interact with the Application that the Development Team is working on.

This way when new releases (a.k.a. builds) are ready for Testing, the QA Engineer only needs to modify the Test Plan Script and then run it as the various User Roles.  This greatly speeds up the process of testing the User Interface of the application, as any failure or inconclusive results will automatically be recorded.

This type of Automated UI Testing, can be applied to any type of User Interface:

• Desktop Applications
• Web Applications
• Tablet Applications
• Mobile Applications

‍Considerations

• While this will significantly speed up Testing of a UI, it is not perfect.
• The Test Plans & Scripts need to mimic how the User will actually use the application.
• A poorly written Test Plan, will produce a False Positive result – meaning you think it is working properly, but it may not in reality.
• Users often will do things or click on things, that the Development Team would never have expected them to do.

• In order to be useful in the long run, the Test Plans & Scripts must be continuously updated with any new UI changes or modifications.

• This does not replace either:
• “Gorilla” Testing – where someone on the Team does manual testing of the application as a User Role, trying to specifically break the application or cause a failure.
• User Acceptance Testing – by the actual Users, where they test the application or release before it is put into Production.

Learn more about user acceptance testing in quality assurance by clicking here.

‍

API or Integration Testing

When integrating two or more applications, you will need to test the various messages and communications going back and forth between the various applications, and automating these Tests is often the best approach.

This would include Testing the Message for:

• Confirmation of Message Delivery
• Correctly Processing the Message
• Confirmation of Receipt and Processing – Return Message
• Performance or Time it Takes
• Security of the Message, Communications Infrastructure, and both Application’s Interfaces (API’s)
• Error Logging and Handling

Typically, this type of testing is done completely independent of the GUI of either application and focuses solely on the data message being sent from one application to the other, and back.

‍

Regression Testing

As the Team is continuing either to build an application, or maintaining an existing application that is already being used in Production, it becomes critical to test all of the modules and functions within the application to ensure they were not affected by the change.  As even a minor change can potentially impact an entire application.

Often using a manual testing approach would be too time consuming and expensive, which is why automating these tests is a more effective and efficient way of detecting and finding defects, before the code is promoted to Production.

However, this is not a panacea.  When dealing with an application with potentially millions of moving parts, even the best Regression Test may not catch everything.  Every Senior IT Leader has their own example story…

When I was the VP of Software Development at Universal Technical Institute (NYSE:  UTI), we had a request from the business to change a single text field in the core database from 256 characters to 512 characters.  We ran a quick automated regression test, and it was fine, so we agreed this was an extremely low risk change.  Unfortunately, our standard Reporting Tool interpreted this change as going from a text field – which you could use for calculations, to a memo field, which it could not use.  So, this very simple change ended up breaking nearly 2,600 standard reports used throughout the entire organization. 

‍

Load or Stress Testing

This form of automated testing is used when you want to find out at what point does the Performance of your application start to slow or even make the application crash.  In other words, how many:

• Users can the application support at a give time.
• Transactions can it process simultaneously.
• Reports can be run.
• API or Integrations calls can be handled.
• Data layer interactions.
• And the overall Performance of the System, over days or even weeks.

There are many different methods for how to conduct this type of Load or Stress Testing, depending upon what your budget allows and the business criticality of the application or module that is being tested:

• Write a simple Server Job to run on a frequent basis, mimicking a particular function.‍
• This is useful for testing User Logins, API access, etc. and is relatively cheap and easy to do.
• The drawback is that it doesn’t fully test the application, or identify specifically where there are performance bottlenecks or issues in the application.

• Use a Cloud Based Stress Testing Tool (rental option).
• Basically you create the Test Plan.
• Run it for the given amount of time you want.
• And are only charged for the Time and Number of Test Runs you use.
• This is a good option for when you have a relatively stable application with only few periodic changes.
• These tools can provide you with a detailed analysis of any areas within the application that are performance bottlenecks, as well as information on what the infrastructure can actually support.
• The main drawbacks are that you have to design the Test Plans and it is a pay as you go system – as you won’t get the 1st Test Plan perfect, you will go through multiple iterations.

• Purchase a Load / Stress Testing Tool.
• This is basically the same as using a Cloud Based Tool.
• The only difference is that you “Own” the tool and can use it to run as many Test Plans, as frequently, and as often as you want.
• This is very good when you are building a brand new enterprise application.
• The downside is that it costs more up front.

‍

Performance Monitoring and Synthetic Testing

This form of automated testing is most commonly used after the application is “Live” and in Production or a Pilot and being used by actual Users.  The QA Engineer will use an Application Performance Monitoring Tool and create one or more scripts to test various parts of the application, which is run on a frequent basis (every 5, 15, 30, or 60 minutes).

This tool will then send an entire transaction set to test the application and whether it is working correctly.  With each function, the application returns to the Tool, information like:

• How long did this single function take to complete?
• Were there any errors?
• Details of every single Sub-Function or Micro-Service Call.
• Total time to complete the entire Synthetic Transaction.
• And performance of the entire System and it’s Components at the time.

Using all of this data, the IT Operations Team and Development Team can look for ways to optimize and improve the application.

‍

Bottom Line

Automating the Testing of your Software Development efforts can enormously help improve the:

• Quality of your software
• Reliability
• Security
• Performance
• Scalability
• Time to Market
• Help detect Problems in Production when they occur

However, it takes an investment in Time, Resources, and Tools, and then must be maintained in the future to continue to receive these benefits.  And it won’t replace the need to do some manual testing as well.

Learn more about automated quality assurance here.

We hope that this has been beneficial and useful to you, if you are in need of software development contact us here.