Automated Testing Frameworks

Photo from https://askgif.com/blog/57/key-features-of-automation-framework/

What are Automated Testing Frameworks:

Software testing is a crucial part of the development cycle as it ensures good quality deliverables to the users. Every company focuses on testing, and most of them are tending towards automation testing over manual testing. Automation testing saves lots of time, effort, and cost needed by resources.

The automation testing framework is a collection of the test environment, tools, processes, and reports of the application under test. The test environment should be a set of infrastructure where the application is going to run, so simulating the same environment in the automation testing framework. Tools can be anything that is used to run automated test cases. The process consists of the prerequisite execution and reverting the states of the application. Report are the results obtained after test execution, a brief summary of the result of the tests executed. Automating the framework focuses on optimizing automated processes resulting into less execution time, fast delivery, quick bug detection, Quality, reusability and saves costs and efforts

Test Automation Framework Types:

There are many types of Automated testing frameworks, like Modular Driven Framework, Behaviour Driven framework, Data-Driven Framework, and Hybrid Testing Framework, so it is crucial to adopt the right one.

Familiarize with the Application:

It’s important that before you put your hand dirty into planning and designing automation framework, you get to know the in and out of the application, not only from the feature but from the design perspective too. This will not only help in designing the automation framework, but it will also provide the visibility of how accurately you model your framework with the application.

Automation Framework life cycle

There are 5 stages of the Automation Framework cycle

Photo: Automation Framework life cycle

1. Scope Of Test Automation:

Whether it is a microservice or a UI application all need to go under test before delivery to the end customer. Now the decision lies whether we need an automation framework or just manual testing is enough. Such designs are based on the criticality of the application, size of the application, efforts, costs, and of course time. It’s good to check the feasibility of automation before developing it. For example, if the application falls under the Agile model where frequent feature development and changes happen in the code. It would be better to have an automated testing framework for the application which is a part of the application’s CI and CD. This, not only checks the new feature but also verifies the regression breakage. On the other hand, if the application is small and follows the waterfall model and it does not require much effort then it’s good to go for manual testing.

Photo from https://testingxperts.blogspot.com/

Before designing the Automation framework, consider the below points-

• how often the test suite will run?
• how many numbers of test cases that will be automated,
• what will be the smoke tests, integration, and feature test cases that will be automated ( as the framework design is influenced by such decision ) based on the application type?
• Which modules of the applications can be automated and which cannot?
• Which test can be automated and how to automate them?
• Factors like cost, team size, and expertise should also be considered.

2. Testing Strategy

Finding the right automation testing tool is a critical phase of an automation testing life cycle. Tool selection is based on the type of application, budget, technologies being used in the project, the familiarity of the tool with resources on-board, intuitiveness, flexibility, and more.

Tool selection: It depends on what type of application is under test,

• if it is a front end — you can use Karma, protractor, selenium, Lambda Test, etc or
• the backend you can use Data Factory, Data Generator good for database testing.
• API Testing Tools such as Postman, SoapUI, Jmeter, Swagger

Reporting tool selection again depends on the type of Programming language, like Allure, jest, ReportPortal.io, etc. These tools help to get insights and analysis

3. Design, development, and testing of the framework

Get ready with the Test-plan, and check the feasibly of the cases which can be automated and which cannot be automated. Once you are ready with the test plan start designing the architecture of the framework. Generally first the HLD and then the LLD with details. The design includes the environment (AWS/Azure/Google cloud or browser-based or desktop etc). Framework based on the language used to build the application. Test suite structuring, test scripts, reporting tools. Get it reviewed and make changes accordingly.

The main task of this phrase is how actually the automation framework will look and where actually it will fit in the SDLC. LLD will subsequently change sometimes so it’s necessary to maintain the document. After the design is ready to start with developing Proof-of-concept (POC) and keep on verifying with the design. Keep in mind to design a well-structured test suite which is easy to maintain and access. Like keeping the helper file in one directory, Page object into another, and test scripts into another. I will be focusing on the test-suite structure and framework design example in my next post.

4. Setting up the Test Environment :

In this phrase your need to decide where your framework will execute, whether it’s a remote machine, VM, or cloud machine. If you are going to use cloud and new technologies, you need to check whether you need to dockerize your framework or you want to use Kubernetes job or pods to run, all depends on the environment where your applications sit and how you wanted to test it. Dockerizing and testing framework saves a lot of cost and time. It also speeds up the testing process as you can avoid the time needed for provisioning and deploying the test environment.

Decide that your framework must run automatically, you can combine dockerize it, combine it into the CI or CD. This automates your task to run your framework from the CI job or as a part of the pipeline. No need to manually trigger the job.

5. Test execution, generation Of Test Results and Test Reports

Now if you want to execute your testing framework, you need to decide whether you want to do it manually (which I will never prefer) or you want to make the trigger automatic. If yes then you need can use CI/ CD tools like Jenkins, Travis Bamboo, etc where you can configure a Job or make this as a part of the CI/CD pipeline. So when CI/CD pipeline starts it will trigger your automation framework, which will be responsible to run the test and get back the logs, results, or reports(as per design).

You can also you webhooks in your SCM (where you are storing your code for automation framework) which will trigger the job whenever there is a change in your repository. Say if there is a push in the SCM and their code needs to be tested, it will trigger the CI/ CD as configured and as a part of the configuration, your automation framework will execute the test. If any breakage in the test the code will not be merged if it is CI or the code will not be delivered if its CD

The tool or the process you decided for getting back the logs, results, and report after the test has executed must be maintained in your test environment which can further be used for Analysis.

6. Analysis + Maintenance

Here we should check the results, report and if there are any breakage or test scripts fails, report it as a bug. Make sure the test scripts meet the test requirements. Adding new test cases, looking towards the stability of the framework, check for changes in the configuration or test job.