Model Based Test Automation

Overview and current challenge in existing testing

Test automation paradigm always keeps "evolving." Organizations have witnessed and been part of this evolution over the years. Test teams are part of the "Been there, done that" club when it comes to conventional test automation techniques. Some of us are literally sitting on the heap of automated test scripts, feeling jittery, unsure whether the project team has adequate test coverage. Automation engineers have always automated the test cases and updated the scripts for changes in test cases, created new test cases, and removed test scripts that are no longer valid. Teams strive to maintain different test case and test script versions; the list is unending.

People struggle to identify and schedule the tests to be executed and address the exponential rise in demand for testing against various environments. Then there is a group that always says, "It's automated… Why not run everything!"

Today the challenges test managers face are not new but are becoming critical with the speed at which things are changing. The response time is getting squeezed like never, and everyone is struggling to bring innovation, AI into test automation.

Automation Strategy Shift

Over the years, traditional test automation approaches have given benefits, but with the huge upfront cost depending on multiple factors like infrastructure, tools, technologies, skill sets. We are witnessing a shift from conventional test automation methods that use test cases/stories as a base for creating test automation to a "Model" based testing approach.

Even though the transition is not difficult, there is a greater need to change the mindset and gain a higher comfort level with model-based test automation. Test managers who are used to maintaining vast sets and versions of test cases may get a little uncomfortable seeing a model and no test cases at all!

There are already tools available to create a system's model, and many of them provide a mechanism to run a model covering different paths and points of verification. With a few iterations and practice, one can create and run models to comprehend how and what kinds of virtual test cases are developed and implemented from the model.

What is Model-Based Testing (MBT)

MBT is a testing technique when we define an abstract model which describes a software behavior and then use this model to test the real software based on predictions of paths.

The Abstract test suit acts as a basis on which the executable test suite is derived. The executable test suite (automation scripts) mapped with the abstract test cases can directly communicate with the SUT(System Under Test).

Key components of model-based testing

Model

A model is a graphical representation of the system's behavior with States and Actions. The model starts with initial conditions and ends with exit conditions. Behavior can be described in terms of input sequences, actions, conditions, output, and data flow from input to output.

A Model is interpreted as or translated to a state transition system or finite-state automation. The state transition diagram is represented by the model of the application, a directed graph where nodes represent the states of the application and edges represent the transitions.

A typical model could be represented in different forms based on the tool selected. A few such examples are shown below:

Test Paths

Model outputs only user scenarios of possible paths based on initial and exit conditions. Test paths include all possible permutations and combinations of going through the intermediate nodes between initial and exit conditions. Unlike the traditional test automation approach, test paths are not conditional on written test scripts. The path taken during execution is random and depends upon the execution engine algorithm, considering various criteria such as "coverage percentage, "weightage," etc.

How Model-Based Testing Can Help

With the emergence of Model-Based Test Automation on the horizon, the possibilities of futuristic test automation ideas started making rounds of discussions. Some of our customers, the product companies, could immediately realize the advantages and benefits of MBT in ever-changing environments.

MBT has helped test management drastically reduce the efforts of test cases, script generation, review, and rework. Yet, one of the primary benefits of using MBT is test coverage. With the conventional test automation approach, coverage is influenced by the number of manual tests created or the number of test cases automated. 

There is a risk of missing multiple test scenarios while authoring the manual or automation test scripts. In the case of models, a test engineer or business analyst defines all possible states and actions for a screen or use case scenario. The test coverage percentage is enhanced drastically since the tests are generated automatically for all possible combinations of states and actions between initial and exit states. 

There are immense possibilities for further improvement in test coverage with Model-Based Test Automation using AI/ML.

Conventional Vs. Model-Based Testing

Test cases follow set procedures in the conventional testing approach, where human judgment is the foundation. To cover different paths or procedures, one must create new tests. However, in MBT, most paths are derived from Models which are tightly mapped with the requirements and automatically generated test scripts or suites. The difference is evident if we see the below workflows.

Benefits and conclusion

The conventional approach demands substantial effort to test automation, even for a slight change in the business requirement. However, updating the respective model component using MBT incorporates the changes quickly. For test professionals, MBT is a compelling methodology with the following benefits:

• It starts with specifications by reinforcing the idea that QA involvement belongs at the beginning of the discovery stage.  
• It typically finds design and specification bugs before the code even exists.
• Precisely define functional requirements to match user expectations better. 
• Reduce planning time by providing components of functional specification documents.
• Test coverage can be found even before any test scripts have been written using offline test runs using models without SUT.
• Automate test-case generation, creating more test cases in less time
• Maximize test coverage with lesser test cases
• Less maintenance of Automation scripts 
• Shorter testing cycle time