Whenever I go into detail when I tell people what I do for a living, I frequently hear how model-based testing is ‘The wave of the future!’. While it’s true that model-based testing (or “MBT”) tools continue to evolve and new tools are being brought to market, the concept and application of MBT has been around for over two decades. I have been implementing or managing MBT projects myself for over 10 years.
There are a few “flavors” of model-based testing and I’ve gotten into deep conversations about the distinctions between “Model Centric”, “Model Driven”, and “Model Based” approaches, but that’s a can of worms we’ll leave closed for now.
In short, what I mean by the term MBT is creating a diagram that represents the desired behavior of the system under test, then using some repeatable, predicable process to derive tests that are used to verify that the system behaves as desired.
Since MBT has been around a while, some nice folks (Anne Kramer and Bruno Legeard) have put together some interesting statistics around MBT usage. I mention some of these statistics in a recent webinar, Model-Based Testing using Cause Effect Models. To me, the most interesting statistic was that the vast majority of survey respondents felt that MBT was partly or completely fulfilling the expectations they had from adopting an MBT approach to testing.
Question 3 from the 2019 MBT User Survey
One statistic that I didn’t mention in the webinar that I’ll point out now is that the overall number of hours reported to achieve maximum value from MBT has dropped 75% from 2014 to 2019!
I believe there are a few of reasons for this:
- The goals and benefits of MBT implementation are better understood by users
- Increased management buy-in, resulting in more support for those learning the process
- MBT tools have improved, shallowing the learning curve
So, the people who are using an MBT approach to testing are getting what they expected out of it and the reported amount of time to attain maximum value is significantly less than it was just five years ago; both great things.
What do these statistics mean for organizations who are looking to adopt an MBT approach to their testing?
In the aforementioned webinar, I discuss a long list of advantages and benefits that come along with the MBT approach, so I won’t rehash those here.
Instead, I’ll focus on what adopting MBT really looks like and share some of the experiences I’ve had in helping many of our customers develop their MBT capabilities.
The first thing I call out prior to starting an MBT project is the set of expectations. In my experience, there’s an assumption that implementing MBT is simply a matter of handing a new tool to the testers and expecting them to hand back a test suite and a big list of defects they detected and prevented. This is simply not the case.
If the HR department of an organization changed their time tracking and time off system, there would be an entire initiative to retrain the users and those impacted by the change (management, accounting, up- and down-stream vendors). The same has to be true when an organization changes the way they develop the tests for their software.
You see, MBT is a process, not simply a tool.
One of the crucial aspects to a successful MBT approach is collaboration. Collaboration has to happen at every step of the process to achieve the maximum value to all involved. This collaboration is not solely for the benefit of the tester; in fact, the opposite is true.
By creating a model of the desired system or application (or of a change to the system or application), you create a view of the system that allows stakeholders of nearly any role, expertise, or technical skill level to engage on the same level. In other words, you’ve created a layer of abstraction that encourages people to think in terms of ‘What’ without getting bogged down in the ‘How’.
In order to enjoy stakeholder communication benefit, the expectation must be set that collaboration will happen early and often in the development process. The ‘hurl these over the wall and have QA test them’ mantra simply doesn’t work.
One great thing that this collaboration allows teams to do, and I’ve seen this first hand with the successful MBT adoptions I’ve been a part of, is to use the model as the starting point when someone has an idea for a new feature or a change that they think will improve an existing feature.
Rather than trying to define their idea in a text-based artifact (requirements, user stores, use cases, etc.), team members will get into the habit of using the model to draw a picture of their idea as their starting point. Sharing an idea in the form of a model is a very effective way to convey the idea and at the appropriate level of detail.
This ability to go from idea to a minimum viable product to a fully functional, completely fleshed out definition using the model as the layer of abstraction saves time, saves effort, and eliminates rework.
I’ll talk more about the process of going from idea to completely defined product in the second episode of our Model-Based Testing webinar series, which you can register for here (or view on demand if you’re reading after the date of the webinar).
