The Headache with Some Web Performance and Load Test Projects
If you have used Visual Studio for load testing, you may have experienced playback issues of user scenarios. When Visual Studio replays a recorded web test, errors may appear in the playback view, and at the end the test fails. This happens more often when you test a highly dynamic website or a web application with enhanced security. For example, you will get the "failed test" error after recording virtually any scenario in Microsoft Dynamics CRM and in some other enterprise applications.
What is the root cause of these errors? After recording a scenario, Visual Studio scans your test for dynamic parameters and creates extractors. This process is called autocorrelation. Unfortunately, it misses many necessary parameters that must be also correlated for successful playback. Running a load test before all
correlation issues are resolved is pointless because broken correlation causes errors in some transactions which will fail to execute correctly and instead will end sooner. As a result, the average response times in the test report will be largely inaccurate. Relying on such test result can cause troubles for your team or clients.
This two-minute video shows an example of playback errors in Visual Studio.
Visual Studio Load Testing Errors on Playback
Note: Before recording this test scenario we disabled duplicate detection in Dynamics 365 to allow creating identical records on playback. So the errors shown in the video are caused not by the duplicate constraint violation, but by the missing dynamic parameters.
There are several
Microsoft troubleshooting instructions for assisting with playback errors. Generally they recommend finding dynamic parameters and creating extractors manually. Unfortunately, these recommendations are too general and do not offer specific resolution steps for your web test.
Manual correlation is difficult, requires experience, and is not very efficient when you need to get results quickly.
Web Test Builder for Visual Studio
The solution to this problem is included with the new version of StresStimulus (v4.6). It is called Web Test Builder for Visual Studio. Its goal is to eliminate the daunting task of manual correlation in Visual Studio. Web Test Builder brings the power of the StresStimulus auto-correlation engine to Visual Studio users. It allows to save time on developing web tests and extending testing coverage of applications that were difficult or virtually impossible to test.
StresStimulus is used as a script builder. It will create all necessary parameters and then generate a web test that will playback successfully in Visual Studio. The next two-minute video shows how to create a web test in StresStimulus and run it in Visual Studio error-free.
Fixing Visual Studio playback errors with Web Test Builder
Here is the step-by-step instruction on creating a proper web test:
- In StresStimulus start the test wizard and record your test scenario (see this video)
- After recording is complete, the wizard will automatically create all necessary extractors and related parameters.
- Save the recording as a web test.
- In Visual Studio add the web test to an existing or a new Web Performance and Load Test project.
- Make sure that the StresStimulus.Webtest.dll located in the StresStimulus installation folder is referenced in the Visual Studio project. This is a runtime module for handling custom extraction rules that are necessary to pass the test.
After completing these steps the web test will run correctly.
It should be noted that if you need to modify the recorded scenario to meet some specific test case requirements, then it may be necessary to create some additional custom parameters. For example, if your application imposes duplicate constraints, then you need to parameterize requests so on every test iteration, a unique record will be inserted in the application database.
Only parameters created out of extractors, which are typically used for correlation purposes, will be migrated to the web test. Other parameters based on the data sets, data generators and functions, which are used for custom parameterization purposes will be not migrated. Such parameters should be created in Visual Studio. If after importing the web test more elaborative changes are necessary, it can be converted to a coded test.
Takeaways: Web Test Builder for Visual Studio provides an easier way to overcome web test playback errors automatically without manual correlation. Finding and creating all of these parameters manually in Visual Studio would be difficult and time-consuming. Using StresStimulus as a test generator for Visual Studio when you have playback issues will not only save you time, but also enable you to test some applications which previously were virtually un-testable. Web Test Builder for Visual Studio is available
here.