|
Chris Posts: 2
3/24/2011
|
Hi,
We have been really interested in the StresStimulus plugin, it works really well for most scenarios and it has been a joy to use.
However, I have started working with the Parameterization feature, to load-test data entry sections of an application.
Firstly, I am unsure how the ramping is affecting the Virtual Users Parameters as it cycles through the iterations. I am assuming from the results we have, that on the first iteration, VU1 is active, on the second iteration VU1 and VU2 are active, and so on.
Is it the case that Parameterization uses each data row for each VU, whereas for testing data-entry in the scenario of unique logins, ideally each data row is used only once, not on every iteration? This would be a great feature to understand and control. Envisage the scenario where unique logins are required - each line of the parameters list will only be allowed once per testing run.
Secondly, following on, at present we are a little unsure how well this feature is working, and would really appreciate a session log to review the requests it is generating. This would be with an aim to compare client requests to server logs.
For example, I have configured a session with ten Virtual Users, and created a list of ten Parameterization entries. However, the tested application rarely records all of these especially if the Step Duration is short, and it would be a nice feature to see how this is behaving and responding with HTTP status codes.
Many thanks for your time and efforts,
Chris
|
|
|
0
link
|
|
Vadim @StresStimulus Administrator Posts: 583
3/25/2011
|
Chris,
Thanks for complimenting StresStimulus.
On your first question about virtual users and iterations. The assumption that VUs are added on certain iterations is not fully correct. Virtual users and iterations are ramping independently. Every VU once it is created will cycle through its sequence of iterations (independently from other users) until the test completion. At the same time, VU ramping is a liner progression over time. Let's take an example where Start VU=10, Max VU = 100, Step VU increase = 5, and Step Duration =10s. In this case test starts with 10 VU and then every 10 seconds the number of VU will increases by 5 until it reaches 100 (unless the test will end sooner due to selecting a short test duration). This VU ramping is completely independent from the pace at which each VU cycling through its own succession of iterations. This approach emulates real-world scenario where users navigate web application independently.
On your question about how parameterization is related to VUs. At this time, they are independent. On every request (regardless of which VU initiated it) a recorded parameter is replaced by one of the data row values selected randomly from the dataset’s designated field. For example, if dataset has values apple, banana, pear and, orange, then on first three iterations VU1 can request banana, apple and orange and VU2 can request apple, orange, apple. This can work to simulate scenarios when multiple VU independently populate web forms (i.e. order entry form) with a set of values (i.e. product inventory). You are making a good point that this approach does not support uniqueness when each value must be used not more than once.
On your question about sessions log. StresStimulus saves internally every request / iteration created by every VU. This information is not added to Fiddler during the test for better performance, but can be displayed after the test completion. In Load Testing Parameters -> Other options –> “Add Load Test Sessions to Fiddler“, select “All users” (see screenshot below). Help topic “Fiddler Grid” describes how load test sessions are tagged with VU number in the column “Process”, and with iteration/request number in the column “Comment”. You can analyze these load test sessions in Fiddler or use available export options. Also, if you need to see status codes of HTTP responses, un-check “Purge test session’s response body” in StresStimulus options.
You are correct in observation that in short scenario with 10 users and 10 values not all values may be used. This is because they are selected randomly, as I showed above. We will add other patterns of handling datasets and parameterization. For example, instead of selecting data rows randomly, it will be an option to pick them sequentially. I wonder if this will address your requirement of logging uniqueness. We will also look into more fine-grained control over binding VU to datasets.
Please let me know if I answered all of your questions.
-Vadim
|
|
|
0
link
|
|
Chris Posts: 2
3/29/2011
|
Vadim,
Many thanks to you in taking the time for such a comprehensive reply.
This is really useful information and apolgies for not fully investigating the options before posting. Now that it has been explained and the Test Sessions can be viewed, overriding the randomisation in preference for serialised sequences would not be desireable except for rare occasions.
I think options for controlling or binding uneque parameters are all that are missing and the option to force unequeness of the dataset entries would be useful.
Optionally, it may be invisaged in future that a tester would not want to clear the entries in the tested application after each load-testing run, so the option to persist those marked data entries across several runs of the same loaded test session may also be desireable.
Many thanks for working on such a great project,
Regds,
Chris
|
|
|
0
link
|
|
Vadim @StresStimulus Administrator Posts: 583
3/30/2011
|
Chris,
Thanks for your feedback. We will keep your thoughts in mind while adding features to StresSTimulus
Cheers,
-Vadim
|
|
|
0
link
|
|
Vadim @StresStimulus Administrator Posts: 583
5/11/2011
|
Update: In the version release today we developed 2 more options for data-binding in parameterization: sequential and VU-bound. This should fix some of the issues addressed in this post.
For detailed check this blog.
Note: These options are available in PRO Edition only.
|
|
|
0
link
|