|
Brianna Blanchard Posts: 76
11/22/2013
|
***Was developed in v3.5***
I am using Version 3.0.5061. I will upload one of the test cases where I saw this occur to the support repository.
I have a fairly long dataset (3900 rows). I have a set of 16 requests, which are all parameterized using the data set, one parameter per column. I selected the "Sequential" option. This is how I ran my tests for months earlier this year. I recently returned to using StresStimulus, and upgraded to Version 3.0.5061.
When I run my load test, some of the values for the second column are randomly swapped, meaning that the parameter from column 1 will be from one row, but the parameter from column 2 in the same URL will be from another row. From what I have seen so far, the two values are swapped, so one URL will have, say, parameter 1 from row 10 and parameter 2 from row 11, while the next URL will have parameter 1 from row 11 and parameter 2 from row 10.
This essentially renders the tests unusable, because I have to comb through all of the results to determine which errors are caused by this bug, and which, if any, are from my web application.
Which URLs, VUs, and rows are affected is different every time I run the test, and I see this error both with an old dataset I was using successfully before, and a new dataset.
|
|
|
0
link
|
|
Brianna Blanchard Posts: 76
11/23/2013
|
I am also seeing this issue with a shorter dataset of 100 rows. With the shorter data set and a lower load, I see the issue less often, but still consistently.
|
|
|
0
link
|
|
Max @StresStimulus Administrator Posts: 101
11/25/2013
|
There is no change in 3.0.5061 in respect to databinding. The sequential data binding method always worked the same as described in paragraph 3 of this post http://www.stresstimulus.com/blog/post/pro-edition-beta-part-1
"Sequential (default) - dataset rows are selected in round-robin order as they got consumed by every parameterized request of every VU on every iteration. Every parameter has its own independent counter of the next data set row to be used."
The following example shows how 4 records from a dataset will be consumed by a test case with 5 requests (from I to V), each of which has 2 parameters A and B, bounded to the dataset fields A and B. Altogether there are 10 parameters and each of them has its own counter. In this example two VUs run the test through two iterations. Note that the VUs traverse through their respective iterations asynchronously, as they emulate independent physical users, so there is no synchronization between requests issued by different users.
| Request | VU1 | VU2 | VU1 | VU2 | | Iteration 1 | Iteration 2 | | A | B | A | B | A | B | A | B | | I | 1 | 1 | 2 | 2 | 3 | 4 | 4 | 3 | | II | 1 | 2 | 2 | 1 | 4 | 4 | 3 | 3 | | III | 2 | 2 | 1 | 1 | 4 | 3 | 3 | 4 | | IV | 1 | 2 | 2 | 1 | 4 | 3 | 3 | 4 | | V | 1 | 1 | 2 | 2 | 3 | 3 | 4 | 4 |
I inspected the test that you uploaded and saw that each of your 16 requests has two parameters, using both data set columns. If you need parameters A and B in the same request to be bound to the same record in the data set, then, you need to use a different data binding method. For example, if you use VU-bound method, you'll get the following result:
| Request | VU1 | VU2 | VU1 | VU2 | | Iteration 1 | Iteration 2 | | A | B | A | B | A | B | A | B | | I | 1 | 1 | 2 | 2 | 1 | 1 | 2 | 2 | | II | 1 | 1 | 2 | 2 | 1 | 1 | 2 | 2 | | III | 1 | 1 | 2 | 2 | 1 | 1 | 2 | 2 | | IV | 1 | 1 | 2 | 2 | 1 | 1 | 2 | 2 | | V | 1 | 1 | 2 | 2 | 1 | 1 | 2 | 2 |
Will this work for you? In v 3.5 we will add several more data binding methods. The beta will be released in a few weeks. If you describe how you want to connect requests, parameters, iterations, and VUs with data set records, we will check if we can add your data binding method it to v 3.5.
Does this make sense?
|
|
|
0
link
|
|
Brianna Blanchard Posts: 76
11/25/2013
|
I understand what you are saying and I appreciate the detailed response, but what doesn't make sense to me is that I used the sequential method for about 3 months earlier this year without ever once encountering the behavior I'm seeing now. Perhaps there was a bug before that was fixed, thus changing the behavior.
VU bound does not work for me in about half of my tests, unless I misunderstand how that works. For example, I want to have a load of 20 users, but I want to cycle through at least 100 different parameters over the course of my test.
I have to have the first parameter in the URL be from the same row in my dataset as the second parameter. It would be best if all of the requests in a single page or a single iteration used the same two parameters, but it is not vital. Is there another way to achieve this besides VU-bound? If there isn't, would it be possible to get an earlier version of SS, say from August?
|
|
|
0
link
|
|
Max @StresStimulus Administrator Posts: 101
11/25/2013
|
I don't remember us fixing bug in this area, but I know for sure that we did not have data binding method that you need. I will email you v2.8 that was a prod version in Aug-13. Try it and let me know if it works differently, which I doubt.
Based on your clarification, I agree that VU bound will not work for you either. I believe that here's what you need:
| Request | VU1 | VU2 | VU1 | VU2 | | Iteration 1 | Iteration 2 | | A | B | A | B | A | B | A | B | | I | 1 | 1 | 2 | 2 | 3 | 3 | 4 | 4 | | II | 1 | 1 | 2 | 2 | 3 | 3 | 4 | 4 | | III | 1 | 1 | 2 | 2 | 3 | 3 | 4 | 4 | | IV | 1 | 1 | 2 | 2 | 3 | 3 | 4 | 4 | | V | 1 | 1 | 2 | 2 | 3 | 3 | 4 | 4
|
If you confirm it, we will develop it in v3.5
|
|
|
0
link
|
|
Brianna Blanchard Posts: 76
11/25/2013
|
That would be great: what you described is exactly what I need. I'll let you know what I see when I use the older version.
|
|
|
0
link
|
|
Max @StresStimulus Administrator Posts: 101
11/26/2013
|
OK. This will be a new VU-Iteration data-binding method. Every VU on every iteration gets a unique value. The value is the same for all requests within the iteration.
This feature will be included in the initial 3.5 beta release. Expected availability time - several weeks from today.
I moved this post to the feature request forum.
|
|
|
0
link
|
|
Max @StresStimulus Administrator Posts: 101
1/2/2014
|
Just confirming that this feature is available in v3.5 beta that was released on 12/13/2013. The feature is described here in item 2.
|
|
|
0
link
|
|
Brianna Blanchard Posts: 76
1/2/2014
|
The new binding method VU-Iter-Bound does what I need. Thanks!
|
|
|
0
link
|
|
Max @StresStimulus Administrator Posts: 101
1/2/2014
|
Thanks for confirming.
|
|
|
0
link
|