|
Brianna Blanchard Posts: 76
7/9/2013
|
***The issue was resolved***
During a test with up to 100 users and 2000 iterations (each iteration is 15 requests), between 2 and 11 requests consistently do not appear to complete. By this I mean the test stays at 98% complete for a long time (several hours in one case), waiting on these requests, until I click on the "Skip" button and these requests are moved to timeout. However, when I view the error afterwards, the requests that are listed as having timed out have response code 200, and show the data I was expecting in the response body. Furthermore, our server machine shows no errors.
Between the requests that have issues over different runs, the parameters substituted and the VU are different. It does seem like the same base 2 requests (before the substitution) are involved, but it only happens in a few of 2000 requests that use the same URL. This has occured on at least two test cases (different datasources, otherwise the same)
As a note, when I tried running tests that use the same base URLs with datasources that result in reponse times that 3-4 times slower + more data being sent, the issue does not occur.
I took a run using SQL CE and will upload a support case.
|
|
|
0
link
|
|
Brianna Blanchard Posts: 76
7/9/2013
|
Hmm, I seem to be getting errors when trying to upload the support case. Not sure what to do.
|
|
|
0
link
|
|
Max @StresStimulus Administrator Posts: 101
7/10/2013
|
Once you click "Skip", all currently pending requests are (logically) removed from the "pending" group, so they are no longer blocking other queued requests from being sent. These requests are not actually aborted, but just marked as timeout in StresStimulus. So, the actual response can come later, but StresStimulus will count them as timeout, even if the response code is 200. This is why "skipped" request may have 200 and will be marked as time out. Therefore, not marking the skipped 200 responses as completed is normal. Also, it is normal for the server to not report any errors because slow responses are not an error for the server.
However the test should not wait several hours to receive slow responses. The timeout mechanism in StresStimulus is specifically designed to avoid this. Check this post describing the timeout concept http://www.stresstimulus.com/blog/post/timeouts-for-smoother-performance-testing The default timeout is 300 seconds. Make sure that this value is not too high in your tests.
Which version of StresStimulus you use? Try to run the same test with v 3.0
A side note: when the test completion condition is set to "number of iterations", the progress bar moves every time a request is sent. So 98% complete means that almost all requests are sent, but StresStimulus is still waiting for responses. This behavior is changed in 3.0 where the progress bar moves every time a response is received. So, large number of pending requests will be reflected in a lower percentage value on the progress bar. BTW, in 3.0 the progress bar is replaced by a runtime dashboard that displays much more information.
Not sure why you could not upload the support case. What error message did you get?
|
|
|
0
link
|
|
Brianna Blanchard Posts: 76
7/11/2013
|
I am using v2.5 because licensing changes after that version remove the automated start feature we need. We may be upgrading soon, but the changes made to licensing eliminate updating the version as an option for now.
The sessions that "timeout" appear as 200 errors immediately (as in as soon as I can click over) after I hit abort --- if it were truly a very slow response from the server, it doesn't make sense that it would always get the response the moment I give up. Furthermore, the tests that were giving the errors are actually some of my lightest load tests, and very similar tests previously showed response times of below 30 ms, so it seems unlikely (though obviously not impossible) that it is taking the server over 20 minutes to return a response.
As for the "timeout" setting: I have made never made a changes to the timeout setting, but I don't actually know if that setting applies here: I am sending invidual requests which are not counted as pages, so I cannot even see the timeout setting under the "Pages" node. Whether this is because the automatic timeout is only for "pages" or if there is a bug in the timeout, it is definitely not automatically timing out after 300 seconds. The most recent case was 23 minutes, and as I said before it has taken several hours.
Everytime I tried to upload the test, I get a Windows dialog informing me that "SSUpload has stopped working". I sent an email to support with the some files that Windows said was related to the issue. I've tried only sending the test case, as well as trying to send all three. As a note, I was originally saving data using SQL server, and switched to SQL compact to take a run to send in.
|
|
|
0
link
|
|
Max @StresStimulus Administrator Posts: 101
11/26/2013
|
The issue with uploading test case is resolved, because the user was able to upload the test cases thereafter.
|
|
|
0
link
|