|
Shasha Posts: 5
8/16/2012
|
Hi, Team
I'm so glad to find your extremely easy to user tool that our company is considering to user your stresStimulus as our load test tool for our product. I have downloaded 7 days trial with unlimited VUs. Since I'm a fresh man for load testing, I have several questions about your tool.
I'm wondering how the SteadyLoad and StepLoad working. I have known VUs are working independently after I view discussions between you an your other clients. But I still have some;
1, If I select steadyLoad, does it mean all VUs are working at same time(ie. Login funcion testing, all VUs click Login button at the same time)? Or they just start at the same time?
2, If I select StepLoad. The setting is Start VU is1, Max VU is 50, Step VU Increase is 1. Step Duration is 5s. Now my confusion is all about Step Duration. Does is mean we add 1 VU for every 5 sec? How about the test case can't complete within the Step duration, like I have a complicated test case(just for example) it need more than 5 sec, but my Step Duration is 5, what will it happen?
3, If I select SetpLoad, does it work as the way that the first VU starts test case, after the time which is defined by Step Duration, the second VU starts running(I set increase is 1). Here the first user may be not completed the case for one time or maybe the first user has already run the case for several times, is that correct? Other user don't care if the previous user is finished.
4, After the whole test finished, the average response time is calculated per same session( sum all user's same session) no mater which Load I choose, Steady or Load. Is that correct?
That's all my questions. Besides that, I really like your analysis of test result, but considering my low level, I'm wondering where I can see the detail examination for the technical term so that I can learn what's meaning of each noun or curve. Do you have any department for load test green hand? :P
Waiting for your reply.
Thank you.
|
|
|
0
link
|
|
Vadim @StresStimulus Administrator Posts: 583
8/16/2012
|
Hi Shasha,
Thanks for your kind feedback. Here are the answers to your questions:
1. Let's say, that in your test scenario the first requests is issued to the login page, and the second request invokes the logging function. In steady load, StresStimulus tries to instantiate every VU and issue its first request as fast as possible. So, you can say that all VUs start at the same time. However, if the number of VUs is very large and the test machine is not too powerful, then there will be some delay between the first and the last VU sending their first request. After the initial start, all VUs proceed, as you correctly noted, independently. It means that each of them maintains its own set of TCP connections to the server and waits for their own response.
Depending on your web application performance, concurrent requests, even identical, might require different processing time. Therefore, while all requests #1 are issued almost concurrently, their respective responses might arrive with different delays. It means that because each VU waits its own response before issuing the second request, the logging function will be called by VUs not necessarily at the same time. You can use the analogy of runners starting at the same time but crossing the finish line on their own time. The same happens with VUs. The longer your load test goes on, the greater the chances are that the VUs will be more apart.
Sometimes it is desirable that the first request from VUs is issued with some small interval. In this case you can spread VU launching moments by using the step load with a very small (e.g. fraction of a second) step duration.
2. For simplicity, let's imagine that your test is set to run for a very long time. In your example of the step load, as you correctly noted, 1 VU will be added every 5 sec until the number of VUs reaches 50. It should take 250 seconds to go from 1 VU to 50 VUs. After that the number of VUs will be 50 until the test completion. Following your example, let's say that the test case requires 10 second to complete. Then in 10 seconds VU1 will complete its first iteration, VU2 will complete 1/2 of its first iteration, and VU 3 will just start.
3. All your assumptions here are correct. Every subsequent VU will start its test iterations completely regardless of all previous VUs. This is another way to say that VUs are independent.
4. Correct. Every test iteration is used to calculate the average response time, regardless of what VU performed it.
For more information about meaning of the performance characteristics, check the following posts:
On your last question, if you're interested in our service department, send me a private message.
Cheers,
-Vadim
|
|
|
0
link
|
|
Shasha Posts: 5
8/17/2012
|
Hi, Vadim
Thank you for your detailed answers.
Would you please answer me the following questions?
1, In the test result I can see the Performance Counters with a graph. It contains Total and Fiddler processor Time and Virtual Bytes. Is there any other place where I can see the same result with detail statistic table?
2, My understanding is Total processor is CPU usage in all, including all processes running in current computer. Fiddler processor means how much usage the Fiddler process occupies. If I'm correct, I have question. From one test result of my websites, there is a spot that Fiddler processor time is bigger than Total processor time, what does it mean?
3, Is there any way to see windows performance counters on the Server via StresStimulus?
Thank you in advance.
Shasha
|
|
|
0
link
|
|
Vadim @StresStimulus Administrator Posts: 583
8/18/2012
|
Hi Sasha,
I copied your questions to this new discussion for better searchability. Please find answers there.
Cheers,
-Vadim
|
|
|
0
link
|
|
Shasha Posts: 5
8/20/2012
|
HI, Vadim. Thank you for your explanation. But after I viewed the extra information, I still have a question(it's also about my first question of this discussion, I'm afraid that I didn't say it clear).
Is there a way to see the average CPU usage by same request? Since the graph is showing the general result of CPU usage, I'd like to see the average CPU usage statistic which is the same as the table in Request Details. After all, if I found a request is timeout or costs too much time, I want to know the server CPU usage at that time(calculate in a same way with requests as my understanding, per user's request) So is it practical?
Thank you.
|
|
|
0
link
|
|
Shasha Posts: 5
8/20/2012
|
Sorry for asking again. Besides the previous one, there is another question.
I can't connect to our server to get CPU counter via StresStimulus, so I launched perfmon. I got an weird error "No such interface supported" when I connect to other computer. I have done extentive google search and I have tried:
1, Added the 'Performance Logs and Alerts' to the firewall exception
2, Started the 'Remote Registry Service' (without it I get the 'path not found error'
3, Start the 'Performance Logs and Alerts' service.
4, Make sure I'm administrator on local and server.
5, Make sure it's a member of the "Performance Monitor Users" group on the target machine.
These solutions can't fix the issue for now. I know you are focusing on StresStimulus and my question is all about perfmon which belongs to microsoft. But you might have clients who had the same issue as mine. Maybe you can provide good suggestion or idear. I hope you can point me out.
Thank you very much.
Shasha
|
|
|
0
link
|
|
Vadim @StresStimulus Administrator Posts: 583
8/20/2012
|
Shasha,
As I understand, you need an average server CPU usage during the time-period of processing a request. Perform returns instant values. If you need to aggregate Perform data, you can do it manually or using a third-party tool (which I did not try) as provided in this post http://serverfault.com/questions/128445/how-to-aggregate-data-in-wi...
Shasha said:
Is there a way to see the average CPU usage by same request? Since the graph is showing the general result of CPU usage, I'd like to see the average CPU usage statistic which is the same as the table in Request Details. After all, if I found a request is timeout or costs too much time, I want to know the server CPU usage at that time(calculate in a same way with requests as my understanding, per user's request)
Thank you.
|
|
|
0
link
|
|
Vadim @StresStimulus Administrator Posts: 583
8/20/2012
|
If you cannot connect Windows Perform to your server due to a network security issue, then StresStimulus will not be able to access your server performance counters as well. I don't know enough about your network topology / security to assist you. Check if you can get support on this issue from Microsoft.
Good luck,
-Vadim
Shasha said:
Sorry for asking again. Besides the previous one, there is another question.
I can't connect to our server to get CPU counter via StresStimulus, so I launched perfmon. I got an weird error "No such interface supported" when I connect to other computer. I have done extentive google search and I have tried:
1, Added the 'Performance Logs and Alerts' to the firewall exception
2, Started the 'Remote Registry Service' (without it I get the 'path not found error'
3, Start the 'Performance Logs and Alerts' service.
4, Make sure I'm administrator on local and server.
5, Make sure it's a member of the "Performance Monitor Users" group on the target machine.
These solutions can't fix the issue for now. I know you are focusing on StresStimulus and my question is all about perfmon which belongs to microsoft. But you might have clients who had the same issue as mine. Maybe you can provide good suggestion or idear. I hope you can point me out.
Thank you very much.
Shasha
|
|
|
0
link
|
|
Shasha Posts: 5
8/22/2012
|
Hi, Vadim
Really thank you your patient reply. All your answers help me a lot. Thank you again.
I have gone deep into StresStimulus these days and started to use parametrization part. As I said at the very beginning, I'm new on load testing. So I have some questions about this part. Thank you in advance.
My application is a silverlight system. Here's a very simple test case; add a new record and save. User need to input ItemName and Price. The ItemName can not be duplicate.So there will be a validation when he submits, if what he typed has already been existing in DB, he will get an error and save failed, but my test case is typing an valid value and save successfully.
I want to simulate 200 users perform this action at the same time(SteadyLoad), and number of iterations is 1. My questions (all my questions are based on this precondition):
1, Now is it required to prepare 200 test data which has different Itemname and price for my test case? If I don't set any data for my test case, what will it happened? I think all 200 new records can not save to DB, so the save action is not simulated correctly as my expected, is that correct?
2, If the answer of NO.1 is Yes. I have another question. If I want to test in the mode of step load, increase 1 user per 5 sec. Due to the fact that I'm not sure how many times the test case will be performed since the first user must run more times than the last user. How many test data I need to prepare?
3, If I want to test many users, ie. tens of thousands of users, do I need to prepare same numbers of test data for all new added records?
Thank you very much
shasha
|
|
|
0
link
|
|
Vadim @StresStimulus Administrator Posts: 583
8/22/2012
|
Hi Sasha,
For better searchability, I copied your questions to this new discussion and pasted the response there. If you need to further post on these questions, please do it in this new discussion.
We need to keep every distinctive question in a separate discussion, so other community members can easily find the answers.
Thanks,
-Vadim
|
|
|
0
link
|