7 days ago
Topic:
Concurrency mode
George @StresStimulusAdministrator Posts: 378
|
Since you contacted support they will reply directly.
- Cheers
|
|
|
8 days ago
Topic:
Concurrency mode
sachpatelPosts: 8
|
sachpatel wrote:
Does SS have a way of dynamically handling the way requests are processed different with each iteration?
Different in what way? Can you please clarify the question. StresStimulus handles requests according to the recorded script, your script settings and your real-time server responsiveness.
I was referring to how the waterfall model can differ with each recording.
I have sent an email to the support team after running several tests with different settings, as recommended, and provided the outcome. If you could please refer to that and provide any further suggestions of options to try, that would be much appreciated. Thank you.
|
|
|
8 days ago
Topic:
Concurrency mode
George @StresStimulusAdministrator Posts: 378
|
sachpatel wrote:
Is there a best practice in terms of recording a script to capture accurate concurrency for the application?
StresStimulus recorder is designed to automatically records correct concurrency.
sachpatel wrote:
Because when I do different recordings of the same steps, the waterfall diagrams are not necessarily the same.
You are correct, two recorded waterfalls won’t be exactly the same due to multiple random factors. So the response time will not be exactly the same. However, the waterfall should have sufficient information for a rough comparison of the browser and StresStimulus concurrencies.
sachpatel wrote:
Does SS have a way of dynamically handling the way requests are processed different with each iteration?
Different in what way? Can you please clarify the question. StresStimulus handles requests according to the recorded script, your script settings and your real-time server responsiveness.
sachpatel wrote:
We recently faced an issue where scripts were ported from another test environment and a different version of CRM. In that scenario it would be worth checking the waterfall diagrams before re-scripting?
The main reason for checking the waterfall diagrams is if you want to double-check that your script settings are correct. Once your script is configured, you can run your test multiple times without changing settings or checking the waterfall diagram.
- Cheers
|
|
|
9 days ago
Topic:
Concurrency mode
sachpatelPosts: 8
|
Hi George,
Thank you for the detailed explanation. It all makes sense.
One more observation. Is there a best practice in terms of recording a script to capture accurate concurrency for the application? Because when I do different recordings of the same steps, the waterfall diagrams are not necessarily the same. This is the same observation within DevTools. Does SS have a way of dynamically handling the way requests are processed different with each iteration?
We recently faced an issue where scripts were ported from another test environment and a different version of CRM. In that scenario it would be worth checking the waterfall diagrams before re-scripting? Because the scripts work fine in the new environment, after using the "Remap Hosts" option.
|
|
|
10 days ago
Topic:
Concurrency mode
George @StresStimulusAdministrator Posts: 378
|
Let me unpack it:
Regarding your visual observation: StresStimulus measures: StresStimulus page response time and visual impression of page responses can be apple and oranges. Here is one reason for that. StresStimulus measures time between the first and last request. On the other hand, when you visually determine that the page is loaded, you do not know whether any resources are still being loaded and whether the page DOM below the fold is still being generated.
Regarding DevTools and StresStimulus time: Response time is a product of a client sending requests and the server sending back responses. To investigate your findings, you need to compare the behaviors of DevTools and StresStimulus. Execute a test with one VU - one iteration to provide the same server load as with DevTools. Then compare the waterfall diagrams in StresStimulus and Dev tools. StresStimulus waterfall diagram is described here. Then execute StresStimulus tests with various concurrency settings and see the difference
Regarding StresStimulus Concurrency modes: You are correct that Recorded Concurrency determines requests are sequential or parallel based on the recorded waterfall. A request is parallel if its response comes after the next request is sent, otherwise, it is sequential.
The Set-cookie header enforces sequential requests property is described here. You can set it to No if you have many responses with Set-cookie header that is causing sequential requests and you wish to send them in parallel, however, make sure setting this property doesn’t break your script.
- Cheers
|
|
|
10 days ago
Topic:
Concurrency mode
sachpatelPosts: 8
|
Hi George,
Thank you for the your response.
I understand the differences you have highlighted. However, what I have observed when replaying a script is responses times reported as being higher than that seen when running the same steps manually and captured the responses times from DevTools. From your explanation, DevTools should be reporting a higher response time since it includes the client side rendering.
The same thing is also observed when running the script during a load test. SS reports the average response time being 60 seconds (for example). But when I manually navigate to the page, whilst the test is running, it doesn't look to take 60 seconds to load.
The above two examples have been observed whilst having the Concurrency mode set to "Content-type". However, this led to my initial post when running a POC with different settings to observe the response times captured. And it was inconsistent based on the following factors: re-running an older version of the script compared to a newer version and the time of day. We just want to ensure the method we use is accurate and consistent.
Can I confirm when selecting "Recorded Concurrency", this will automatically determine whether the requests for a transaction as sequential or parallel? Does the Set-Cookie property also needs to be set to "No"?
|
|
|
11 days ago
Topic:
Concurrency mode
George @StresStimulusAdministrator Posts: 378
|
Dev tools capture the request/response time (server + network time) and page rendering with javascript processing time (client time).
Load testing only measures times that depend on the load/concurrency of shared resources (server + network time).
However, you can configure StresStimulus to emulate client time by adding recorded delays into the script as described here.
I would recommend using recorded concurrency mode with adding the emulated client delays for the most accurate results.
- Cheers
|
|
|
11 days ago
Topic:
Concurrency mode
sachpatelPosts: 8
|
Hi,
This question was with regards to replaying scripts and customising the "Concurrency mode" option. I understand there are a couple of options at the script level that can be used (Concurrency mode: Content-type, recorded concurrency). I have tested both options using the following scenario: 3 users, 2 iterations. These are the tests I ran:
1) Concurrency mode = Content-type (at the script level)
2) Concurrency mode = Recorded Concurrency (at the script level)
3) Customised the concurrency mode to "parallel" for a couple of transactions (at a transaction level)
Looking at the 95th percentile, the response times varied differently compared to what is captured in Dev Tools when going through the transactions manually.
1) Much slower
2) Much faster
3) More accurate
What is the best/quickest way to determine the script is replayed in an accurate manner to obtain more realistic response times when running as part of a load test? It seems very time consuming to have to go through each transaction with Dev tools open to determine which transactions are executed in parallel, sequential, etc.
Thank you.
|
|
|
16 days ago
Topic:
Extractor Array
sachpatelPosts: 8
|
That's great. Thank you George.
|
|
|
17 days ago
Topic:
Extractor Array
George @StresStimulusAdministrator Posts: 378
|
This can be done in StresStimulus v5.4 (currently in public beta) using a Multiple occurrence extractor and a loop as described here
v5.4 can be downloaded from here.
- Cheers
|
|
|
17 days ago
Topic:
Extractor Array
sachpatelPosts: 8
|
Hi,
My question was around the below scenario:
- An extractor has been created at request #100
- At request #100 from the UI there are two documents shown, with two documents IDs in the response.
- The goal is to open both of the documents.
- I have used two extractors to store the document IDs: documentId_1, documentId_2. Then wrapped the "Open Document" requests in a "If...Then" loop. If documentId_1 isn't empty, run the child objects.
My question is if there is a way in SS to use an array for the extractors to use in the loop. Rather than have to creating multiple instances of the same extractor?
Thank you.
|
|
|
21 days ago
Topic:
Correlations are getting screwed up
George @StresStimulusAdministrator Posts: 378
|
Thanks for reporting your experience.
StresStimulus is designed to maintain script integrity while a user can make any required script changes provided that they are correctly made. The behavior described can be caused either by a user error or by a potential software glitch. We have not received similar reports so we will investigate based on your input. There are two possibilities of how user error can cause this behavior:
1. When manual editing the request's recorded body after correlation has been applied. StresStimulus gives a warning that changing the request can break parameters. If you need to make the change, then after that, rerun autocorrelation.
2. When manual modifying the autocorrelation parameters stored in the .ssconfig file. Such modification can be made using script editor
Check if you can replicate the issue. To do so:
- Rerun the autocorrelation
- Make sure that the parameters are not shifted.
- Perform your script changes one more time considering the above recommendations.
- If you can still replicate the shifting parameters, please describe your exact steps, and we will investigate.
- Cheers
|
|
|
21 days ago
Topic:
Correlations are getting screwed up
prashanth.bvbPosts: 8
|
Lets say I have 5-10 auto correlations in a request body. if the restore any one to recorded value, the remaining correlations are pointing to wrong places. There is no way to revert it. I will have to delete all correlations and recreate them.
|
|
|
23 days ago
Topic:
Load test results not getting saved due to error
George @StresStimulusAdministrator Posts: 378
|
Can you please check what version of StresStimulus you are using and are all the load agents are on the same version as the controller? If one of the agents uses an older StresStimulus version that misses a primary key field added in the newer version, it may cause a duplicate error.
Also, can you please describe the following information:
- Cheers
|
|
|
23 days ago
Topic:
Load test results not getting saved due to error
prashanth.bvbPosts: 8
|
I am running a load test execution for 13 scenario using 6 agents. The load test execution is going fine. But while saving the results, i am getting the following error. Due to this none of the results are getting saved apart from Agent details.
Below are the trace logs
01:31:27.057 - ---Begin saving results---
01:31:27.758 - Saving Summary
01:31:28.555 - Saving Sessions
01:31:32.664 - Saving Transactions
01:31:34.305 - Saving Agents
01:31:34.977 - Saving VUs
01:31:35.617 - System.Exception: Duplicate key in StresStimulus.Results.Tables.VUResultTableItem ---> System.Data.Linq.DuplicateKeyException: Cannot add an entity with a key that is already in use.
at System.Data.Linq.ChangeProcessor.TrackUntrackedObjects(MetaType type, Object item, Dictionary`2 visited)
at System.Data.Linq.ChangeProcessor.TrackUntrackedObjects()
at System.Data.Linq.ChangeProcessor.SubmitChanges(ConflictMode failureMode)
at System.Data.Linq.DataContext.SubmitChanges(ConflictMode failureMode)
at StresStimulus.Results.TestResultSaver.trySubmitChanges()
--- End of inner exception stack trace ---
01:31:36.133 - Saving Errors
01:31:36.758 - Saving Test Cases
01:31:36.961 - System.Exception: Duplicate key in StresStimulus.Results.Tables.VUResultTableItem ---> System.Data.Linq.DuplicateKeyException: Cannot add an entity with a key that is already in use.
at System.Data.Linq.ChangeProcessor.TrackUntrackedObjects(MetaType type, Object item, Dictionary`2 visited)
at System.Data.Linq.ChangeProcessor.TrackUntrackedObjects()
at System.Data.Linq.ChangeProcessor.SubmitChanges(ConflictMode failureMode)
at System.Data.Linq.DataContext.SubmitChanges(ConflictMode failureMode)
at StresStimulus.Results.TestResultSaver.trySubmitChanges()
--- End of inner exception stack trace ---
01:31:37.039 - System.Exception: Duplicate key in StresStimulus.Results.Tables.VUResultTableItem ---> System.Data.Linq.DuplicateKeyException: Cannot add an entity with a key that is already in use.
at System.Data.Linq.ChangeProcessor.TrackUntrackedObjects(MetaType type, Object item, Dictionary`2 visited)
at System.Data.Linq.ChangeProcessor.TrackUntrackedObjects()
at System.Data.Linq.ChangeProcessor.SubmitChanges(ConflictMode failureMode)
at System.Data.Linq.DataContext.SubmitChanges(ConflictMode failureMode)
at StresStimulus.Results.TestResultSaver.trySubmitChanges()
--- End of inner exception stack trace ---
01:31:37.039 - ---End saving results---
01:31:37.711 - System.Data.Linq.DuplicateKeyException: Cannot add an entity with a key that is already in use.
at System.Data.Linq.ChangeProcessor.TrackUntrackedObjects(MetaType type, Object item, Dictionary`2 visited)
at System.Data.Linq.ChangeProcessor.TrackUntrackedObjects()
at System.Data.Linq.ChangeProcessor.SubmitChanges(ConflictMode failureMode)
at System.Data.Linq.DataContext.SubmitChanges(ConflictMode failureMode)
at StresStimulus.Runtime.PanelHelper.SaveGraphCurves(IGraphInfo[] graphs)
03:11:00.105 - End of test
|
|
|
28 days ago
Topic:
Unable to open an existing script
George @StresStimulusAdministrator Posts: 378
|
This issue may be caused by data, program, or system corruption. Your post does not describe the effort done to troubleshoot or resolve it. For example, did you try any of the following:
1. Reinstall StresStimulus.
2. Reboot your computer.
3. Restore from a backup the last good version of the script.
4. Record a new script to see if you can re-open it.
5. Try to open your script on a different machine
6. Try to open a script created by your colleague on a different machine by copying it to your machine
These steps may resolve the issue or point you in the right direction. Otherwise, contact your support rep as they may need to inspect your system.
- Cheers
|
|
|
28 days ago
Topic:
Unable to open an existing script
HarshaThadaniPosts: 1
|
Hi team,
I have a recorded & enhanced script, that consists of 3 test cases, ready for execution.
For some reason, I had to restart the stressStimulus application but after the restart, I have not been able to open the script again.
I get a Loading message (see screenshot below) but nothing beyond that.
Have tried to open different scripts, even on different StressStimulus Versions (5.3 / 5.4 - beta) but no luck in opening any script.
Have been facing this issue since yesterday (04/02/2021) afternoon.
Please can someone help me?
Thank you,
Harsha Thadani
|
|
|
1/20/2021
Topic:
Load test execution getting failed with error
George @StresStimulusAdministrator Posts: 378
|
This error message appears when StresStimulus is having issues communicating with an SQL Server database during the test.
If you use SQL Server as your test repository, make sure the StresStimulus computer has a good connection to the SQL Server throught the test. You can also try to address this issue by increasing the SQL Command timeout from 30 seconds to maybe 120 as described here
Alternatively, you can switch back to the default test repository, local SQL CE, that would not have the network connection issues.
As a side note, it is generally not recommended to run SQL Server on the same machine as StresStimulus because both of these products are resource-hungry and compete for the same system resources.
- Cheers
|
|
|
1/20/2021
Topic:
Replacing a test case
George @StresStimulusAdministrator Posts: 378
|
If you already have an updated version of a test case that you can do the following:
- If your test is an SS. config file, then delete the old test case and import the updated test case.
- If your test is a solution (.ssln file) referencing the old version of the test case in folder A, then delete all files from A, copy all files of the updated version of the test case in A, and reopen the solution.
If you do not have an updated version of the test case, then record it first, as described here.
- Cheers
|
|
|
1/19/2021
Topic:
Replacing a test case
alexdvPosts: 27
|
Let's say in Managing Test Cases I have many test cases. How can I replace one perticular test case with updated version of same test case with same name without messing up the order of the test cases?
|
|
|