Need help with StresStimulus? Start here.
Detailed Iteration data for time series plots ?
|
Andrew McDonald Posts: 1
8/9/2012
|
Hi,
I currently evaluating StresStimulus.
I have the 30 day trial of the Pro Edition v1.7 (limited to 3 VUs)
http://www.stresstimulus.com/Editions.aspx
It seems like the Pro edition will fulfil most of our requirements but there is one requirement that seems problematic.
We need to run several long running tests (moderate load) but to monitor how server performance varies through-out the test. This will be done on globally available _intranet_ sites.
Ideally we wish to graph a time series of page load times i.e if we had a hundred iterations we would have a hundred data points at fixed time intervals for each page being tested. The intent is to determine how performance varies with time.
Initially I set a fixed number of iterations and a fixed think time between iterations to achieve this.
However it seems as if obtaining the detailed data from the tool to plot a time series is not possible without some work ?
I am trying to determine if this is possible using the tool and, if so, the simplest way of doing it. I apologise in advance if I have misunderstood or overlooked something obvious.
I examined the various reports and grids and also read the following blog posts:
Validating StresStimulus data - Part 2: Page Settings - StresStimulus Fiddler Load Testing Place
http://www.stresstimulus.com/blog/post/validating-stresstimulus-data-1
Validating StresStimulus data – Part 4: Request Details - StresStimulus Fiddler Load Testing Place
http://www.stresstimulus.com/blog/post/validating-stresstimulus-data-part-4-request-details
I then examined the following options:
- The Page Settings under Test Case has a Duration field but of course is only intended to characterise the timing of the (recorded) test case not of the results.
- The Page Details contains test results and statistics for page load times but it only has aggregate statistics across all iterations, not the individual statistics from each iteration.
- The Request Details is similar to the Page Details and only has aggregates. Also of course this is less suitable as we are interested in a page level analysis.
- The [show sessions] feature on either page could be used to show sessions from all iterations. The includes a column, Iter-URL, that could be used to determine the start/end of iterations (pages) (or I could even just use the fact that each iteration has fixed number of sessions), however, by default, the sessions list does _not_ show any of the timer columns. I can supply a customRules.js to display the necessary columns and then filter based on Iter-Url (or fixed count) and then cut and paste from sesssion list into Excel and complete the calculations and plot the time series. However I am a little concerned about overloading Fiddler memory-wise at it will be required to (temporarily) load and display a great many sessions (100,000+) prior to filtering and the cut and paste. Additionally I was hoping minimsie any coding/scripting required to acheive the goal. I also am not sure whether customising the session list possible when using the [show sessions] feature in StresStimulus ?
- I examined the results database. Most tables contain data that corresponds directly to the screens and graphs and thus only have aggregate statistics. However the ResultData table seems to contain individual lines for all requests and iterations which seemed promising. The table includes explicit TTFB and TTLB columns however these are not sufficient to calculate page load duration. There is a field called Timers, binary(112), which I assume contains some serialised form of the Fiddler session timers. If so then this timer data would certainly be sufficient to generate the report, however I do not know to decode the Timers field.
To sum up, it seems I may be able to produce the time series we want by customising the session list and cutting pasting. I have not yet confirmed this.
If I could decode the Timers field in the database I could certainly produce the report I want quite simply. One advantage of this method is that it would then be simple to produce almost any desired report that was not already available directly from the UI, and this data could even be loaded into standard reporting tools.
Could you advise as to whether StresStimulus can be used to produce the desired time series report and if so the simplest course for doing so ?
Regards,
Andrew
|
|
|
0
link
|
|
Vadim @StresStimulus Administrator Posts: 583
8/9/2012
|
Hi Andrew,
There is no build-in report for data series at this time, while I see how it can be helpful for performance monitoring tasks.
You are correct that ResultData table contains un-aggregate raw data for every session. However, I see a challenge for an external reporting tool to aggregate request information into page load duration, and here is why.
On every page, some requests are issued sequentially; some others are issued in parallel. Generally, page response time is the interval between the moments of issuing the first request and receiving the last response. In reality, StresStimulus uses a more complex algorithm. For example, it will not count some Ajax responses received after the page is presumed rendered in the browser. StresStimulus does all calculations in runtime using internal object model that is not available in the database. For the external reporting tool, it will be very difficult, if possible, to create SQL procedures encompassing the adequate logic of determining for each iteration page response time.
One workaround I can think of is instead of running 1 test with 100 iterations, you can schedule at desired intervals 100 tests with 1 iteration each. As a result, each test result in the page detailed grid will have the data you need for this iteration. At this point you would need to pull data from the same table across 100 SQL CE files or 100 databases in an external SQL Server. To schedule multiple tests, use the command line as described in item #3 of this post http://www.stresstimulus.com/blog/post/what-s-new-in-v2-0-part-3-pro-other-enhancements
Alternatively, we can probably develop the page-iteration-details table or report with the information you need. If you are interested, send me a PM with your timetable and other requirements, so we can discuss it.
Regards,
-Vadim
|
|
|
0
link
|
Copyright © 2024 Stimulus Technology