Archive
Interpreting Performance Results – Part 1
I was recently asked by a blog visitor what i do with my jmeter performance results, and how i fill those bottlenecks. I think one of my last experiences is the best answer, and is also a good starting point in discussing the subject.
So, the facts are: we have the performance results, the nice looking graphs telling us lots of values. How do you tell there’s a performance bottleneck based on these results, and, more important, how do you fill the bottleneck?
I will use one of the tests i have last performed in order to answer this question. This testplan works with 30 Virtual Users, logging in, joining a particular session, and doing some things afterwards (irelevant for the purpose of this discussion)
Let’s see the test results for this relatively small load :
Jmeter Load Results in Jasper Report
Like i said, after loggin in, the users join a specific session. The business case behind the Join Session request, requires several DB transactions (mainly selects, collecting history information about the user joining the session)
The red part displays the following information: the 50% line, 60% line, 70% line, 80% line, 90% line, where the last column (the grey one) displays the average number of business transactions / second.
I will repeat the definition of the 90 % line, so it is clear for everyone reading this post. The 90 % line is the maximum response time that 90 % of all business transactions receive. In other words, 90 % of the users receive a response time of 10,3 seconds, or better.
Well, 10 seconds for a request of this kind shows a problem with the DB transactions. Therefore, the next step was enabling the eclipse persistence logging, and doing a detailed analysis of all SQL querys run against the database in one “join session” business transaction. Once i filtered this information, i grouped the querys, and run them one by one against the database, also enabling the explain plan (oracle db)
This is how i found three querys running on unindexed large tables, consuming a lot of CPU Time on the DB server. Enabling indexes on those tables dramatically improved the response times for the “join session” transaction:
Interpreting Jmeter Results in Jasper Reports
As you can see, the response time improved from 10 seconds to less than 100 milliseconds.
Well, this is just one example of how i use JMeter results to detect and solve performance problems. More complicated stuff will come. I hope that by this i have answered Nirali’s question.
Cheers and have a good day
Alex
Load and Performance Reporting with JMeter, JasperServer and Pentaho BI – Part 1
Building a Load and Performance Reporting solution using JMeter, MySQL, Pentaho Data Integration, JasperReports and IReport, JasperServer and Shell Scripting
The subject i think, has been going on through the testing community for a while, and i am sure everybody comes with a custom solution to the problem. My solution a couple of years ago, when i last performed a load testing project, was composed of :
OpenSTA for load testing
Microsoft Visual Basic for Applications (VBA) for data import
Microsoft Excel for Pivot Tables and Graphics
I can still remember spending about 3 days on building my VBA scripts, in order to have a pseudo-automated solution of generating testreports.
Well, here i am, 2009, doing load and performance testing for a new project. This time Excel was out of the question. Why ? – someone still using it would ask…Cause it’s reports are so hard to manage, history reports do not exist, and if they do, they require a lot of maintanance, it is time consuming, and …it is so Windows platform based 
(no portability whatsoever)
Now, in a time where ETL (Extract Transform Load) is on everybody’s lips, i decided to go the hard way. That’s what they say, isn’t it? The hard way is always the best way.
Therefore, in the article to follow, i will describe the solution i implemented using SHELL scripts, JMeter, MySQL , Pentaho Data Integration, IReport, JasperReports, JasperServer and Hudson for a fully automated load and performance test and reporting. Sound a little bit complicated, and so many tools involved, but trust me, it is worth the time!
Here we go. Let me explain a little how i imagined this would work, and how i implemented it.
Part 1 – JMeter and Shell Scripting – WHY ?
Well, the first thing i asked myself, was …how can i build those scripts so that i can group them in “performance scenarios”. It is a question where automated tools offer an answer, by providing the test scenario functionality. But i did not have any test scenarios…i only had JMeter test plans. That was my starting point. And i needed scenarios:
- I thought that the best way to control JMeter scripts is using command line interface. (1)
- Simply starting a JMeter testplan from the command line was surely not enough, therefore i wanted it to be able to run JMeter testplans with arguments.(2)
- I also wanted to be able of grouping multiple JMeter Testplans into a single Testscenario, so i could control the workflow (3)
- Only starting one JMeter test after the other was not exactly what i had in mind. I needed to control what happens in the beginning, in between, and when the tests ended (4)
So here i was, having 4 “requirements” that were all pointing in one and single direction: SHELL SCRIPTING
Part 2 – Collection of JMeter test results – Where ?
I definitely needed a central point to store the jmeter results, in order to be able to create reports from them. But creating dynamic reports requires a database of course. So i turned to my old friend, MySQL.
Basically, i have created about 5 tables, to store the JMeter samples, JMAP samples (heap maps), but also information about the tests i have run. I wanted a full report, not a partial one.
Basically i neede one table for storing the version of the application, one table for storing information about the test scripts that i have run, one table for saving the relations between script and version, one table for storing the actual JMeter results, and last, but not least (it was the last one added, that’s true ), one table for storing the JMAP Heap statistics (number of objects in heap, and size of the objects in heap)
Five tables alltogether. That is all you need to get started.The design of the tables, specially the one for the jmeter samples, depends very much on what information you want to collect and filter.
I will get back to the subject, by detailing the structure in the post that will be related to this.
Part 3 – ETL – Extract, Transform, Load
There were several opportunities here, like creating some stored procedures that could be called from the shell, but i wanted something more… user friendly, easier to maintain, easier to design, and easier to integrate. I needed an ETL tool. And Pentaho had the solution to my problem.
Pentaho has gone truly far in this direction. They offer Business Intelligence Products of amazing quality, and i do take the time to congratulate them on their products. They offer java based tools for designing transformations, grouping them into jobs, executing them via GUI or command line, and so on.
I decided to give Pentaho Data Integration a try, and although it took me more time than expected to get it alltogether, it was worth every minute.
So using Spoon, which is the GUI for designing transformations from Pentaho Data Integration, i managed to create a Data Transformation, extracting and loading test information and test results in my database, and even more, automating it via shell scripts.
I think on a scale of complexity, this was on the highest point…, only a few points down of integrating JasperServer But, as i said, it was worth every minute
Part 4 – Creating and running the reports
Now, i had the data in a database. The querys were quite simple, so all i had to do now was to find a smart tool to generate the reports. And in a DYNAMIC WAY. Any desktop based application was out of the question, i needed this reports to be called at any time, by anyone, from everywhere… i needed WEB BASED REPORTS
So i turned to Jasper Reports and Jasper Server. And of course, IReport for designing the reports. All these, part of Jasper Server Package ( no need for the proffesional version, i used the free one)
That is all i needed to get me started. And the best part of it ? It is free, it is customizable!
To summarize, once again, this is what you need:
MySQL DB, JMeter, Shell Scripting, Jasper Server, IReport, Pentaho Data Integration
My next post will start detailing the first steps, the database structure, and importing the data using Pentaho Data Integration. I will speak about the transformation itself: how to create it, what u have to take care of, and provide a sample, for both the sql structure, and for the Pentaho Transformation
Cheers,
Alex
Latest comments