Archive
IReport / Jasper Reports – Working with subreports and collections in Jasper Reports
Hi, and sorry for not updating for a while. I am currently under heavy load, and can scarcely find time to write, although i have several new topics prepared. I am also working on a presentation on Java Performance Testing and Monitoring which i will probably hold here in Hamburg, on the 18th of July. More on that for those interested in a follow-up post.
Now let’s dive into the subject: Working with subreports and collections in Jasper Reports
It seems that several users have been facing this problem, so i thought i wrote an explanatory post on this topic.
1. Building the main report
Let’s start with the main report. This looks like this:
The sub-report is the grey box with a yellowish highlighting. The main report is passing the four following parameters to the sub-report, of which one is a collection:
- http_request
- filterstop
- ic_testconfig
- filterstart
Please notice how the name matches EXACTLY the expression (the passed parameter has to be named exactly the same as the local variable used in the subreport)
Also notice the properties of the sub-report, highlighted in the screenshot below:
- Subreport Expression has to be: “repo:statistics”, where statistics is the name of the IReport file containing the designed subreport
- Expression Class: java.lang.String
- Connection type: Use a connection expression
2. Creating the sub-report
Let’s create a sub-report in the repository, with the name we just configured in the main report: statistics
My recommendation is to create a single folder in your repository, called “subreports”, and add all sub-reports there. In my case, i have three sub-reports (we will only focus on the statistics subreport):
- statistics
- detailed_statistics
- hudson_statistics
The structure in the repository looks like this:
Let’s take a look at my sub-report. This will receive the collection as input parameters. The collection is actually a series of test id’s that i use to build a report over several test runs (for example, if i run a test twice, once with id 123, and once with id 124, and i want to see a single report of all transactions for both test-runs, i will give both parameters as input:
The query is the one that takes the collection input parameter and processes it. Let’s see how that looks like:
select
count(t) as totaltransactions,
avg(t) as responseaverage,
……
from
testresults tr
where $X{IN,tr.testrun_id,ic_testconfig} and DATE_FORMAT(DATE_ADD(’1970-01-01 00:00:00′ ,INTERVAL ts*1000 MICROSECOND),’%H:%i:%s’) between $P{filterstart} and $P{filterstop}
group by tr.lb
We will now add this sub-report into the Jasper Server Repository, by adding a new resource from the JRXML File we just created for our subreport. We will have to assign two identifiers:
- Label: statistics
- Name: rootavg
Jasper Server Repository – Adding a JRXML Resource
Now we add the two identifiers mentioned above
Jasper Server Repository – Labeling the JRXML Resource
We can now refresh the repository in the IReport local instance, and see the sub-report added in the location we chose, with the identifiers we just assigned (name is statistics, id is rootavg):
Subreport in IReport Repository
3. Adding the sub-report as a resource to the main report
We have now added both the main report and the sub-report. Well, it is not enough to define the sub-report in the main report. The main report has to know where the called sub-report resides, therefore we need to add it as a resource of the main report. Remember that these resources have to be defined and available in the JasperServer Repository (Server Side).
We start by editing the main report on the server side, and adding the resources:
Add subreport as resource in jasper server repository
We still have to add the parameters on the server side. Remember we want to use a collection. In order to do that, we use a “Multi Select Query Type” Input Control:
Jasper Server Add Input control
We configure it as a “Multi Select Query Type” with the same name we are going to use in our report, that being “ic_testconfig”:
Jasper Server Collection Input Control
After refreshing the Repository in IReport, our Report looks like this. Notice the input controls, and how the collection item is now available
Configured repository with subreports
4. Final
Let’s review the steps once again:
- Create the main report, and decide on the parameters you want to pass to the sub-report. Upload the main report to the Jasper Server, and add the parameters on the server side too. The resources always have to bee synchronized
- Create the sub-report, and the query that will receive the collection. Upload the sub-report to Jasper Server
- Add the sub-report as a resource of the main report in Jasper Server
- Watch out the query syntax when using collections:
- where $X{IN,tr.testrun_id,ic_testconfig}
I think that’s it. Tried to put it as explanatory as possible, in the short amount of time that i have at disposal these days. I am really sorry for the delay in replying to comments, and posting new content. Hope to get things off my head in the near future
Cheers,
Alex
10 + Points why to separate Backoffice from Frontend
I came upon the idea of writing this while i was checking on the side effects of some Business Cases of our Backoffice, running in the same JVM with our Front-End. The first hint i got (and this is where CI – continous integration shows its benefits) was after integrating my Jmeter Load Tests in Hudson, our CI System. By doing this, i radically changed the behaviour of the system. The data was getting filled to the database right after deployment, and not manually (therefore using a deterministic behaviour)
After having had one of the automatic tests failed, i turned first to the response times. I was suprised to see the radical increase of response times from one build to the other. Expecting a response times under 500 ms, i was getting 20 seconds back:
JMeter Hudson - Response Times
I needed to look deeper into the problem, so the first thing (the one that i always do) was to launch VisualVM and check on the status of JVM, which looked something like this:
Visual VM - Full Heap - Full GC
And this is how the memory footprint looked like:
Visual VM - Full Heap - Memory Footprint
There it was. My JVM was getting filled, therefore going into Full Garbage Collections (hence the CPU spikes in the image above). Doing a simple “jmap -histo pid | grep pattern” i could see a lot of objects that belonged to our Backoffice.
Doing a heap dump, i discovered that the parent of all this object was a scheduled timer, that was going through a list of entries in the database, doing some operations at the end, but transferring all of them first local (so filling the heap with objects needed for calculation and other operations)
It was not the first time i had seen this ( i had seen it before, but after manually starting a Backoffice operation, which was pretty much working in the same way: going to the database, and transferring all objects local), so i decided to take a deeper look, and decide if having a Backoffice and Frontend together in the same JVM / Machine is a good idea after all. After doing some brainstorming for a couple of hours, i came up with about 10 points why it is not a good idea.
10 + Points why to separate Backoffice from Frontend
- Queries – Long running queries need resources. In order to go to database, you need a connection. In order to get a connection you need a thread. You will get both of them from a pool (http thread pool and jdbc connection pool) While one or two connections may not sound critic, if you have a support department employing more than a couple of Backoffice users, you will “steal” expensive resources which may be well used by the Frontend at peak times.
- CPU - If you are doing local calculations on sets of data retrieved from the database, you need CPU, specially if you are working with highly complex mathematical functions, or (and) with large sets of data. Again, doing this a couple of times ( once for each Backoffice user) may well steal CPU power, that may be used at peak (but also normal) times by your front-end. Take this very serious, we are talking JVM here. We need CPU for finding objects, working with them, cleaning them up. You do not want to give your CPU away to your Backoffice while your front-end has to eventually wait
- Result sets – Sometimes you do computations either on the database side (taking off the burden of computing on the application side) or local (taking the burden off the database) According to expected result sets, and your business needs of course, one may use one or the other method. Yet, chosing without predicting can be fatal. Transfering large quantities of objects from the database to the application server for doing the calculation, can easily fill up the Heap, specially if you are using JPA in a domain driven design application. Not only that the shared heap will be filled with objects that the front-end does not need, but you may well find yourself in a FULL GC situation, just because you do not have enough space anymore to accomodate objects from your result set. Full GC’s will immediately translate into huge response times on the user side (see images above). Are you willing to risk that?
On the other side, Garbage Collection will last a lot longer, depending on the object tree. Although you work well with your front-end, not having leaks, you have to risk response times because of unneeded Garbage Collection - Optimum debugging of memory problems – Have you ever tried dumping a filled heap of 8 GB size? Have you tried loading it into a Heap Analyzer??? If you did, than you know by now that this is a no go. And even if you’d work with a smaller heap, detecting memory leaks is much easier once you get fast to the root references. You will definitely achieve this by running your Backoffice in a separate JVM. It will be much easier to separate objects belonging to one application or the other (cause there will be only one)
- Memory leaks do not cause operation disruption
Any memory leak will translate – sooner or later – into Full Garbage Collections and eventually OOM (much feared out of memory) Any business case that you have not tested, or that you are not aware of can cause something like this. Before going to the roots of the problem, and solving it by modifying the code, you will need to restart your application server, hence restart your front-end server. Are you willing to be “not available”?
Running the Backoffice into a separate JVM will allow you to:- Restart Backoffice JVM
- Online redeploy of newer, fixed versions of software
- JDBC statement caching
As most of you probably know, JDBC allows configuring the number of statements to be cached / jdbc connection. As i am aware of, the algorithm chosen for managing the cache is LRU. Now, running the Backoffice and front-end in the same JVM, may well dismiss critical cached statements of a connection, just because your Backoffice business case needs a lot of statements to be run. Although it is not an effect that you see in the day to day operation, you will notice it in a OLTP system. Guaranteed! - SQL Debugging
Running the Backoffice in the same JVM, over the same JDBC connection will make isolating Backoffice queries almost impossible to acchieve. I am referring to the Oracle trace files, or views (v$sql, v$sqlarea, etc.), which will show you a bunch of statements running over the same connection, same user, etc. Having the possibility of filtering after hostname or even better, username, will save you a lot of time. You can do that either by using a second jdbc connection pool, configured with another user, or by running the Backoffice in a separate JVM, separate application server - Custom tuning for busines needs
Configuring your Backoffice into another JVM will allow you to fine tune things like:- heap size
- garbage collection strategy
- jdbc settings (validation,timeout,statement caching)
- thread pool
- etc.
The fine tuning for Backoffice will almost never meet the needs of front-end
- Sampling, profiling
No need to point this out. Having a single application to profile (or sample) makes things much easier, and faster to detect and fix. - High availability
I was pointed about this one by Niels, one of my collegues. If your production environment is configured using the “staging” principle, where shutting down means actually moving over, you will then need to double the requirements on Heap, both for your front end and back end. Now, imagine you have a 16 GB server, with a configured heap size of 8GB for both Frontend and Backoffice. In order to create the stage system, and to have them both running, 16 GB will therefore not be enough, not anymore ( you need to double the resources). Moving the Backoffice to another server, and another JVM will allow you to “stage” gracefully.
I will get back to this list whenever i feel like adding another point. I would be happy to see your comments.
Cheers,
Alex
Latest comments