Out Of Memory when streaming to pdf

Zuke · June 27, 2011, 10:53am

Hi All

I am using the following code to print/stream a report to pdf on disk.

  	var file = plugins.file.createTempFile('JobCards','.pdf')
  	var pdf = (forms.fabman_rpt_budgeting_jobcard.controller.print(false,false,plugins.pdf_output.getPDFPrinter(file.getAbsolutePath())));

Also I have a calculation which is grabbing an image for each record from the app server and putting it into each page of the report using the following

	if (image_file_path!=null)
	{
	var imgURL = application.getServerURL()+'/uploads'+image_file_path.split(' ').join('%20')
	var imgTemp = plugins.http.getMediaData(imgURL);
	return imgTemp;
	}

I get the following in my console when running from smart client on a test app server

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError:
Java heap space
       at java.util.Arrays.copyOf(Unknown Source)
       at java.io.ByteArrayOutputStream.write(Unknown Source)
       at java.util.zip.DeflaterOutputStream.deflate(Unknown Source)
       at java.util.zip.DeflaterOutputStream.write(Unknown Source)
       at java.io.FilterOutputStream.write(Unknown Source)
       at com.lowagie.text.pdf.PdfStream.flateCompress(Unknown Source)
       at com.lowagie.text.pdf.PdfImage.<init>(Unknown Source)
       at com.lowagie.text.pdf.PdfWriter.addDirectImageSimple(Unknown Source)
       at com.lowagie.text.pdf.PdfWriter.addDirectImageSimple(Unknown Source)
       at com.lowagie.text.pdf.PdfContentByte.addImage(Unknown Source)
       at com.lowagie.text.pdf.PdfContentByte.addImage(Unknown Source)
       at com.lowagie.text.pdf.PdfGraphics2D.drawImage(Unknown Source)
       at com.lowagie.text.pdf.PdfGraphics2D.drawImage(Unknown Source)
       at com.lowagie.text.pdf.PdfGraphics2D.drawImage(Unknown Source)
       at com.lowagie.text.pdf.PdfGraphics2D.drawImage(Unknown Source)
       at javax.swing.ImageIcon.paintIcon(Unknown Source)
       at com.servoy.j2db.util.gui.MyImageIcon.paintIcon(MyImageIcon.java:317)
       at javax.swing.plaf.basic.BasicLabelUI.paint(Unknown Source)
       at javax.swing.plaf.ComponentUI.update(Unknown Source)
       at javax.swing.JComponent.paintComponent(Unknown Source)
       at com.servoy.j2db.smart.dataui.AbstractScriptLabel.paintComponent(AbstractScriptLabel.java:160)
       at javax.swing.JComponent.printComponent(Unknown Source)
       at javax.swing.JComponent.paint(Unknown Source)
       at com.servoy.j2db.smart.dataui.AbstractScriptLabel.paint(AbstractScriptLabel.java:240)
       at javax.swing.JComponent.print(Unknown Source)
       at com.servoy.j2db.smart.dataui.AbstractScriptLabel.print(AbstractScriptLabel.java:215)
       at javax.swing.JComponent.printAll(Unknown Source)
       at javax.swing.JComponent.paintChildren(Unknown Source)
       at javax.swing.JComponent.printChildren(Unknown Source)
       at javax.swing.JComponent.paint(Unknown Source)
       at javax.swing.JViewport.paint(Unknown Source)
       at javax.swing.JComponent.print(Unknown Source)

I was under the impression that when using a pdf printer with a file name, that the report is streamed to disk, however I am getting out of memory errors when trying to print more than a handful of records.
The images are no bigger than 1 mb each.

Am I doing something wrong?

Any help would be much appreciated.

Kahuna · June 27, 2011, 12:33pm

Any support on this one would be welcome as it’s causing some concern with Clients getting huge PDF’s and out of memory messages.

jcompagner · June 27, 2011, 3:50pm

up the clients memory is the only quick solution i can think of.

Zuke · June 27, 2011, 4:14pm

So are the images to blame? are they not streamed to disk?
I thought this was a feature of the plugin?

sbutler · June 28, 2011, 2:24am

Zuke:
So are the images to blame? are they not streamed to disk?
I thought this was a feature of the plugin?

Do you have large resolution images in your PDF? if so, that is to blame. What I do is to keep 2 copies of the image, one high res for storage, and use a table event to shink the size of the images with the image plugin. Then use the smaller images on the printed versions. No need to embed an image larger than the size your printing.

ptalbot · June 28, 2011, 4:17am

Yes, size does matter But more so resolution. And it’s not strictly true that you can use low definition images to print, actually, quite the contrary, because of printers’ resolutions.

The problem with PDF is that the iText library (which is the one main library used in most Java projects to manipulate low level PDF objects), is laying out images entirely in memory, and even if you are using a compressed format, like JPEG, this format will ultimately be rasterized as a pixel image.
Now, imagine a 400x300 pixels image, which might be super-compressed on disk, it’s still going to be 120.000 pixels. And your image will likely be RGB, so that’s times 3: 360.000 bytes (480.000 if you have an image with alpha transparency).

The problem is that if you print this image, there is a fair chance that it will be aliased, because most printers use a much higher resolution than the 72 or 96 dpi of your screen.
The least you can use is 300 dpi, so again about 4 times the size of your 72 dpi image: that’s already 1.440.000 = 1406Kb = 1.37Mb (and 300 dpi is fairly low nowadays, so I wouldn’t call that large resolution).

Add a few images like that in your document, and knowing that iText is laying out the entire pdf document in one go (because of a dumb calculation for table of contents amongst other things), and you will understand that this adds up pretty fast…
I also believe that iText is not that clever and is holding more than one instance when it is creating the images, in any case, there’s no dump to disk until the whole PDF is layed out entirely in memory.

End of the line: either use a dedicated server (chuck full of RAM) to handle PDF creation, then stream them back to the client (you can think of using web services for that, or plain servlets), or forget about big PDF creation client-side, and build html instead, if your layout doesn’t have to be pixel precise, that’s one avenue to explore…

Harjo · June 28, 2011, 6:22am

You can also use jasperreport plugin for that, there is a function now, that allows you to use the disc, as buffer, and you won’t run out of of memory

Kahuna · June 28, 2011, 7:19am

ptalbot:
End of the line: either use a dedicated server (chuck full of RAM) to handle PDF creation, then stream them back to the client (you can think of using web services for that, or plain servlets), or forget about big PDF creation client-side, and build html instead, if your layout doesn’t have to be pixel precise, that’s one avenue to explore…

Good feedback thanks Patrick. The work we are doing here involves building a ‘Work Pack’ that includes a number of separate documents, only one of which has a number of images. These are all created as separate PDF’s then added together to form 1 PDF.

If we use HTML only for the report with the images can we still produce that as a PDF for inclusion in the main one? Or is it the creation of the PDF itself (even based on HTML) that is causing the challenge?

Kahuna · June 28, 2011, 7:23am

Harjo:
You can also use jasperreport plugin for that, there is a function now, that allows you to use the disc, as buffer, and you won’t run out of of memory

Thanks Harjo - we currently don’t use Jasper - but are you suggesting we can use that to render the PDF’s or will that only work with actual Jasper reports.

Harjo · June 28, 2011, 7:58am

yes, you can render a jasperreport to PDF also…

Zuke · June 28, 2011, 8:04am

Do you have any examples of the usage of the plugin for this purpose.

Thanks

Harjo · June 28, 2011, 8:15am

for the parameters to the report, you need a parameter Hashmap,

there you feed, these settings:

var vParameters = new java.util.HashMap(); //object
vParameters.put('VIRTUALIZER_TYPE', "gZip"); //possible options: file, swapFile, gZip
vParameters.put("PAGE_OUT_DIR", plugins.it2be_tools.server().getApplicationServerDir() + '/server/work/');

and the rest of your parameters of course…
Hope this helps

Zuke · June 28, 2011, 8:20am

Hi Harjo

Maybe I missed the point.

The question really was this

‘Can we use Jasper reports to render a servoy form/report to pdf?’

Or do we have to completely rebuild our reports in Jasper.
As im not to experienced with Jasper, I didnt really understand your last post.

Regards

Harjo · June 28, 2011, 8:24am

ah, sorry, no you can’t use jasper to render a Servoy form, you need to build that report/form again, with iReport

Zuke · June 28, 2011, 10:00am

Im just trying out other possibilities here

I am now producing my report one page at a time as a temp pdf files on disk to be combined later.

So thats 1 page per pdf containing 1 image written to disk, then move on to the next record, im using loadRecords with a single pk each time.

However I am still running out of memory.

I cant figure out what is being cached behind the scenes, surly after each report is written to a temp file, the memory gets released?

Could Johan please comment.

Zuke · June 28, 2011, 12:52pm

Johan could you comment?

jcompagner · June 28, 2011, 2:24pm

hard to see what happens then, need to have a test case so that i can look into the memory what is being kept. There are many variables.

Or you could generate a out of memory dump somehow that i could look into.

Riccardino · June 28, 2011, 6:25pm

Harjo:
for the parameters to the report, you need a parameter Hashmap,

there you feed, these settings:
var vParameters = new java.util.HashMap(); //object

vParameters.put(‘VIRTUALIZER_TYPE’, “gZip”); //possible options: file, swapFile, gZip
vParameters.put(“PAGE_OUT_DIR”, plugins.it2be_tools.server().getApplicationServerDir() + ‘/server/work/’);




and the rest of your parameters of course....
Hope this helps

Great tip, Harjo: thanks

A couple of questions.
why are you passing a directory if you’re using gZip Virtualizer, that compress pages in memory? Is this param needed anyway?
I noticed that, using a file swap virtualizer, the tmp folder is never emptied: do you use a procedure to do that?

Harjo · June 28, 2011, 6:58pm

no, you need that folder, that’s why, you dont run out of memory!

You can find info about this on the servoyForge website

Topic		Replies	Views
Jasper report and java.lang.outOfMemory error Classic Servoy	4	3404	May 23, 2008
Out of memory error in smart client Classic Servoy	12	8197	January 31, 2012
PDF Output - Linux Classic Servoy	2	2197	November 1, 2011
Getting a PDF out of the Headless Client Classic Servoy	5	6134	September 28, 2009
Scheduler Memory Leak?? Classic Servoy	0	1896	September 17, 2012

Out Of Memory when streaming to pdf

Related topics