Archi 4 : java.lang.OutOfMemoryError: Java heap space

Started by Hervé, October 03, 2016, 15:12:18 PM

Previous topic - Next topic

Hervé

Dear all,

I experience an issue when I load images in Archi 4. Here is the stack trace :

java.lang.Exception: java.lang.OutOfMemoryError: Java heap space
at org.eclipse.swt.internal.image.PNGFileFormat.setPixelData(PNGFileFormat.java:239)
at org.eclipse.swt.internal.image.PNGFileFormat.setImageDataValues(PNGFileFormat.java:301)
at org.eclipse.swt.internal.image.PNGFileFormat.readNonInterlacedImage(PNGFileFormat.java:491)
at org.eclipse.swt.internal.image.PNGFileFormat.readPixelData(PNGFileFormat.java:320)
at org.eclipse.swt.internal.image.PNGFileFormat.readNextChunk(PNGFileFormat.java:136)
at org.eclipse.swt.internal.image.PNGFileFormat.loadFromByteStream(PNGFileFormat.java:77)
at org.eclipse.swt.internal.image.FileFormat.loadFromStream(FileFormat.java:55)
at org.eclipse.swt.internal.image.FileFormat.load(FileFormat.java:86)
at org.eclipse.swt.graphics.ImageLoader.load(ImageLoader.java:146)
at org.eclipse.swt.graphics.ImageDataLoader.load(ImageDataLoader.java:22)
at org.eclipse.swt.graphics.ImageData.<init>(ImageData.java:332)
at org.eclipse.swt.graphics.Image.<init>(Image.java:604)
at com.archimatetool.editor.model.impl.ArchiveManager.testImageBytesValid(ArchiveManager.java:138)
at com.archimatetool.editor.model.impl.ArchiveManager.addByteContentEntry(ArchiveManager.java:244)
at org.archicontribs.database.DBImporter.importImages(DBImporter.java:1373)
at org.archicontribs.database.DBImporter.doImport(DBImporter.java:419)
at org.archicontribs.database.DBImporter.doImport(DBImporter.java:139)
at com.archimatetool.editor.actions.ImportModelAction.run(ImportModelAction.java:40)
at org.eclipse.jface.action.Action.runWithEvent(Action.java:473)
at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:565)
at org.eclipse.jface.action.ActionContributionItem.lambda$4(ActionContributionItem.java:397)
at org.eclipse.jface.action.ActionContributionItem$$Lambda$27/13054157.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4410)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1079)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4228)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3816)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$4.run(PartRenderingEngine.java:1121)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1022)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:150)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:687)


Here is my code :

IArchiveManager archiveMgr = model.getAdapter(IArchiveManager.class);
String imagePath;

PreparedStatement pstmt = db.prepareStatement("SELECT path, image FROM images", ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
ResultSet result = pstmt.executeQuery();
while ( result.next() ) {
imagePath = archiveMgr.addByteContentEntry(path, result.getBytes("image"));
if ( result.getString("path").equals(imagePath) ) {
debug("... image imported");
} else {
debug("... image imported but with new path "+imagePath);
}
}
result.close();


The image if a 4 MB png which loads without any issue in Archi 3. What I tried so far but without any success :
   - raise the JVM memory (-DXmx=1200m)
   - save the result.getBytes("image") into a file to verify that the image is correct (and I can open the image file without any issue)
   - analyse the source of PNGFileFormat.java : the exception is raised on a 8 MB array allocation :
            byte[] rgbData = new byte[destBytesPerLine * height];
   - dump the JVM memory when the exception is raised : the dump file is 400 MB (which is far away from the 1200 MB configured with the Xmx parameter), and run Eclipse Memory Analyzer against the dump file. It didn't found any memory leak ...
   - I even tried to foce the garbage collector to run before the image loading :
            system.gc()

I'm using a 32 bit JDK as my work is using 32bit OSes.

Now on, Im stuck and do not know where to search anymore  :-[ Did one of you ever experienced such an issue and could give me a hand ?

Thanks and regards
Hervé

Phil Beauvoir

Hi,

the image handling code was written 5 years ago so I'd need to refresh my memory on what it does. I do remember, though, testing it extensively with really big image files and it seemed to be OK.

It seems to fail at the testImageBytesValid() method.

I made some changes to ArchiveManager earlier this year. Need to check if something broke there.

Phil
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Hervé

Hi Phil,

Thanks for your quick answer.

I do not beleive it comes from your code as your testImageBytesValid() method just creates an image and releases it right after. If no exception is raised, then the image is correct.

It didn't change between Archi 3 and Archi 4.

I posted this message just in case someone already experienced such an error message because I really do not understand what happens here.

Best regards
Hervé

Phil Beauvoir

If you add the following line to the VM Arguments in the Eclipse Launch Configuration dialog it will show a heap size widget in the top toolbar in Archi:

-Dshowheap=true

You can monitor heap allocation with this.

I tried loading some huge images in Archi 4 code on Mac and it seemed OK.

Phil
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Hervé

After some investigation ...

Your testImageBytesValid() method creates an Image to validate that the byte[] contains a vadid image.

Unfortunately, the Image object is preparing the image for the Display. With Eclipse 4.4, the SWT librairy did not do any scaling, so the Image contains a 1:1 bitmap and therefore is not too big.

But Eclise 4.6 introduced a scale factor for hi dpi displays ... which is my case. I invested into a 4k display 2 months ago. So now on, the same Image, is automatically scaled to a 2x factor, consuming 4x more memory (2xwidth x 2xheight) !!!

When I disable the scaling factor, using -Dswt.autoScale=false, I can load the images but the Archi window is so small that I can barely read it.

What I suggest is to update your testImageBytesValid() method, replacing :

    private void testImageBytesValid(byte[] bytes) throws IOException {
        try {
            Image img = new Image(Display.getCurrent(), new ByteArrayInputStream(bytes));
            img.dispose();
        }
        catch(Throwable ex) {
            throw new IOException("Not a supported image file", ex); //$NON-NLS-1$
        }
    }

by

    private void testImageBytesValid(byte[] bytes) throws IOException {
        try {
            new ImageData(new ByteArrayInputStream(bytes));
        }
        catch(Throwable ex) {
            throw new IOException("Not a supported image file", ex); //$NON-NLS-1$
        }
    }


The ImageData object does not prepare the image for being displayed, so no graphical object is allocated and no scaling factor is applied. It just converts the compressed image (png, jpg, ...) to a bitmap and raises an Exception if it fails.

Best regards
Hervé

Phil Beauvoir

Good find, Hervé.

One of those things that show up under a certain set of circumstances....  :-[

Thanks for the suggestion for the fix. I have committed this to GitHub.

It may be worth thinking whether big images on hi-res screens will cause memory problems anyway in Archi...

Regards,

Phil
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Hervé

maybe or maybe not ...

less and less people are on 32 bit systems. On my side, I need to stick to a 32bit JVM because my company is using 32bit OSes but one day, with a bit of luck and the end of support of 32 bit OSes by editors, I may have a 64 bit desktop.

And this day, I will not care anymore about the SWT scaling factor  :P ;D

Zanda

Quote from: Hervé on October 04, 2016, 23:36:29 PMmaybe or maybe not ...

less and less people are on 32 bit systems. On my side, I need to stick to Testoprime review because my company is using 32bit OSes but one day, with a bit of luck and the end of support of 32 bit OSes by editors, I may have a 64 bit desktop.

And this day, I will not care anymore about the SWT scaling factor  :P ;D

True, in a few years I doubt there will be many people left using 32 bit.

tommy_ng@gwu.edu

I am getting Java heap space error when I try to export to Image as PDF.  It works fine when I export to Image as JPEG.  Any thoughts?

Phil Beauvoir

Quote from: tommy_ng@gwu.edu on April 05, 2017, 18:46:15 PM
I am getting Java heap space error when I try to export to Image as PDF.  It works fine when I export to Image as JPEG.  Any thoughts?

Depends on your system and which version of Archi you use.
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Hervé

Hi,

You may try to increase the Xmx parameter in the archi.ini file.

On my profesional laptop (32bit), I increased to 1200MB. If you've got a 64 bit system, you may even allocate more memory.

Best regards
Hervé

tommy_ng@gwu.edu

I am using version 3.3.2
I trying allocating 1200m in ini file and still get heap space error exporting to PDF image

Phil Beauvoir

Quote from: tommy_ng@gwu.edu on April 06, 2017, 15:51:52 PM
I am using version 3.3.2
I trying allocating 1200m in ini file and still get heap space error exporting to PDF image

Windows, Mac, or Linux? How much memory are you using? Does it do it for all diagrams exported to PDF?
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

tommy_ng@gwu.edu

windows OS.  all images work except for PDF

this is my archi.ini file
-clean
--launcher.defaultAction
openFile
-vmargs
-Xms128m
-Xmx512m
-XX:MaxPermSize=1200m
-Dosgi.requiredJavaVersion=1.7
-Dosgi.instance.area=@user.home/AppData/Roaming/Archi
-Dosgi.configuration.area=@user.home/AppData/Roaming/Archi/.config

Phil Beauvoir

Quote from: tommy_ng@gwu.edu on April 06, 2017, 16:25:09 PM
windows OS.  all images work except for PDF

this is my archi.ini file
-clean
--launcher.defaultAction
openFile
-vmargs
-Xms128m
-Xmx512m
-XX:MaxPermSize=1200m
-Dosgi.requiredJavaVersion=1.7
-Dosgi.instance.area=@user.home/AppData/Roaming/Archi
-Dosgi.configuration.area=@user.home/AppData/Roaming/Archi/.config

But are all images when exporting to PDF causing the error? i.e try a very simple image.

Also, try increasing the following:

-Xmx1024m

And keep

-XX:MaxPermSize=256m

If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.