Archi 4.0.3 Beta 1 - faster load times!

Started by Phil Beauvoir, July 20, 2017, 15:42:32 PM

Previous topic - Next topic

Phil Beauvoir

Hi,

some of you may have noticed that large Archi models can be quite slow to load. I had one example model that took around 45 seconds to load. I may have found the cure. With a simple change, this test model now loads in under 1 second.  :o

However, there may be side-effects that I am not aware of. If you'd like to test the beta version please download it here:

https://www.archimatetool.com/dev/beta

Please only test on copies of models, or have backups. There may be side-effects!

Phil
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

xbrossard

Hi,

I hope that next Archi version include your improvment; actually my 6,5 Mo model took about 3 minutes to load at startup...
when I watch source code, I've see that the problem is apparently when creating views, when searching the corresponding model in the tree model object. Is that the optimization you have done?

Phil Beauvoir

Best thing is to try it and see. If you still have long load times then try closing some Views.
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Hervé

Hi Phil,

I just came back from vacation and hadn't time do do a lot of tests, but on one of my test archimate file (16MB size) that took 25 minutes to load with previous version, the new 4.0.3 beta 1 takes only few seconds.

Very impressive

Well done mate  ;D

Phil Beauvoir

Thanks, Hervé.

It's an interesting story...

I'd always assumed that the slow load speed was due to EMF taking time to load a monolithic XML file and had resigned myself to the idea that this doesn't scale on large models. I was then planning on making deep changes to the persistence format to save and load multiple files and loading them on demand.

Then when testing the new repository plugin, using JB's "grafico" format we noticed that a large model loads very quickly. The grafico format uses multiple XML files for each object. I started to think...

So I Googled "EMF slow load" and found this:

https://www.eclipse.org/forums/index.php/t/796300/

Ding!

So I added:

resource.getDefaultLoadOptions().put(XMLResource.OPTION_DEFER_IDREF_RESOLUTION, Boolean.TRUE);
resource.setIntrinsicIDToEObjectMap(new HashMap());

and made some adjustments elsewhere and bingo!

Now, the thing is, I want to know that if storing the IDs in a HashMap has any side effects. So far, I don't think it has because the Resource is discarded once the model has loaded.

Phil


If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Hervé

I must admit that I do not know the Eclipse framework so I do not understand the impact of your two lines but I use several hashmaps in my database plugin without any side effect so far.

So I do not expect to discover side effects with your hashmap ... but I will do some more testing ...

Best regards
Hervé

Phil Beauvoir

Well, it's not that it's a Hashmap, it's that IDs are now being stored there. The danger is when changing an ID and not updating the Hashmap.
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

ejess

tested with following mode l: 7,5Mo ; 241 View(s) ; 3600 Element(s) ; 6473 Relationship(s)

from 17 minutes to 13 seconds

Bravo!

I'll continue testing and look at side effects.

Phil Beauvoir

Thanks for testing. 13 seconds is still quite a long time.... ;-)
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Hervé

Hi Phil,

Just to give you more precise time on my side:
     load duration : 30 seconds
     model size : 16.5 MB
     # elements : 9.375
     # relationships : 11.509
     # views : 266
     # images : 25

Best regards
Hervé

Phil Beauvoir

Thanks Hervé,

that's quite a large model, so I don't think I can improve the load time on that one.

Phil
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Hervé

Hi Phil,

Going from more than 25 minutes to 30 seconds is very impressive indeed. From my point of view, improving more is not really necessary  ;D

Best regards
Hervé