Author Topic: Jarchi - Performance  (Read 165 times)

rfamans

  • Member
  • **
  • Posts: 17
Jarchi - Performance
« on: January 02, 2019, 19:51:55 PM »
Hi,

I have created several Jarchi scripts that traverse through a medium size Archi model (apx. 2500 elements and 6000 relationships). The structure of my Jarchi code looks more or less like this:

    $('.'+element.name).inRels().filter('flow-relationship').each(function(flow) {
        source = flow.source.name;
        $('#'+flow.id).outRels().filter('association-relationship').each(function(assoc) {
           var intf = $('.'+assoc.target.name).filter('application-interface').first();
           $('.'+intf.name).outRels().filter('access-relationship').each(function(accesses) {
              var pobj = $('.'+accesses.target.name).filter('business-object').first();
           });
        });
     });

It seems that traversing this way through a medium size Archi model is exhausting my workstation's resources in such way that processing a single top level element with 2 child levels below (each sublevel has max 20 elements) can take up to one hour.

From a distance it looks like Jarchi runs into memory management issues because I am re-instantiating (big??) object variables in a nested loop structure.

Does somebody have a clue about what may be the cause of the heavy load that these kind of Jarchi scripts do generate? Is there a workaround that can be used to reduce the heavy system load?

// Roy

Phil Beauvoir

  • Archi Master
  • Hero Member
  • *****
  • Posts: 938
    • Archi
Re: Jarchi - Performance
« Reply #1 on: January 02, 2019, 19:57:15 PM »
At this point it's hard to say if there is an overhead from the Java libraries involved or something else in the script.

I suggest opening an issue here - https://github.com/archimatetool/archi-scripting-plugin/issues - together with an attached example model and full script that demonstrates the problem. Then we can take a look at what's going on.
If you value and use Archi please consider making a donation! https://www.archimatetool.com/donate

Phil Beauvoir

  • Archi Master
  • Hero Member
  • *****
  • Posts: 938
    • Archi
Re: Jarchi - Performance
« Reply #2 on: January 02, 2019, 19:59:21 PM »
Perhaps you could check those nested calls by inserting some log messages.
If you value and use Archi please consider making a donation! https://www.archimatetool.com/donate

Phil Beauvoir

  • Archi Master
  • Hero Member
  • *****
  • Posts: 938
    • Archi
Re: Jarchi - Performance
« Reply #3 on: January 02, 2019, 20:11:20 PM »
A quick comment.

In your code you are not using some variables. Instead of:

$('#'+flow.id).outRels()

You could do:

$(flow).outRels()

And instead of:

$('.'+intf.name).outRels()

You could do:

$(intf).outRels()
If you value and use Archi please consider making a donation! https://www.archimatetool.com/donate

rfamans

  • Member
  • **
  • Posts: 17
Re: Jarchi - Performance
« Reply #4 on: January 02, 2019, 20:50:38 PM »
Thanks for your reply. I will try your suggestion and let you know if it improves the script performance.

Phil Beauvoir

  • Archi Master
  • Hero Member
  • *****
  • Posts: 938
    • Archi
Re: Jarchi - Performance
« Reply #5 on: January 02, 2019, 21:01:30 PM »
It will be slow if you are re-querying the whole model to get elements by their IDs and names. Without checking it, it could be optimised as:

Code: [Select]
$(element).inRels().filter('flow-relationship').each(function(flow) {
    source = flow.source.name;
    $(flow).outRels().filter('association-relationship').each(function(assoc) {
       var intf = $(assoc).filter('application-interface').first();
       $(intf).outRels().filter('access-relationship').each(function(accesses) {
          var pobj = $(accesses).filter('business-object').first();
       });
    });
 });
If you value and use Archi please consider making a donation! https://www.archimatetool.com/donate

rfamans

  • Member
  • **
  • Posts: 17
Re: Jarchi - Performance
« Reply #6 on: January 03, 2019, 08:43:35 AM »
I see a very significant improvement after following your solution. I also avoid re-definition of object variables in for-loops. I  now define objects once outside the loop without re-declaring them (with the var suffix).  This results into stable memory usage of the archi executable (apx. 380mb). When I re-define objects within the loop, memory consumption soon reaches the JVM max allowed memory consumption level. I have read several posts that say that the Nashorn engine can suffer from poor performance due to memory leaks and garbage collection issues when reaching the JVM memory thresholds.

Phil Beauvoir

  • Archi Master
  • Hero Member
  • *****
  • Posts: 938
    • Archi
Re: Jarchi - Performance
« Reply #7 on: January 03, 2019, 09:06:21 AM »
jArchi uses object Collections (the Java ArrayList class) quite extensively. You could try releasing these when they are no longer needed by calling clear() on the collection. Also, rather than chaining collections, declare them separately and release them:

var rels = $(element).inRels();
var flows = rels.filter('flow-relationship');

// Do stuff

rels.clear();
flows.clear();
If you value and use Archi please consider making a donation! https://www.archimatetool.com/donate

rfamans

  • Member
  • **
  • Posts: 17
Re: Jarchi - Performance
« Reply #8 on: January 07, 2019, 21:20:39 PM »
For the record: in addition to your first suggestion, this is the ultimate performing solution:

Code: [Select]
$(element).inRels().filter('flow-relationship').each(function(flow) {
        source = flow.source.name;
        $(flow).outRels().filter('association-relationship').each(function(assoc) {
           var intf = $(assoc.target).filter('application-interface').first();
           $(intf).outRels().filter('access-relationship').each(function(accesses) {
              var pobj = $(accesses.target).filter('business-object').first();
           });
        });
     });

This way, minutes of execution time are reduced to just a few seconds.
« Last Edit: January 07, 2019, 21:24:36 PM by rfamans »

Phil Beauvoir

  • Archi Master
  • Hero Member
  • *****
  • Posts: 938
    • Archi
Re: Jarchi - Performance
« Reply #9 on: January 07, 2019, 21:22:08 PM »
Nice.  8)
If you value and use Archi please consider making a donation! https://www.archimatetool.com/donate

Jean-Baptiste Sarrodie

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 419
  • Archi Evangelist and Contributor. Visionary.
    • About me...
Re: Jarchi - Performance
« Reply #10 on: January 08, 2019, 19:48:25 PM »
Hi,

For the record: in addition to your first suggestion, this is the ultimate performing solution:

In fact you can simplify a bit more (and maybe expect some performance increase too): inRels() and outRels() take a filter as an argument, so instead of calling:
Code: [Select]
inRels().filter('flow-relationship')
you can simply call:
Code: [Select]
inRels('flow-relationship')
Regards,

JB
If you value and use Archi please consider making a donation! https://www.archimatetool.com/donate

rfamans

  • Member
  • **
  • Posts: 17
Re: Jarchi - Performance
« Reply #11 on: January 14, 2019, 10:41:20 AM »
Ok, I will try that as well. In the past I faced some issues with adding selectors directly to inRels but maybe the errors had to the other parts of the statement chain.