Neo4j

Started by Phil Beauvoir, August 26, 2014, 07:51:51 AM

Previous topic - Next topic

Phil Beauvoir

Quote from: adeze on November 26, 2014, 02:59:43 AM
by leveraging the archimate exchange format (https://www2.opengroup.org/ogsys/catalog/S142), and the code from the plug in, a lot of the work conceptually has been done-- all thats needed is a mechanism to CRUD the nodes in neo4j.
i've been contemplating the analysis of models for quite a while, and whilst the queries could be very clever, you could also consider syncing with a CMS like structr.org as the analysis tool/viewer, rather than a static report.

I look forward to the day when the ArchiMate exchange format is validated and widely used.
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Morat

#16
Quote from: Jean-Baptiste Sarrodie on September 09, 2014, 09:37:49 AM

Can someone explain me the real (not marketing) advantage of using Neo4J ?


I registered to do just that. I've been using Archi extensively to document an existing, complex legacy applications architecture involving tens of systems and their associated interfaces, business processes and data objects. I pulled the Archi model into Neo4j and also translated it into GEXF, an open XML-based format for graph storage.

The advantages of having the model available in Neo4j were huge:


  • Once you have it in Neo4j you can use cypher (Neo's query language) to analyse it, which helps with impact analysis, consistency checking and examining subsets of the architecture.
  • You can visualise it using Gephi and other similar tools, which can really help illustrate complexity of the model without the overhead of having to manually produce the diagrams. Graphviz is also very handy for this.
  • If you use properties, for example to show which parts of the architecture exist at which points in time, you can examine slices of the model through time. Especially useful for relationships, which aren't first-party citizens in Archimate and can't be contained in plateaus.
  • You can link other external resources to the elements of the model.
  • You can easily access the model programatically through the many adapter libraries available (Java, Ruby, Python, R, etc...).

It also opens up the model to being stored as a centralised, shareable resource in a way that isn't currently safe with the file-based storage we have.

Native neo4j storage would be incredible.

Phil Beauvoir

Hi Morat,

thanks for sharing this information, it sounds very interesting. I guess what we really need is an architecture where models are stored internall in EMF (as they are now) and then to have persistence connectors to allow to save to various formats - XML, Neo4J, SQL DB, NOSQL DB, etc...

Phil
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Jean-Baptiste Sarrodie

Hi,

First of all, thank you for helping me to understand neo4j ;-)

Quote
I've been using Archi extensively to document an existing, complex legacy applications architecture involving tens of systems and their associated interfaces, business processes and data objects. I pulled the Archi model into Neo4j and also translated it into GEXF, an open XML-based format for graph storage.

I'm still not sure about neo4j, but I do know Gephi and thus GEXF format. That's a really powerful graph exploration solution. I'm currently working on a new HTML export for Archi, and the java library used for that (StringTemplate) could easily allow to build generic "exporter" and thus a GEXF one. Would you find it usefull or not ?

Quote
The advantages of having the model available in Neo4j were huge:

  • Once you have it in Neo4j you can use cypher (Neo's query language) to analyse it, which helps with impact analysis, consistency checking and examining subsets of the architecture.
  • You can visualise it using Gephi and other similar tools, which can really help illustrate complexity of the model without the overhead of having to manually produce the diagrams. Graphviz is also very handy for this.
  • If you use properties, for example to show which parts of the architecture exist at which points in time, you can examine slices of the model through time. Especially useful for relationships, which aren't first-party citizens in Archimate and can't be contained in plateaus.
  • You can link other external resources to the elements of the model.
  • You can easily access the model programatically through the many adapter libraries available (Java, Ruby, Python, R, etc...).

From that list I'd say that most of the points are related to Gephi, the one that interrest me the most is the use of cypher (Neo's query language). Could you elaborate and provide some example of what you can do with it in EA context ?

Quote
It also opens up the model to being stored as a centralised, shareable resource in a way that isn't currently safe with the file-based storage we have.

I don't see flat files as less safe than central, binary based solution. I'd add that export/import to a central neo4j instance doesn't mean that it become 'automagically" shareable. These are 2 separate things, several Archi users have developed central DB repository, but each time without the multi/concurrent users feature.

Quote
Native neo4j storage would be incredible.

I like such enthusiasm ;-)

Regards

JB
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Morat

Quotethe one that interrest me the most is the use of cypher (Neo's query language). Could you elaborate and provide some example of what you can do with it in EA context

It's very useful for impact analysis and determining relationships due to the ability to follow chains of relationships in a way that's more challenging in a traditional relational database. I did some analysis to determine which business processes were ultimately reliant on which data objects and through which application functions with this. It's also good for finding interesting features in your model, e.g. 'show me the data objects that are written by more than one application function' or 'which application functions do not yet have certain kinds of relationships modelled'.

I got most value out of it when the model is combined with other more detailed information, for example a mapping of interface data usage and system to logical data models. We were able to determine from querying the model that one of the systems has internal movement of data between tables due to gaps in the expected results when we looked for the linkage between an upstream system mastering data and the downstream system that was using it.

This work was done in a solutions architect context, but the ability to query your model in this way and make inferences based on the relationships is relevant whether you're working in the SA or EA space, especially as the architecture model becomes complex. It's like having the visualiser window on steroids.

Stef Joosten

This topic should be linked with multi-user Archi.

Collaborative editing of Archi models requires a persistent store with transactional facilities.
To develop that on MySQL (or MariaDB) takes more effort than doing that on a graph
database (like Neo4j). Even with the learning included, neo4j will be preferrable. The better performance is a nice perq, but not essential.

That I would applaud building Archi on top of neo4j, just to get a collaborative Archi.
(By the way, a MySQL-based Archi repo could just as well fulfill the needs for collaborative use of Archi. But why use worse technology over better?)