Collaboration++: inter-model relations

Started by MarcF88, August 15, 2017, 14:55:39 PM

Previous topic - Next topic

MarcF88

Multi-user Archi is coming along very nicely with the new collaboration plugin. This plugin does a lot of things very well and we are using it in my organisation quite intensively.
To really achieve collaboration, however, we think a next step Archi is required: we would like to propose that inter-model references are made possible. So far, I cannot see anything like that in the https://www.archimatetool.com/dev/roadmap. Please allow me to explain the requirement for inter-model references.

The situation is that we have a substantially large model, in which several separate groups are working on their proper piece of the model. Inevitably there are fringes and overlaps, where merge conflicts arise and seperate elements are defined for the same underlying entity.

What we want to achieve is that each group have their own part of the model, and that they have (read-only, could be enforced by Git so not necessarily in Archi) access to adjoining models so that shared elements can truly be shared. This could IMO be a powerful concept, and - hopefully - be achieved in a fairly straightforward way, following a couple of rules:

If a model contains an element of a different model, this would mean that 1. the element's name and other properties are taken from the originating model; 2. the element  has a 'namespace' or somesuch so that is easy distinguishable and does not conflict; 3. the relations the element has are part of the model they are made in. 

The interesting thing is what happens if element definitions are updated - what can you see in the model that uses the element? In Hervé's database plugin, you can see he faced the same questions and I think the solution he chose is the right one: when you link to an element from a remote model, you get to choose either a link or a copy of the element. Using a link, the element gets updated along with the element in the remote model. Using a copy, it does not get updated (unless you force an explicit refresh - I don't know if that function is available in the database plugin?)

Using these rules it should be possible to achieve productive collaboration between teams each maintaining their own model, and the teams are still able to maintain coherence and build on each other's work.

We have considered using the database plugin along with the collaboration plugin to achieve this functionality. There are a couple of reasons why we hesitate to do this. For one, the inter-model reference cannot be used from the model tree in the user interface (left pane where all your elements are in), you need to open the plugin. In addition, there's quite a setup required, ok if you have a db and need the model in there for other purposes but not so much if it is a means to an end. In addition, (in this moment) it is not possible to tell in the UI if an element is remote or locally defined. Having said this, the database plugin does make clear that it is perfectly possible to achieve what we want.

This is something that should eventually be discussed in the Archi development forum (if it gets that far) but as it has a lot of functional implications so I figured it would better be discussed here. What do you all think, good idea and should be in the roadmap or meh, better not?

Kind regards,
Marc

Phil Beauvoir

Hi Marc,

thanks for raising the issue here on the forum.

The original idea behind Archi was to keeep things simple and portable - thus a single xml file for each "model". It seems that we may have reached the point where Archi has to grow beyond that.

So it seems the main argument is to support a global scope for ArchiMate concepts so that they can be referenced and re-used in more than one model.

Some quick thoughts:

What is a "model"? Is it a container for concepts and views? And if these concepts are referenced in another model, is there a container above that?

And where/how would the concepts be stored? How to make it portable?

Phil

If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Hervé

Hi Marc,

To answer your questions, my database plugin does not create "links". It exports and import the same object from the database. So every updates done on a shared component through one model will be available to the others models (not real-time, of course, because it is just an export/import plugin).

When you import a model from the database, you can choose either a specific version of that model and see the components as they were in that specific version of the model, or alternatively, you can import the latest version of all the components.


This said, my ultimate goal is to transform my plugin to a real-time collaborative solution, i.e. all the updates done on one component will be seen in real-time by all the other users who work on the same component. JB already oriented me to the Eclipse CDO framework, but unfortunately it is far from my development competences. So until I learn those skills, I'm opened to any suggestion you may have to improve my plugin.

Best regards
Hervé

MarcF88

Quote from: Phil Beauvoir on August 15, 2017, 19:27:23 PM
What is a "model"? Is it a container for concepts and views? And if these concepts are referenced in another model, is there a container above that?

And where/how would the concepts be stored? How to make it portable?
Hi Phil,

Very much agree on simple and portable. I think a model would still be what you get when you go File -> New -> Empty model: a collection of elements, their properties and relations, contained within their own namespace and all its views, all in a single happy xml container (I was going to say "file", but wih the recent developments as db plugin and GRAFICO that concept got a bit blurred).

Storage is still basically in the xml container itself. Let me see if I can sketch the situations that occur.

Model Personnel:
- Business:
-- Business Actor: Alex <Documentation: ..., Properties: ...>
-- Business Role: Developer <Documentation: ..., Properties: ...>
...
- Relations:
-- Assignment relation (Alex - Developer)

Model AcmeDevelopment:
- Business:
-- Business Actor: Personnel::Alex <>
-- Business Role: Personnel::Developer <>
-- Business Role: ScrumMaster <Documentation: ..., Properties: ...>
...
- Relations:
-- Assignment Relation Personnel::(Personnel::Alex - Personnel::Developer)
-- Assignment Relation (Personnel::Alex - ScrumMaster)

So this illustrates that AcmeDevelopment contains *links* to Alex and Developer (because their properties are empty - if you want their values you need to check runtime in the source model) and creates one new role ScrumMaster and a new assignment. The originating Personnel knows nothing of this new role or assignment.

The fact that you need to refer to the source model to get at the properties for Alex ("link" semantics) has advantages and drawbacks: models that re-use elements are always up-to-date wrt properties, drawback is that the originating model should be accessible to Archi. That is why it is also a good idea to have "copy" semantics also available, this would copy the relevant properties from the source. You would still recognize a copied element from its namespace.

What happens if you use link semantics and the source model is not available? Check for an NPE and come up empty, nothing much you can do. What happens if Alex is removed from Personnel? Depends, you could offer to either remove the element from the borrowing model or turn it into a "real" element.

Regards,
Marc

MarcF88

Quote from: Hervé on August 15, 2017, 19:58:03 PM
To answer your questions, my database plugin does not create "links". It exports and import the same object from the database. So every updates done on a shared component through one model will be available to the others models (not real-time, of course, because it is just an export/import plugin).

When you import a model from the database, you can choose either a specific version of that model and see the components as they were in that specific version of the model, or alternatively, you can import the latest version of all the components.

Hello Hervé,

Thanks for the clarification. I have not really thought much about versioning, I get the feeling that it has the power to complicate matters immensly but maybe it is just exactly what is needed. It seems though that you do have made the right decision to offer developers either to hang on to a known version, or to keep current.

I am not so sure about being able to update en element in the source model from a borrowing model. Maybe it simply does not fit because of our use case: separate teams who all "own" a piece of the combined model, that is available for re-use in other models. For situations where there is a need to collaborate on common parts of the model, there's always Git (well, at least since a couple of weeks that we have the collaboration plugin).

Regards,
Marc

Phil Beauvoir

(Thinking aloud...)

I tend to think about this in the "classical" format of:


  • A global repository of elements and relations (canonical)
  • Models that reference or link to the concepts in the global repository

And then I wonder about Views. Does a View belong to a container "model", and whether it, too, could be re-used in another "model". And then I think that maybe there should just be a repository of "stuff" that can be re-constituted in various ways whether that is called a "model" or part of a model. And then this leads me on to how this is all stored, and where...
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Hervé

Hi Phil,

That's exactly what I implemented in my database plugin: elements, relationships and views are stored independently of the models.

Of course, they are created into a model, so they are "linked" to this model. But any other model can import them (what I called "import individual components").

When an element, relationship or view is updated in one of the models (and after those updates are exported to the database of course), then these updates are available to the other models.

As all element/relationship or view are individually versioned, the plugin can detect conflicts when two models try to export updates on the same component. Please let me give you an example:

Let's create an empty model A
     - create a device "pc". This is version 1 of the component.
     - export the model to a database.
Let's create another empty model B
     - import the device "pc" from the database (you imported the version 1 of the component)
     - export the model in the database
==> now, we've got both models A and B that share the same device "pc" (let say that they both have got links to the same device "pc" if you prefer  :))

In model A
     - add documentation "This is a pc" to the device "pc"
     - export the model A to the database
            (on the export window, the plugin will tell you that you've got one updated element: it detected that the device "pc" has got updated values in your model compared to the database)
            The plugin registers a version 2 of the device "pc".
In model B
     - At the moment, your device "pc" is still version 1: it does not has any documentation (because unfortunately, my plugin is not real time)
     - export the model B to the database
            (on the export window, the plugin will tell you that you've got one updated element: you've got the version 1 of the device "pc" but the database contains version)
            So the export fails and shows you a table comparing the properties of both versions (yours and the database's one) and you can choose between
                   -> export: the model's B version will be exported to the database, becoming the latest version in the database (i.e. version 3)
                   -> do not export: the model's B version will no be exported to the database (the model B will keep the version 1 without without the documentation). In this case, the next export will raise another conflict on the same element.
                   -> import:  the plugin will import the missing documentation field from the database into your model B

In the same way, when you import a model, you've got the choice to import it as it was (i.e. with the versions of elements/relationships/views as they were at the time of the model export) or to use the latest version of all the components.

Importing elements and relationships is quite straight forward, but when importing views, things are becoming a bit more complex :
   - if some components were removed from a view, the corresponding elements and relationships are not removed from the model (they still are in the model's tree)
   - but those which were added to the view (and were not already part of the model) are automatically imported

Quite very clear in my head, but very difficult to explain with words (especially in English  ;)) so please do not hesitate to drop me a line if I do not make myself clear.

I know I'm very late on the documentation. I even began to write down inline documentation pages but it's quite empty at this stage. I really need to give it a higher priority (but I still have too much to do in this plugin ;D).

Hope this helps.

Best regards
Hervé

MarcF88

Quote from: Phil Beauvoir on August 17, 2017, 16:29:13 PM
(Thinking aloud...)

I tend to think about this in the "classical" format of:


  • A global repository of elements and relations (canonical)
  • Models that reference or link to the concepts in the global repository

Hmm, could this be https://en.wikipedia.org/wiki/Conway%27s_law at work here? In my line of work, there's several groups working in parallel on adjoining models, and there is a (although not always very strictly defined nor enforced) sense of ownership over the subject area of the model. Borrowing is encouraged, joint authorship... not so much.

If I read about a global repository with all the pieces of the puzzle in there, I wonder who places them there, who knows about pieces being placed, and who gets to change them? And what if you have a puzzle of your own, but want to re-use one or two pieces, how would it work if someone else updates them? What kind of organisational structure do you presume?

Regards,
Marc