jArchi - import/export CSV data

Started by Xavier Mayeur, February 04, 2022, 07:39:22 AM

Previous topic - Next topic

Xavier Mayeur

Hello,

Do we have a mean using jArchi to import or export model's CSV data? or is the ACLI the only option?

Xavier

Jean-Baptiste Sarrodie

Hi,

Quote from: Xavier Mayeur on February 04, 2022, 07:39:22 AMDo we have a mean using jArchi to import or export model's CSV data? or is the ACLI the only option?

There's no option to run the built-in CSV export/import, but you can easily generate CSV by just printing it to the console or saving it to a file. Importing can be done using JS libs such as papaparse.

Regards,

JB
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Xavier Mayeur

Thanks for the suggestion.

I also think about it, but there is a big difference:
with a CSV import jArchi script, I am not able to set the concept ID, while the ACLI import can do it.

In interoperability situations, I need to ensure creating concepts with defined IDs

Xavier

Jean-Baptiste Sarrodie

Hi,

Quotewith a CSV import jArchi script, I am not able to set the concept ID, while the ACLI import can do it.

That's on purpose: you should never (ever!) set the internal ID. This can lead to all sort of issues (and believe me, I've had all of them in the past). TBH, I even think about changing the way CSV import work so that it never use the provided ID when creating new object, but use it only when updating.

QuoteIn interoperability situations, I need to ensure creating concepts with defined IDs

The only real way of managing external references is through properties. I've shared a script for this some time ago: https://gist.github.com/jbsarrodie/77e1e6779ace738a15c0bdcf551d68ec (I've now an updated version that I should cleanup and share, but this first version works very well too).

Regards,

JB
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

smileham


Alberto

QuoteYou should never (ever!) set the internal ID. This can lead to all sort of issues (and believe me, I've had all of them in the past). TBH, I even think about changing the way CSV import work so that it never use the provided ID when creating new object, but use it only when updating.

JB,  As someone that relies on the CSV Import for data integrations into Archi, and have been using predetermined IDs for both elements and relationships without issues (observing certain rules of course), this is a big red flag that tells me I'm doing something wrong and that it could change any minute. Could you elaborate as to why I should "never (ever!)" (and you are rarely this assertive) set internal IDs?  I haven't used the CSV import via jArchi but something tells me I should transition sooner rather than latter.

Jean-Baptiste Sarrodie

Hi,

Quote from: Alberto on March 22, 2022, 19:59:33 PMCould you elaborate as to why I should "never (ever!)" (and you are rarely this assertive) set internal IDs?

Yes of course. ThatÅ› a generic principle that applies (IMHO) to Archi but also any other software...

object's id is a needed thing inside any software as soon as we need to uniquely identify and refer to a specific object. In software, ids are everywhere and we should distinguish physical (or internal) ids from logical ones.

Logical ids are meant to be used across systems and usually follow some "business" rules which are known and managed.

But internal ids are a different beast: they are local to one system only and follow mostly constraint-based rules related to the software managing the object. This means that (except for the development team in charge of the software), the way these ids are generated, used and potentially updated, are not documented and subject to changes over time.

In Archi (non exhaustive list):
- Ids are assumed to respect XML constraints (for example they always start with a letter). Don't follow this rule and you won't be able to export to a valid Open Exchange Format file.
- Ids are assumed to be URL-safe. Don't follow this rule and your HTML export won't work.
- Ids are assumed to be SQL-safe. Don't follow this rule and the DB embedded into the HTML report won't work.
- Ids are assumed to be FilePath-safe. Don't follow this rule and coArchi might fail to commit you changes because each object gets exported to a single small file which name is based on the id.
- Ids are assumed to represent the same concept across models. Don't follow this rule and you might have surprises when importing a model into another model: you might then update an element by mistake, or try to update a element with an element from a different type. Archi's behavior in these cases is not predictable and it might clearly fail or seem to succeed but end up with some no-more valid relationships, and then Archi will refuse to reopen the model.

In all these cases, be sure to have recent backup of your models, because you won't be able to fix your models easily as there's no API to change the id.

For all these reasons (and remember that this list is not exhaustive) I highly recommend to never (ever!) force internal ids. Of course, for simple use-cases this might work, but still risky. For example, Archi used UUID in the past, but this changed some time ago. Did you (or anyone else) noticed it and updated your "non supported scripts" to make sure ids always start with a letter?

That's why jArchi exists: with its rich API it becomes possible to do things that otherwise would be really hard or impossible to do. And with a bit of work, some scripts can be made generic enough and shared with the community. I shared such script to import a CSV file and synchronize its content with Archi without using the internal id but instead through a logical id that you can store or compute (using a property being the easiest way). This makes it possible to have a single ArchiMate element synchronized with multiple sources, each with their own logical id.

Hope this helps ;-)

Regards,

JB
If you value and use Archi, please consider making a donation!
Ask your ArchiMate related questions to the ArchiMate Community's Discussion Board.

Xavier Mayeur

Thanks for this comprehensive explanation. I fully agree and also stopped managing ID's.
I tried to do so, because the exchange of Archimate models between ARIS & Archi does not work as expected. If I export a model from Archi to Open Exchange Format and import it twice in ARIS, all objects are duplicated, meaning that the assumption " Ids are assumed to represent the same concept across models." is indeed an assumption and not a constraint for interops between systems...

Have a good day!