GL-11: Adding provenance/attribution from ESM tools

[LuiggiTenorioK]

In GitLab by @kinow on Oct 9, 2024, 14:46

Closes BSC-ES/autosubmit#2505

cc @mandresm, @Hussam-Turjman. The code is not working yet, but just added a simple test, and the provenance.py from ESM tools. Tomorrow we can discuss briefly just where/how the provenance is hooked into the existing ruamel parsers/loaders (if we have time for that). Feel free to take over this MR and add/edit commits 👍

[mandresm]

Hi @LuiggiTenorioK, I'm happy to help with this. When you say tomorrow meeting, do you mean the 23rd of October? I could have time for a meeting indeed. What time would it work for you?

[kinow]

Hi @LuiggiTenorioK, I'm happy to help with this. When you say tomorrow meeting, do you mean the 23rd of October? I could have time for a meeting indeed. What time would it work for you?

Hi Miguel!

Sorry the confusion. We migrated the code from GitLab to GitHub, and the person running the import process becomes the OP of every issue and pull request.

What happened here is that @LuiggiTenorioK ran the tool, and it looks like he's commenting, but the comment above was mine, from the week of the hackathon 🙂 We can discuss about it later, but for now we are just moving things around. Sorry for the spam!

One repo migrated, over next days will migrate GUI and API, and in December when it's quieter we will handle Autosubmit... then it will be time for the great automation work to commence!!! ⚙️ 👨‍🔧

[dbeltrankyl]

Any updates for this one?

[dbeltrankyl]

Any updates for this one?

I just commented because the last update is from 22 October

[mandresm]

Hi @dbeltrankyl, I will have a look at it today or tomorrow. One important thing: we just realized that with this approach we cannot serialize the WithProvenance objects following the standard ways (pickling them for example is not possible). We are working in a solution. We will keep you posted.

[dbeltrankyl]

Thanks Miguel!, there is no need for rush we're in the Autosubmit meeting revising PR, issues etc

[mandresm]

Yes, but is a very nice to have feature. I'm though not super happy about the serialization issue, I would say we should not merge until we have a solution for that, it will create problems in the future.

[mandresm]

I cannot review this ATM and I would recommend that this is merged in a more calmed period. In principle, this should not make any problems, but since we are subclassing objects from the configuration recursive methods operating on them might lead to unpredictable lengthy computations or excessive memory consumption. The serialization issue has been solved, though but I still need to get in contact with the person that solved it.

[kinow]

Sounds good to me, @mandresm . The code was still a WIP, and it needs some further work. It was a draft on GitLab but I didn't mark it as a draft in GitHub (just did so). Thanks!

[kinow]

Migrated to new/old repo: BSC-ES/autosubmit#2523