Rework ml model serialization

[LukasBeiske]

This is a first rework of how we save our ml models. To move away from pickle completely some more changes are still needed, but this fixes #2720 and supersedes #2724.

TODO:

• Tests
• What kind of metadata do we want to save in the output?

TODO to move away from pickle -> Maybe as a follow up PR:

• Save and apply models using ONNX (https://onnx.ai/sklearn-onnx/)
• Properly serialize the SubarrayDescription

[maxnoe]

but this fixes #2720 which is needed for #2731

Why does adding a new combiner depend on the possibility to change the combiner between training and application?

[maxnoe]

This looks like you are implementing your own way of

https://docs.python.org/3/library/pickle.html#object.__getstate__
and
https://docs.python.org/3/library/pickle.html#object.__setstate__

[LukasBeiske]

I started this with the intention of completely removing pickle. I can also convert this to draft until I can finish this.

[LukasBeiske]

but this fixes #2720 which is needed for #2731

Why does adding a new combiner depend on the possibility to change the combiner between training and application?

It does not really, my bad, but the changes from #2724 are included in #2731. I adjusted the comment above.

[ctao-dpps-sonarqube]

• 22.97% Duplicated Lines (%) on New Code (is greater than 3.00%)
• 79.20% Coverage on New Code (is less than 80.00%)

Analysis Details

1 Issue

• 0 Bugs
• 0 Vulnerabilities
• 1 Code Smell

Coverage and Duplications

• 79.20% Coverage (94.20% Estimated after merge)
• 22.97% Duplicated Code (0.80% Estimated after merge)

Project ID: cta-observatory_ctapipe_AY52EYhuvuGcMFidNyUs

View in SonarQube