CPU Performance comparison 4.1.16 vs 4.2+ for a DestinE multimember stress test experiment

[dbeltrankyl]

Hello @kinow , @VindeeR

We already did some tests for the Coding sprint regarding 4.2 postgres vs sqlite and vs 4.1.16. #2627

This issue is only for CPU, memory and size was checked already and it was fine.

Opening this issue to do a comparison once the 4.2 branch is fully finished and new a PR ( to link when it is open and needed) for the optimization part.

I was testing this for @orioltinto due the Operational issue where the experiment is taking 6min on submit a new job.

Tests based on
https://gitlab.earth.bsc.es/digital-twins/de_340-2/workflow/-/issues/1323#note_406835

Since it is a stress multimember experiment created by @franra9

---

TODO list:

(set safetysleeptime to 0 for the tests)

• 4.2 branch stress experiment test with 2 splits ( with the simple time.time or the profiler)
• 4.2 branch stress experiment test with 10 splits ( with the simple time.time or the profiler)
• 4.2 branch stress experiment test with auto splits. (original) ( with the simple time.time or the profiler)

Other things we can check:

• Optimize DB index (jobs and structure tables)
• Optimize DB fields ( too many strings)
• Check if our submission process is efficient for all platforms (ps, slurm..)
• Check if our job status pooling is efficient for all platforms ( ps, slurm..)

---

Results:

4.1.16:

10 splits 136807

2025-11-06 13:53:40,350 Iteration time: 57.527984380722046 seconds
2025-11-06 13:54:32,379 Iteration time: 51.84905290603638 seconds
2025-11-06 13:55:20,726 Iteration time: 48.169872999191284 seconds
2025-11-06 13:56:13,518 Iteration time: 52.617791175842285 seconds
2025-11-06 13:56:58,579 Iteration time: 44.86899137496948 seconds

2 splits 33127 jobs

2025-11-06 12:32:51,131 Iteration time: 6.9863433837890625 seconds
2025-11-06 12:33:03,522 Iteration time: 12.348278045654297 seconds
2025-11-06 12:33:18,438 Iteration time: 14.872258186340332 seconds
2025-11-06 12:33:29,457 Iteration time: 10.975615739822388 seconds
2025-11-06 12:33:44,755 Iteration time: 15.253679752349854 seconds
2025-11-06 12:33:51,342 Iteration time: 6.543314456939697 seconds
2025-11-06 12:33:57,825 Iteration time: 6.440295934677124 seconds
2025-11-06 12:34:04,223 Iteration time: 6.362769365310669 seconds
2025-11-06 12:34:10,752 Iteration time: 6.493175268173218 seconds
2025-11-06 12:34:17,682 Iteration time: 6.895272970199585 seconds
2025-11-06 12:34:24,235 Iteration time: 6.4666314125061035 seconds

4.2 - (TODO)

4.2 (optimized if needed) - (TODO)

---

Note: I did some cpu tests. during the coding sprint, with an operational experiment and it looked better than 4.1.16. This is to confirm it

[dbeltrankyl]

related to #2627 , #2619

[kinow]

Good idea these tests!

One other thing that I would check, not related to wrappers or AS per se, but the schemas in Postgres. I am starting to worry if we get more than 20... 50 schemas... I think in DestinE we probably have passed 100's already? It'd be nice to check the performance of Postgres with 1000 schemas.... I think it might affect performance as it needs to manage each separately, and some queries/actions in the server work across schemas, I think (probably backup, consistency, vacuum cleaning, security...). Even if we perform well, I guess there's the risk that in 4.2 having too many schemas could impact our overall performance. Maybe for a separate issue though.