Add Bokeh interface for the DQM of NectaRTA

[JulianHamo]

Summary

This adds the retrieval of RTA files for NectarCAM and their display via a Bokeh webpage for data quality monitoring.

Motivation

Real-time analysis will be processed on on-site servers. This interface is to be run locally so that shifters can access the received data in real time. It is a similar but independent instance of the existing DQM of nectarchain.

Functionalities

• Display the metadata of the run.
• Display various visualisations of the latest event analysed by the RTA pipeline (e.g. camera mapping, histogram, timeline).
• Ability to select the number of bins and the number of events to average for histograms.
• Draw a Hillas ellipse over the camera mapping in real time.
• Ability to switch between real-time mode and reading mode. Select the run to display.

Testing

The tests/test.py directory starts the interface using example data (collected in 2025 at CEA).

Current use

The current README explains how to use the interface.

Next steps

• Add DL2 and DL3 data when computed by the RTA.
• Organise the displayed panels for readability.
• Add an event navigator for the events in the same file in reading mode.
• Convert the starting process into an executable to run (with CLI for specific case: debug mode, test mode, etc.).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 45.98%. Comparing base (57c1b15) to head (7205d6a).
⚠️ Report is 20 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #272      +/-   ##
==========================================
- Coverage   51.43%   45.98%   -5.46%     
==========================================
  Files          84       89       +5     
  Lines        7160     8538    +1378     
==========================================
+ Hits         3683     3926     +243     
- Misses       3477     4612    +1135     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jlenain]

Hi @JulianHamo !

Thank you very much for this pull request ! The test app works beautifully !

I did not go through the whole code yet, but from a quick look, it is very well written. I wish I was as proficient with Bokeh as you !

I already left a couple of comments which may require some discussion, I guess.

[jlenain]

As a general rule, we should avoid having data files in the git repo (see e.g. https://www.reddit.com/r/mlops/comments/13s716f/give_me_one_good_reason_why_i_shouldnt_keep_my/).

[JulianHamo]

I am not sure I can create fake data as my test tries the whole interface at once. This means that I check that my whole pipeline is working, from reading .h5 files to displaying all the data available in DL files.

How should I proceed (maybe when such RTA data will be available on the GRID, I can download some test file from here each time the tests are ran)?

[jlenain]

Hi @JulianHamo,

Thank you very much for this pull request! And sorry it took so much time on my side for the review.

Concerning the embedded example file, shall we try to move them to the dedicated test server at CC-IN2P3 ? In such a case, is the code easily adaptable to make use of the ctapipe.utils.get_dataset_path method ?

[JulianHamo]

Hi @jlenain,
I was waiting to get good runs from CEA (by the end of the month) to switch the ones currently available here to ones I can be sure of the content (because we had a issue with the output type of the camera). Once that is done, I'll add the ctapipe.utils.get_dataset_path method to finalize the module if that's fine by you.

[jlenain]

Hi @jlenain, I was waiting to get good runs from CEA (by the end of the month) to switch the ones currently available here to ones I can be sure of the content (because we had a issue with the output type of the camera). Once that is done, I'll add the ctapipe.utils.get_dataset_path method to finalize the module if that's fine by you.

Hi @JulianHamo,

Absolutely no problem!