Enhancement of the muon analysis pipeline.

[burmist-git]

Please describe the use case that requires this feature.
The muon analysis pipeline has recently been updated to incorporate the weighted Teubin fit, with initial parameters now estimated using the Chaudhuri and Kundu method ( Muon fit improvement #2736 ).

To be verified: this new change may fix the ring radius distribution, which previously showed unphysical behavior around 1 deg.

An additional important issue relates to the accuracy and correctness of impact point reconstruction. The intensity fitter uses a log-likelihood fit to simultaneously determine the optical efficiency for muons and the impact point within a single fitting function. The muon impact point is one of the parameters used in the quality cuts.
The unphysical behavior can be seen in the distribution:

There are a number of other issues, each of them will have its own separate sub-issue summarized below:

1. The chord calculation has a bug.
   It does not account for non-physical solutions in the chord calculation for points outside the mirror. This is important for fitting with the central hole in the mirror, or when accounting for camera shadowing. (MR: Muon fit chord length fix Muon fit chord length fix #2803). (Sub-issue: The chord calculation has a bug The chord calculation has a bug #2804)

2. The default value of the LST mirror hole is incorrect, and the 'usual' YAML file used for analysis also contains the wrong value. It is very small. (The sub-issue The reflector’s hole radius is defined incorrectly #2807 and its MR: Muon fit chord length fix hole radius correction m #2808 )

3. The oversampling for the log-likelihood fit is too small. The oversampling needs to be adjusted — we should use fewer bins than the angular dimensions.

4. The atmospheric absorption, mirror reflectivity, and PMT photon detection efficiency must be taken from the Monte Carlo simulation. Within this approach it is ignored.

5. The muon inclination is not included in the fit of the muon impact point or in the PDF calculation.
   The PDF calculation for the expected number of photoelectrons has a weak dependence on the muon inclination,and on the determination of the muon impact point.

Using a poorly reconstructed impact point reduces efficiency and introduces additional systematic uncertainties, which in turn affect optical throughput and PSF measurements

Describe the solution you'd like
A revision of the muon fit sequence is necessary, along with the implementation of relevant validation tests.
Muon inclination should be incorporated into the fit of the impact point and into the PDF.

Describe alternatives you've considered
An alternative solution is to exclude the muon impact point quality cuts and having cuts applied to the fitted ring only. It is important to note that the muon impact point is derived from fitting the azimuthal distribution of Cherenkov photons—an additional step that follows the ring fitting process.

Additional context
The implementation of relevant validation tests need to be added.

Related issue:
#2815
#2844 (comment)

Related MR:
#2814
#2845 (comment)

[burmist-git]

1. The chord calculation has a bug.
   It does not account for non-physical solutions in the chord calculation for points outside the mirror. This is important for fitting with the central hole in the mirror, or when accounting for camera shadowing. (MR: Muon fit chord length fix Muon fit chord length fix #2803 )
2. The default value of the LST mirror hole is incorrect, and the 'usual' YAML file used for analysis also contains the wrong value. It is very small.
3. The oversampling for the log-likelihood fit is too small. The oversampling needs to be adjusted — we should use fewer bins than the angular dimensions.
4. The atmospheric absorption, mirror reflectivity, and PMT photon detection efficiency must be taken from the Monte Carlo simulation. Within this approach it is ignored ...

[burmist-git]

The code to reproduce this error is here:
https://github.com/burmist-git/ctapipe_run_dir/blob/master/muons_chord_D.ipynb

The values from the side are non-physical.

Filtering out non-physical solutions using this inequality equation (equation 7, page 3 from https://doi.org/10.1016/0927-6505(94)90012-4):

Resulting :

[burmist-git]

LST mirror hole is incorrect:
YAML:

https://github.com/cta-observatory/ctapipe/blob/ec9b0fa988103f532368c42f051a7657fd60107d/src/ctapipe/image/muon/intensity_fitter.py#L463C1-L463C38

[burmist-git]

Point 3) about the oversampling :

The code to reproduce this error is here:
https://github.com/burmist-git/ctapipe_run_dir/blob/master/muons_ana_correlate1d.ipynb

[kosack]

LST mirror hole is incorrect

Those are default values, probably set many years ago from a very old simulation production, but it looks like the user can just set the correct ones in their config file, right? Maybe we should remove the defaults altogether so it's clear this is information that should be specified in the configuration?

Perhaps a better option would be to add this hole size to the OpticsDescription, if it is something that can be computed from the simulation info (are the mirror facet positions stored?). That way it's the responsibility of the EventSource to set it properly.

[burmist-git]

Those are default values, probably set many years ago from a very old simulation production, but it looks like the user can just set the correct ones in their config file, right?

It is possible to modify the configuration files and set the default values correctly (set it to 0.75m not 0.308 for LST). It was most likely set to smaller values because the two additional imaginary solutions play an important role in spoiling the fit, describing the camera or hole (this is to be tested with new vs old fitter).

[burmist-git]

Perhaps a better option would be to add this hole size to the OpticsDescription

Definitely better (is it simple :-) ?)

[maxnoe]

See #1252

[burmist-git]

@maxnoe So it wasn't resolved. I think I'll first add it to the YAML file and update the configuration for LST in the default values.
#1252 (comment)