reorganized stats/chunk containers

[TjarkMiener]

We noticed in #2996 that containers dealing with chunks and stats needed some maintenance and docstring polishing.

[kosack]

Looks good, just need a changelog entry

[ctao-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[TjarkMiener]

I was thinking of replacing this duplication by

class ChunkStatisticsContainer(ChunkContainer):
    """Container for descriptive statistics of the chunk distribution"""
    stats = Field(
        default_factory=StatisticsContainer,
        description="Statistical description of the chunk distribution",
    )

which would be nice but would also lead to breaking changes here and elsewhere.
@mexanick @maxnoe @kosack

[maxnoe]

What we do elsewhere is have separate containers for the index and the data and write multiple containers to the same table.

I.e. you could have the ChunkContainer, StatsContainer and a HistogramContainer and write like this:

writer.write(table, (chunk_container, stats_container))

like this, you also don't need separate containers for the interpolated result and the chunk storage.

[maxnoe]

See e.g.:

ctapipe/src/ctapipe/io/datawriter.py

Lines 309 to 312 in 8705606

	self._writer.write(
	table_name="simulation/event/subarray/shower",
	containers=[event.index, event.simulation.shower],
	)

[TjarkMiener]

I think n_events should be moved to the StatisticsContainer and then compute_stats() from the PlainAggregator and SigmaClippingAggregator should return StatisticsContainer and the compute_histo() from HistorgramAggregator from #2996 should return a HistogramContainer

[maxnoe]

mmh, that seems a bit weird. the n_events is a property of the chunk and should be the same for all chunk aggregations, regardless of whether you compute a histogram or stats or something else.

n_events should be really the number of events inside the chunk, which is very interesting in case of time-based chunks.

It should also be independent from which values are actually used due to e.g. outlier detection, sigma clipping or under/overflow.

[TjarkMiener]

right, the number of events are related to the chunks and the number of entries are related to the number of values used in the aggregation.