CTAPipe Data Model¶

The DL1 files are HDF5 format files, with the following data set hierarchy. The tables should be written with pytables (not h5py), ideally with the HDF5TableWriter, which ensures the unit and other descriptive metadata are attached to the output. Containers marked with a + should be written without their prefix (all others should use column prefixes).

The following describes the contents of data level 1 (DL1) output files generated by ctapipe (e.g. the ctapipe-process tool which uses the DataWriter component to generate output data).

DL1 Data Model¶

This describes data that change per-event. The following datasets will be written to the group /dl1/event/ in the output file:

Group/Dataset	Description	Contents
`subarray`	event-wise data pertaining to a subarray	(group)
`subarray/trigger`	subarray trigger information	`EventIndexContainer` +, `TriggerContainer`
`telescope/`	Per-telescope Per-event information	(group)
`telescope/parameters/tel_{TEL_ID:03d}`	tables of image parameters (one per telescope)	`TelEventIndexContainer` +, `ImageParametersContainer`
`telescope/images/tel_{TEL_ID:03d}`	tables of telescope images (one per telescope)	`TelEventIndexContainer` +, `DL1CameraContainer`

DL2 Data Model¶

This describes data that change per-event. The following datasets will be written to the group /dl2/event/stereo/<algorithm>/ and or /dl2/event/mono/<algorithm>/, one for each reconstruction algorithm in the output file, where <algorithm> is the identifier of the algorithm (e.g. “Hillas”):

Group/Dataset	Description	Contents
/geometry	shower geometry reconstruction	`EventIndexContainer`, `ReconstructedGeometryContainer`
/energy	shower energy reconstruction	`EventIndexContainer`, `ReconstructedEnergyContainer`
/classification	shower classification parameters	`EventIndexContainer`, `ParticleClassificationContainer`

Simulation Data Model¶

/simulation/event/subarray/shower

true shower parameters from Monte-Carlo simulation

EventIndexContainer +, SimulatedShowerContainer

/simulation/event/telescope/images/tel_{TEL_ID:03d}

simulated camera images

EventIndexContainer +, SimulatedCameraContainer

/simulation/event/telescope/parameters/tel_{TEL_ID:03d}

Parameters derived form the simulated camera images

EventIndexContainer +, ImageParametersContainer

/simulation/service/shower_distribution

simulated shower distribution histograms

SimulatedShowerDistribution

Configuration Data Model¶

The output file should also contain serializations of the instrument, observation (if applicable), simulation (if applicable) configuration information, written to the /configuration group:

Group/Dataset	Description	Contents
`/instrument`	Serialized `ctapipe.instrument.SubarrayDescription`, using `ctapipe.instrument.SubarrayDescription.to_hdf()`	(group)
`/instrument/subarray/layout`	Subarray layout info	result of `SubarrayDescription.to_table()` output as HDF5 using `astropy.table` functionality
`/instrument/telescope/optics`	telescope optics information	result of `SubarrayDescription.to_table(kind='optics')` output as HDF5 using `astropy.table` functionality
`/instrument/telescope/camera/{CAMERA_ID}`	camera geometry information	result of `CameraGeometry.to_table()` output as HDF5 using `astropy.table` functionality
`/simulation`	Monte-Carlo simulation configuration information	(group)
`/simulation/run`	Monte-Carlo simulation run information	`SimulationConfigContainer`

Core Provenance¶

The root group of the file shall contain all of the “CTA Core Provenance Metadata” headers as user attributes, with the hierarchy flattened and separated by spaces (e.g. "CTA ACTIVITY NAME" = "ctapipe-process").