Note
Go to the end to download the full example code.
Using Container classes#
ctapipe.core.Container is the base class for all event-wise data
classes in ctapipe. It works like a object-relational mapper, in that it
defines a set of Fields along with their metadata (description,
unit, default), which can be later translated automatically into an
output table using a ctapipe.io.TableWriter.
Let’s define a few example containers with some dummy fields in them:
26 class SubContainer(Container):
27 junk = Field(-1, "Some junk")
28 value = Field(0.0, "some value", unit=u.deg)
29
30
31 class TelContainer(Container):
32 # defaults should match the other requirements, e.g. the defaults
33 # should have the correct unit. It most often also makes sense to use
34 # an invalid value marker like nan for floats or -1 for positive integers
35 # as default
36 tel_id = Field(-1, "telescope ID number")
37
38 # For mutable structures like lists, arrays or containers, use a `default_factory` function or class
39 # not an instance to assure each container gets a fresh instance and there is no hidden
40 # shared state between containers.
41 image = Field(default_factory=lambda: np.zeros(10), description="camera pixel data")
42
43
44 class EventContainer(Container):
45 event_id = Field(-1, "event id number")
46
47 tels_with_data = Field(
48 default_factory=list, description="list of telescopes with data"
49 )
50 sub = Field(
51 default_factory=SubContainer, description="stuff"
52 ) # a sub-container in the hierarchy
53
54 # A Map is like a defaultdictionary with a specific container type as default.
55 # This can be used to e.g. store a container per telescope
56 # we use partial here to automatically get a function that creates a map with the correct container type
57 # as default
58 tel = Field(default_factory=partial(Map, TelContainer), description="telescopes")
Basic features#
66 ev = EventContainer()
Check that default values are automatically filled in
73 print(ev.event_id)
74 print(ev.sub)
75 print(ev.tel)
76 print(ev.tel.keys())
77
78 # default dict access will create container:
79 print(ev.tel[1])
-1
{'junk': -1, 'value': 0.0}
Map(__main__.TelContainer, {})
dict_keys([])
{'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]), 'tel_id': -1}
print the dict representation
86 print(ev)
{'event_id': -1,
'sub': {'junk': -1, 'value': 0.0},
'tel': {1: {'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]),
'tel_id': -1}},
'tels_with_data': []}
We also get docstrings “for free”
92 help(EventContainer)
Help on class EventContainer in module __main__:
class EventContainer(ctapipe.core.container.Container)
| EventContainer(prefix=None, **fields)
|
| Attributes
| ----------
| event_id : Field(default=-1)
| event id number
| tels_with_data : Field(default=builtins.list)
| list of telescopes with data
| sub : Field(default=__main__.SubContainer)
| stuff
| tel : Field(default=Map(__main__.TelContainer))
| telescopes
| meta : dict
| dict of attached metadata
| prefix : str
| Prefix attached to column names when saved to a table or file
|
| Method resolution order:
| EventContainer
| ctapipe.core.container.Container
| builtins.object
|
| Data descriptors defined here:
|
| event_id
|
| meta
|
| prefix
|
| sub
|
| tel
|
| tels_with_data
|
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| default_prefix = 'event'
|
| fields = {'event_id': Field(default=-1), 'sub': Field(default=__main__...
|
| ----------------------------------------------------------------------
| Methods inherited from ctapipe.core.container.Container:
|
| __getitem__(self, key)
|
| __init__(self, prefix=None, **fields)
| Initialize self. See help(type(self)) for accurate signature.
|
| __repr__(self)
| Return repr(self).
|
| __setitem__(self, key, value)
|
| __str__(self)
| Return str(self).
|
| as_dict(self, recursive=False, flatten=False, add_prefix=False, add_key=False)
| Convert the `Container` into a dictionary
|
| Parameters
| ----------
| recursive: bool
| sub-Containers should also be converted to dicts
| flatten: type
| return a flat dictionary, with any sub-field keys generated
| by appending the sub-Container name.
| add_prefix: bool
| include the container's prefix in the name of each item
| add_key: bool
| include map key
|
| items(self, add_prefix=False)
| Generator over (key, value) pairs for the items
|
| keys(self)
| Get the keys of the container
|
| reset(self)
| Reset all values back to their default values
|
| Parameters
| ----------
| recursive: bool
| If true, also reset all sub-containers
|
| update(self, **values)
| update more than one parameter at once (e.g. ``update(x=3,y=4)``
| or ``update(**dict_of_values)``)
|
| validate(self)
| Check that all fields in the Container have the expected characteristics (as
| defined by the Field metadata). This is not intended to be run every time a
| Container is filled, since it is slow, only for testing a first event.
|
| Raises
| ------
| ValueError:
| if the Container's values are not valid
|
| values(self)
| Get the keys of the container
95 help(SubContainer)
Help on class SubContainer in module __main__:
class SubContainer(ctapipe.core.container.Container)
| SubContainer(prefix=None, **fields)
|
| Attributes
| ----------
| junk : Field(default=-1)
| Some junk
| value : Field(default=0.0, unit=deg)
| some value
| meta : dict
| dict of attached metadata
| prefix : str
| Prefix attached to column names when saved to a table or file
|
| Method resolution order:
| SubContainer
| ctapipe.core.container.Container
| builtins.object
|
| Data descriptors defined here:
|
| junk
|
| meta
|
| prefix
|
| value
|
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| default_prefix = 'sub'
|
| fields = {'junk': Field(default=-1), 'value': Field(default=0.0, unit=...
|
| ----------------------------------------------------------------------
| Methods inherited from ctapipe.core.container.Container:
|
| __getitem__(self, key)
|
| __init__(self, prefix=None, **fields)
| Initialize self. See help(type(self)) for accurate signature.
|
| __repr__(self)
| Return repr(self).
|
| __setitem__(self, key, value)
|
| __str__(self)
| Return str(self).
|
| as_dict(self, recursive=False, flatten=False, add_prefix=False, add_key=False)
| Convert the `Container` into a dictionary
|
| Parameters
| ----------
| recursive: bool
| sub-Containers should also be converted to dicts
| flatten: type
| return a flat dictionary, with any sub-field keys generated
| by appending the sub-Container name.
| add_prefix: bool
| include the container's prefix in the name of each item
| add_key: bool
| include map key
|
| items(self, add_prefix=False)
| Generator over (key, value) pairs for the items
|
| keys(self)
| Get the keys of the container
|
| reset(self)
| Reset all values back to their default values
|
| Parameters
| ----------
| recursive: bool
| If true, also reset all sub-containers
|
| update(self, **values)
| update more than one parameter at once (e.g. ``update(x=3,y=4)``
| or ``update(**dict_of_values)``)
|
| validate(self)
| Check that all fields in the Container have the expected characteristics (as
| defined by the Field metadata). This is not intended to be run every time a
| Container is filled, since it is slow, only for testing a first event.
|
| Raises
| ------
| ValueError:
| if the Container's values are not valid
|
| values(self)
| Get the keys of the container
values can be set as normal for a class:
101 ev.event_id = 100
102 ev.event_id
100
105 ev.as_dict() # by default only shows the bare items, not sub-containers (See later)
{'event_id': 100, 'tels_with_data': [], 'sub': __main__.SubContainer:
junk: Some junk with default -1
value: some value with default 0.0 [deg], 'tel': Map(__main__.TelContainer, {1: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None})}
108 ev.as_dict(recursive=True)
{'event_id': 100, 'tels_with_data': [], 'sub': {'junk': -1, 'value': 0.0}, 'tel': {1: {'tel_id': -1, 'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}}}
and we can add a few of these to the parent container inside the tel dict:
116 ev.tel[10] = TelContainer()
117 ev.tel[5] = TelContainer()
118 ev.tel[42] = TelContainer()
because we are using a default_factory to handle mutable defaults, the images are actually different:
122 ev.tel[42].image is ev.tel[32]
False
Be careful to use the default_factory mechanism for mutable fields,
see this negative example:
131 class DangerousContainer(Container):
132 image = Field(
133 np.zeros(10),
134 description=(
135 "Attention!!!! Globally mutable shared state. Use default_factory instead"
136 ),
137 )
138
139
140 c1 = DangerousContainer()
141 c2 = DangerousContainer()
142
143 c1.image[5] = 9999
144
145 print(c1.image)
146 print(c2.image)
147 print(c1.image is c2.image)
[ 0. 0. 0. 0. 0. 9999. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 9999. 0. 0. 0. 0.]
True
150 ev.tel
Map(__main__.TelContainer, {1: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None, 10: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None, 5: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None, 42: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None, 32: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None})
Conversion to dictionaries#
158 ev.as_dict()
{'event_id': 100, 'tels_with_data': [], 'sub': __main__.SubContainer:
junk: Some junk with default -1
value: some value with default 0.0 [deg], 'tel': Map(__main__.TelContainer, {1: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None, 10: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None, 5: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None, 42: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None, 32: __main__.TelContainer:
tel_id: telescope ID number with default -1
image: camera pixel data with default None})}
161 ev.as_dict(recursive=True, flatten=False)
{'event_id': 100, 'tels_with_data': [], 'sub': {'junk': -1, 'value': 0.0}, 'tel': {1: {'tel_id': -1, 'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}, 10: {'tel_id': -1, 'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}, 5: {'tel_id': -1, 'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}, 42: {'tel_id': -1, 'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}, 32: {'tel_id': -1, 'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}}}
for serialization to a table, we can even flatten the output into a single set of columns
169 ev.as_dict(recursive=True, flatten=True)
{'event_id': 100, 'tels_with_data': [], 'junk': -1, 'value': 0.0, 'tel_id': -1, 'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
Setting and clearing values#
177 ev.tel[5].image[:] = 9
178 print(ev)
{'event_id': 100,
'sub': {'junk': -1, 'value': 0.0},
'tel': {1: {'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]),
'tel_id': -1},
5: {'image': array([9., 9., 9., 9., 9., 9., 9., 9., 9., 9.]),
'tel_id': -1},
10: {'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]),
'tel_id': -1},
32: {'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]),
'tel_id': -1},
42: {'image': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]),
'tel_id': -1}},
'tels_with_data': []}
181 ev.reset()
182 ev.as_dict(recursive=True)
{'event_id': -1, 'tels_with_data': [], 'sub': {'junk': -1, 'value': 0.0}, 'tel': {}}
look at a pre-defined Container#
191 help(SimulatedShowerContainer)
Help on class SimulatedShowerContainer in module ctapipe.containers:
class SimulatedShowerContainer(ctapipe.core.container.Container)
| SimulatedShowerContainer(prefix=None, **fields)
|
| Attributes
| ----------
| energy : Field(default=nan TeV, unit=TeV)
| Simulated Energy
| alt : Field(default=nan deg, unit=deg)
| Simulated altitude
| az : Field(default=nan deg, unit=deg)
| Simulated azimuth
| core_x : Field(default=nan m, unit=m)
| Simulated core position (x)
| core_y : Field(default=nan m, unit=m)
| Simulated core position (y)
| h_first_int : Field(default=nan m, unit=m)
| Height of first interaction
| x_max : Field(default=nan g / cm2, unit=g / cm2)
| Simulated Xmax value
| starting_grammage : Field(default=nan g / cm2, unit=g / cm2)
| Grammage (mass overburden) where the particle was injected into the atmosphere
| shower_primary_id : Field(default=32767)
| Simulated shower primary ID 0 (gamma), 1(e-),2(mu-), 100*A+Z for nucleons and nuclei,negative for antimatter.
| meta : dict
| dict of attached metadata
| prefix : str
| Prefix attached to column names when saved to a table or file
|
| Method resolution order:
| SimulatedShowerContainer
| ctapipe.core.container.Container
| builtins.object
|
| Data descriptors defined here:
|
| alt
|
| az
|
| core_x
|
| core_y
|
| energy
|
| h_first_int
|
| meta
|
| prefix
|
| shower_primary_id
|
| starting_grammage
|
| x_max
|
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| __slotnames__ = ['energy', 'alt', 'az', 'core_x', 'core_y', 'h_first_i...
|
| default_prefix = 'true'
|
| fields = {'alt': Field(default=nan deg, unit=deg), 'az': Field(default...
|
| ----------------------------------------------------------------------
| Methods inherited from ctapipe.core.container.Container:
|
| __getitem__(self, key)
|
| __init__(self, prefix=None, **fields)
| Initialize self. See help(type(self)) for accurate signature.
|
| __repr__(self)
| Return repr(self).
|
| __setitem__(self, key, value)
|
| __str__(self)
| Return str(self).
|
| as_dict(self, recursive=False, flatten=False, add_prefix=False, add_key=False)
| Convert the `Container` into a dictionary
|
| Parameters
| ----------
| recursive: bool
| sub-Containers should also be converted to dicts
| flatten: type
| return a flat dictionary, with any sub-field keys generated
| by appending the sub-Container name.
| add_prefix: bool
| include the container's prefix in the name of each item
| add_key: bool
| include map key
|
| items(self, add_prefix=False)
| Generator over (key, value) pairs for the items
|
| keys(self)
| Get the keys of the container
|
| reset(self)
| Reset all values back to their default values
|
| Parameters
| ----------
| recursive: bool
| If true, also reset all sub-containers
|
| update(self, **values)
| update more than one parameter at once (e.g. ``update(x=3,y=4)``
| or ``update(**dict_of_values)``)
|
| validate(self)
| Check that all fields in the Container have the expected characteristics (as
| defined by the Field metadata). This is not intended to be run every time a
| Container is filled, since it is slow, only for testing a first event.
|
| Raises
| ------
| ValueError:
| if the Container's values are not valid
|
| values(self)
| Get the keys of the container
194 shower = SimulatedShowerContainer()
195 shower
ctapipe.containers.SimulatedShowerContainer:
energy: Simulated Energy with default nan TeV [TeV]
alt: Simulated altitude with default nan deg [deg]
az: Simulated azimuth with default nan deg [deg]
core_x: Simulated core position (x) with default nan m
[m]
core_y: Simulated core position (y) with default nan m
[m]
h_first_int: Height of first interaction with default nan m
[m]
x_max: Simulated Xmax value with default nan g / cm2 [g
/ cm2]
starting_grammage: Grammage (mass overburden) where the particle
was injected into the atmosphere with default
nan g / cm2 [g / cm2]
shower_primary_id: Simulated shower primary ID 0 (gamma),
1(e-),2(mu-), 100*A+Z for nucleons and
nuclei,negative for antimatter. with default
32767
Container prefixes#
To store the same container in the same table in a file or give more information, containers support setting a custom prefix:
{'foo_junk': 5, 'foo_value': 3, 'bar_junk': 10, 'bar_value': 9001}
Total running time of the script: (0 minutes 0.026 seconds)