Tool#
- class ctapipe.core.Tool(**kwargs: Any)[source]#
Bases:
Application
A base class for all executable tools (applications) that handles configuration loading/saving, logging, command-line processing, and provenance meta-data handling. It is based on
traitlets.config.Application
. Tools may contain configurablectapipe.core.Component
classes that do work, and their configuration parameters will propagate automatically to theTool
.Tool developers should create sub-classes, and a name, description, usage examples should be added by defining the
name
,description
andexamples
class attributes as strings. Thealiases
attribute can be set to cause a lower-levelComponent
parameter to become a high-level command-line parameter (See example below). Thesetup
,start
, andfinish
methods should be defined in the sub-class.Additionally, any
ctapipe.core.Component
used within theTool
should have their class in a list in theclasses
attribute, which will automatically add their configuration parameters to the tool.Once a tool is constructed and the abstract methods are implemented, the user can call the
run
method to setup and start it.Tools have an
ExitStack
to guarantee cleanup tasks are run when the tool terminates, also in case of errors. If a task needs a cleanup, it must be a context manager andTool.enter_context
must be called on the object. This will guarantee that the__exit__
method of the context manager is called after the tool has finished executing. This happens after thefinish
method has run or in case of exceptions.User-defined code can raise custom exceptions both in the components or in the tool methods. If these custom exceptions have an
exit_code
attribute, it will be propagated to the final exit code of the tool.from ctapipe.core import Tool from traitlets import (Integer, Float, Dict, Unicode) class MyTool(Tool): name = "mytool" description = "do some things and stuff" aliases = Dict({'infile': 'AdvancedComponent.infile', 'iterations': 'MyTool.iterations'}) # Which classes are registered for configuration classes = [MyComponent, AdvancedComponent, SecondaryMyComponent] # local configuration parameters iterations = Integer(5,help="Number of times to run", allow_none=False).tag(config=True) def setup(self): self.comp = MyComponent(self, parent=self) self.comp2 = SecondaryMyComponent(self, parent=self) # correct use of component that is a context manager # using it like this makes sure __exit__ will be called # at the end of Tool.run, even in case of exceptions self.comp3 = self.enter_context(MyComponent(parent=self)) self.advanced = AdvancedComponent(parent=self) def start(self): self.log.info("Performing {} iterations..." .format(self.iterations)) for ii in range(self.iterations): self.log.info("ITERATION {}".format(ii)) self.comp.do_thing() self.comp2.do_thing() sleep(0.5) def finish(self): self.log.warning("Shutting down.") def main(): tool = MyTool() tool.run() if __name__ == "main": main()
If this
main()
function is registered insetup.py
under entry_points, it will become a command-line tool (see examples in thectapipe/tools
subdirectory).Attributes Summary
List of configuration files with parameters to load in addition to command-line parameters.
An instance of a Python dict.
Filename for the log
Logging Level for File Logging
A boolean (True, False) trait.
A path Trait for input/output files.
A boolean (True, False) trait.
a formatted version string with version, release, and git hash
Methods Summary
add_component
(component_instance)constructs and adds a component to the list of registered components, so that later we can ask for the current configuration of all instances, e.g. in`get_full_config()`.
check_output
(*output_paths)Test if output files exist and if they do, throw an error unless
self.overwrite
is set to True.enter_context
(context_manager)Add a new context manager to the
ExitStack
of this Toolfinish
()Finish up.
return the current configuration as a dict (e.g. the values of all traits, even if they were not set during configuration).
initialize
([argv])handle config and any other low-level setup
load_config_file
(path)Load a configuration file in one of the supported formats, and merge it with the current config if it exists.
run
([argv, raises])Run the tool.
setup
()Set up the tool.
start
()Main function of the tool.
Update the configuration of loggers.
Attributes Documentation
- config_files#
List of configuration files with parameters to load in addition to command-line parameters. The order listed is the order of precedence (later config parameters overwrite earlier ones), however parameters specified on the command line always have the highest precedence. Config files may be in JSON, YAML, TOML, or Python format
- log_config#
An instance of a Python dict.
One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.
Changed in version 5.0: Added key_trait for validating dict keys.
Changed in version 5.0: Deprecated ambiguous
trait
,traits
args in favor ofvalue_trait
,per_key_traits
.
- log_file#
Filename for the log
- log_file_level#
Logging Level for File Logging
- overwrite#
A boolean (True, False) trait.
- provenance_log#
A path Trait for input/output files.
- Attributes:
- exists: boolean or None
If True, path must exist, if False path must not exist
- directory_ok: boolean
If False, path must not be a directory
- file_ok: boolean
If False, path must not be a file
- quiet#
A boolean (True, False) trait.
- version_string#
a formatted version string with version, release, and git hash
Methods Documentation
- add_component(component_instance)[source]#
constructs and adds a component to the list of registered components, so that later we can ask for the current configuration of all instances, e.g. in`get_full_config()`. All sub-components of a tool should be constructed using this function, in order to ensure the configuration is properly traced.
- Parameters:
- component_instance: Component
constructed instance of a component
- Returns:
- Component:
the same component instance that was passed in, so that the call can be chained.
Examples
self.mycomp = self.add_component(MyComponent(parent=self))
- check_output(*output_paths)[source]#
Test if output files exist and if they do, throw an error unless
self.overwrite
is set to True. This should be checked during tool setup to avoid having a tool only realize the output can not be written after some long-running calculations (e.g. training of ML-models). Because we currently do not collect all created output files in the tool (they can be attached to some component), the output files need to be given and can not easily be derived fromself
.- Parameters:
- output_paths: Path
One or more output path to check.
- enter_context(context_manager)[source]#
Add a new context manager to the
ExitStack
of this ToolThis method should be used with things that need a cleanup step, also in case of exception.
enter_context
will callcontext_manager.__enter__
and return its result.This will guarantee that
context_manager.__exit__
is called whenrun
finishes, even when an error occurs.
- abstract finish()[source]#
Finish up.
This is called automatically after
Tool.start
whenTool.run
is called.
- get_current_config()[source]#
return the current configuration as a dict (e.g. the values of all traits, even if they were not set during configuration)
- load_config_file(path: str | Path) None [source]#
Load a configuration file in one of the supported formats, and merge it with the current config if it exists.
- Parameters:
- path: Union[str, pathlib.Path]
config file to load. [yaml, toml, json, py] formats are supported
- run(argv=None, raises=False)[source]#
Run the tool.
This automatically calls
Tool.initialize
,Tool.start
andTool.finish
- Parameters:
- argv: list(str)
command-line arguments, or None to get them from sys.argv automatically
- raisesbool
Whether to raise Exceptions (to test them) or not.
- abstract setup()[source]#
Set up the tool.
This method runs after the configuration and command line options have been parsed.
Here the tool should construct all
Components
, open files, etc.
- abstract start()[source]#
Main function of the tool.
This is automatically called after
Tool.initialize
whenTool.run
is called.