Tool

class ctapipe.core.Tool(**kwargs: Any)

Bases: traitlets.config.application.Application

A base class for all executable tools (applications) that handles configuration loading/saving, logging, command-line processing, and provenance meta-data handling. It is based on traitlets.config.Application. Tools may contain configurable ctapipe.core.Component classes that do work, and their configuration parameters will propagate automatically to the Tool.

Tool developers should create sub-classes, and a name, description, usage examples should be added by defining the name, description and examples class attributes as strings. The aliases attribute can be set to cause a lower-level Component parameter to become a high-level command-line parameter (See example below). The setup, start, and finish methods should be defined in the sub-class.

Additionally, any ctapipe.core.Component used within the Tool should have their class in a list in the classes attribute, which will automatically add their configuration parameters to the tool.

Once a tool is constructed and the abstract methods are implemented, the user can call the run method to setup and start it.

Tools have an ExitStack to guarantee cleanup tasks are run when the tool terminates, also in case of errors. If a task needs a cleanup, it must be a context manager and Tool.enter_context must be called on the object. This will guarantee that the __exit__ method of the context manager is called after the tool has finished executing. This happens after the finish method has run or in case of exceptions.

from ctapipe.core import Tool
from traitlets import (Integer, Float, Dict, Unicode)

class MyTool(Tool):
    name = "mytool"
    description = "do some things and stuff"
    aliases = Dict({'infile': 'AdvancedComponent.infile',
                    'iterations': 'MyTool.iterations'})

    # Which classes are registered for configuration
    classes = [MyComponent, AdvancedComponent, SecondaryMyComponent]

    # local configuration parameters
    iterations = Integer(5,help="Number of times to run",
                         allow_none=False).tag(config=True)

    def setup(self):
        self.comp = MyComponent(self, parent=self)
        self.comp2 = SecondaryMyComponent(self, parent=self)

        # correct use of component that is a context manager
        # using it like this makes sure __exit__ will be called
        # at the end of Tool.run, even in case of exceptions
        self.comp3 = self.enter_context(MyComponent(parent=self))

        self.advanced = AdvancedComponent(parent=self)

    def start(self):
        self.log.info("Performing {} iterations..."                              .format(self.iterations))
        for ii in range(self.iterations):
            self.log.info("ITERATION {}".format(ii))
            self.comp.do_thing()
            self.comp2.do_thing()
            sleep(0.5)

    def finish(self):
        self.log.warning("Shutting down.")

def main():
    tool = MyTool()
    tool.run()

if __name__ == "main":
   main()

If this main() function is registered in setup.py under entry_points, it will become a command-line tool (see examples in the ctapipe/tools subdirectory).

Attributes Summary

config_files	An instance of a Python list.
log_config	An instance of a Python dict.
log_file	Filename for the log
log_file_level	Logging Level for File Logging
overwrite	A boolean (True, False) trait.
provenance_log	A path Trait for input/output files.
quiet	A boolean (True, False) trait.
version_string	a formatted version string with version, release, and git hash

Methods Summary

add_component(component_instance)	constructs and adds a component to the list of registered components, so that later we can ask for the current configuration of all instances, e.g.
check_output(*output_paths)	Test if output files exist and if they do, throw an error unless self.overwrite is set to True.
enter_context(context_manager)	Add a new context manager to the ExitStack of this Tool
finish()	Finish up.
get_current_config()	return the current configuration as a dict (e.g.
initialize([argv])	handle config and any other low-level setup
load_config_file(path)	Load a configuration file in one of the supported formats, and merge it with the current config if it exists.
run([argv, raises])	Run the tool.
setup()	Set up the tool.
start()	Main function of the tool.
update_logging_config()	Update the configuration of loggers.
write_provenance()

Attributes Documentation

config_files

An instance of a Python list.

log_config

An instance of a Python dict.

One or more traits can be passed to the constructor to validate the keys and/or values of the dict. If you need more detailed validation, you may use a custom validator method.

Changed in version 5.0: Added key_trait for validating dict keys.

Changed in version 5.0: Deprecated ambiguous trait, traits args in favor of value_trait, per_key_traits.

log_file

Filename for the log

log_file_level

Logging Level for File Logging

overwrite

A boolean (True, False) trait.

provenance_log

A path Trait for input/output files.

Attributes

exists: boolean or None

If True, path must exist, if False path must not exist

directory_ok: boolean

If False, path must not be a directory

file_ok: boolean

If False, path must not be a file

quiet

A boolean (True, False) trait.

version_string

a formatted version string with version, release, and git hash

Methods Documentation

add_component(component_instance)

constructs and adds a component to the list of registered components, so that later we can ask for the current configuration of all instances, e.g. in`get_full_config()`. All sub-components of a tool should be constructed using this function, in order to ensure the configuration is properly traced.

Parameters

component_instance: Component

constructed instance of a component

Returns

Component:

the same component instance that was passed in, so that the call can be chained.

Examples

self.mycomp = self.add_component(MyComponent(parent=self))

check_output(*output_paths)

Test if output files exist and if they do, throw an error unless self.overwrite is set to True. This should be checked during tool setup to avoid having a tool only realize the output can not be written after some long-running calculations (e.g. training of ML-models). Because we currently do not collect all created output files in the tool (they can be attached to some component), the output files need to be given and can not easily be derived from self.

Parameters

output_paths: Path

One or more output path to check.

enter_context(context_manager)

Add a new context manager to the ExitStack of this Tool

This method should be used with things that need a cleanup step, also in case of exception. enter_context will call context_manager.__enter__ and return its result.

This will guarantee that context_manager.__exit__ is called when run finishes, even when an error occurs.

abstract finish()

Finish up.

This is called automatically after Tool.start when Tool.run is called.

get_current_config()

return the current configuration as a dict (e.g. the values of all traits, even if they were not set during configuration)

initialize(argv=None)

handle config and any other low-level setup

load_config_file(path: Union[str, pathlib.Path]) → None

Load a configuration file in one of the supported formats, and merge it with the current config if it exists.

Parameters

path: Union[str, pathlib.Path]

config file to load. [yaml, toml, json, py] formats are supported

run(argv=None, raises=False)

Run the tool.

This automatically calls Tool.initialize, Tool.start and Tool.finish

Parameters

argv: list(str)

command-line arguments, or None to get them from sys.argv automatically

raisesbool

Whether to raise Exceptions (to test them) or not.

abstract setup()

Set up the tool.

This method runs after the configuration and command line options have been parsed.

Here the tool should construct all Components, open files, etc.

abstract start()

Main function of the tool.

This is automatically called after Tool.initialize when Tool.run is called.

update_logging_config()

Update the configuration of loggers.

write_provenance()