Falaise
4.0.1
SuperNEMO Software Toolkit
|
Though Falaise supplies a set of pipelines for standard reconstruction, it is also possible write your own custom pipeline scripts for use in flreconstruct
. These are nothing more than text files marked up using the datatools::multi_properties
syntax. Custom pipeline scripts can be run in flreconstruct
by passing their path to the -p
(or --pipeline
) command, for example
or
The major use cases for custom pipelines are to study performance improvements to the current pipelines by tuning parameters or implementing new modules. Here we cover the task of writing a script to build a custom pipeline from the standard modules supplied with Falaise. The more advanced step of writing and using new modules is covered in the Writing FLReconstruct Modules section.
A list of pipeline modules known to flreconstruct
can be obtained using its --help-module-list
argument, which will print the module names to stdout:
Details about the purpose of a module and the parameters you can supply to configure it may be obtained by passing the module name as the argument of the --help-module
command. This will print out detailed documentation, if it exists:
This information can be used to select suitable modules and configurations for your own pipeline. The following sections provide some simple examples of pipeline scripts.
To demonstrate the basic syntax and structure of an flreconstruct
pipeline script, the following sections build up functionality from single modules up to using a chain of plugin modules.
If you do not supply a pipeline script to flreconstruct
, it will run a dumb pipeline that simple dumps each event to stdout.
We can reproduce this behaviour using the following simple script to configure the pipeline as a single module:
The script is formatted as a datatools::multi_properties
ASCII file. Note the comments, especially:
@key_label
and @meta_label
multi_properties flagsname
key set to pipeline
. FLReconstruct will use this to build the pipeline.To try this out, copy the above text into a file, e.g. trivial_pipeline.txt
and run flreconstruct
with it:
You should see the same output as the default case.
A pipeline with one module isn't very useful! In most cases we want to plug together a sequence of modules, each performing a well defined task on the event data.
Falaise supplies a special chain_module
for chaining several modules together. We can reproduce the same default dump behaviour using a chain_module
in a pipeline script as follows
Here, the modules
key of the pipeline
module takes a list of modules that will form the chain_module
. To try this out, copy the above text into a file, e.g. single_chain_pipeline.txt
and run flreconstruct
with it:
You should see the same output as the default case.
A full chain pipeline can chain together 1 < N < X number of modules. The previous example showed the basic construct of a chained pipeline with a single module. We can of course go further and add further module to the pipeline. For example, we can chain two dump modules together. To distinguish these, we configure the modules to have different title
and indent
parameters.
The order in which the modules are processed is determined by the order in which you list them in the modules
key of the dpp::chain_module
configuration.
To try this out, copy the above text into a file, e.g. multi_chain_pipeline.txt
and run flreconstruct
with it:
Try swapping the order of the modules to see what happens, and also try adding further dump modules to the pipeline to experiment with the sequence.
Falaise supplies several modules as plugins, that is, functionality loaded into flreconstruct
after it starts running.
You can also write your own modules in C++ and plug them into the pipeline to provide additional functionality. The proceedure for writing and using new modules is covered in several stages, beginning with a simple example.
FLreconstruct scripts use the datatools::multi_properties
format from Bayeux. The script must begin with a mandatory header:
The '#@description
' line is optional but highly recommended.
Comments can be placed at any point in the script. Just prepend a sharp ('#
') symbol to any comment line. You may also prepend a comment at the end of a line:
Note that lines starting with ‘’#‘’ are generally special meta-comments with embedded commands. They should not be considered as comments. As a matter of rule, the use of lines starting with '#@
' is reserved for system use.
After the header, the script contains sections. A section starts with a section definition line with two identifiers:
flreconstruct
scripts it must be "flreconstruct::section"
).The syntax is:
The section body uses the datatools::properties
format from Bayeux. After the definition line, a short description may be optionally provided thanks to the '#@config
' meta-comment:
Then comes the section body which consists in a list of parameter setting directives. The format is:
where NAME
is the identifier for the parameter, TYPE
is its type and VALUE
the selected value for this parameter. Some parameters may use an optional DECORATOR
which gives additional information about the parameter's type or processing. Again, the #@description
line is optional, but recommended. Example:
When we generate simulated events with FLSimulate, the so-called SD
bank contains only raw information about truth calorimeter and tracker hits. In order to be able to reconstruct and analyze the events, we must apply a calibration procedure. This calibration procedure consists of determining physics/geometrical observables associated to hits : deposited energy, particle times of flight, hit positions in the geometry model and so on.
The so-called mock calibration computes the calibrated data associated to raw simulated data.
The mock calibration consists of:
A new CD
bank holding these data is added in the event record, alongside the raw SD
bank.
To see this processing in action, we first prepare a set of 10 simulated events (using the default simulation setup), using the following simu.conf
script for FLSimulate:
and run:
The resulting example.brio
file can be opened with the flvisualize
application. We can visualize the first event where we will see any raw tracker/calorimeter hits, as well as the position of the decay vertex (cross on the source foil). An example event is shown below, and your view may differ as by default flsimulate will use different seeds on each run, and the styling/colours may also vary between machines:
To process this data in flreconstruct
and apply the mock calibration we create the rec.conf
script with a custom pipeline
inline module made of three consecutive modules. The first one is responsible of the calibration of the tracker hits. The second one is responsible of the calibration of the calorimeter hits. The last one simply prints the content of the events in the standard output:
We use default settings for each of the CalibrateTracker
, CalibrateCalorimeters
and Dump
modules so the corresponding sections are empty.
We then run:
The use of the Dump
module prints the events in the terminal. Here is the first event:
We clearly see that a new CD
bank (calibrated data) has been added to the event record. It contains two collections of hits, respectively calorimeter and tracker hits.
The example-cd.brio
file output by flreconstruct
and the pipeline script may be opened in flvisualize
as before. Looking at the first event again, we can see additional rendering and information associated to the CD
hits (as before, your output will differ due to different random number seeds used in flsimulate
):
The superimposed white circles on top of the colored ones represent the calibrated tracker hits for which the drift radius and longitudinal position along the anode wire have been computed from the truth hits' timestamps. Here, there is only one calorimeter hit, the total deposited energy in the block and time of entering the scintillator block are also computed (with respect to the arbitrary decay time set by the simulation engine).
Now we know how to perform the calibration step, we are interested in the reconstruction of electron tracks in the tracking chamber. For that we need additional processing on top of the CD
bank which is now available in the event record.
The reconstruction of tracks associated to charged particles traversing the tracking chamber is a two step procedure:
We create a new rec.conf
script with a custom pipeline
inline module made of five consecutive modules. The first two perform the mock calibration as shown in the previous section. Then we use a new module CATTrackerClusterizer
to run the clustering algorithm, followed by the TrackFitting
module to run the line/helix fitting on the found clusters. As before, we add a dump module at the end of the pipeline to print each event to standard output.
As we now use modules implemented by dedicated Falaise plugins, we must explicitly provide a flreconstruct.plugins
section with the list of plugins that must be dynamically loaded to allow the pipeline to work:
We run:
Now two banks have been added in the event records:
TCD
: the tracker clustering data contains the result of the tracker hit clusterization with one or several solutions, each containing a collection of candidate clusters of hit cells. These clusters are used as the input of the track fitting algorithmTTD
: the tracker trajectory data contains the result of the tracker fitting with one or several solutions, each containing a collection of candidate fitted tracks (helix or line).The display shows the best found clusterization/fitting solutions with two clusters of tracker hits (red and blue) and the best associated tracks that have been computed from these clusters. We clearly recognize a two electrons event pattern, but here only one is associated to a calorimeter block.
Document more reconstruction steps:
The FLReconstruct script contains up to five sections of type flreconstruct::section
with the following names:
flreconstruct
: this is the system/base section where to set general parameters such as:numberOfEvents
: the number of events to be processed from the input (integer, optional, default is: 0
which means all events will be processed),experimentalSetupUrn
: the experimental setup tag (default is: urn:snemo:demonstrator:setup:1.0
),flreconstruct.variantService
: this is the variants section where the Bayeux variant service dedicated to the management of variant parameters and configurations is configured. In principle, this section inherits the configuration of the variant service used to generate simulated data.
Parameters of interest are:
configUrn
: the configuration tag for the variant service associated to the simulation setup (string, optional). If not set, it is automatically resolved from the experiment setup tag.config
: the path to the main configuration file for the variant service associated to the simulation setup (string/path, optional). If not set, it is automatically resolved from the configUrn
tag.profileUrn
: the configuration tag for the variant profile chosen by the user to perform the simulation (string, optional). If not set, it may be automatically resolved from the configUrn
tag if the variant configuration has a registered default profile.profile
: the path to the variant profile chosen by the user to perform the simulation (string/path,optional). If not set, it is automatically resolved from the profileUrn
tag or from input metadata.settings
: a list of explicit setting for variant parameters chosen by the user to perform the simulation (array of strings, optional). If not set, it is automatically resolved from the profileUrn
tag or from input metadata.flreconstruct.services
: this is the services section where explicit configuration for the embedded Bayeux/datatools service manager is defined (by tag or explicit configuration file).
Parameters of interest are:
configUrn
: the configuration tag for the service manager associated to the data producer setup (string, optional). If not set, it is automatically resolved from the experimental setup tag.config
: the path to the main configuration file for the service manager service associated to the reconstrcution setup (string/path, optional). If not set, it is automatically resolved from the configUrn
tag.flreconstruct.plugins
: this is the plugins section where explicit directives are defined to load Falaise plugin libraries which define various types (classes) of processing modules.
Parameters of interest are:
plugins
: the list of plugins to be loaded.PLUGIN_NAME.directory
: the directory from where the plugin dynamic library named PLUGIN_NAME
should be loaded (default: "@falaise.plugins:", i.e. the standard location for the installation of Falaise's plugins).flreconstruct.pipeline
: this is the pipeline section where general setup of the reconstruction pipeline is provided.
Parameters of interest are:
configUrn
: the tag of a registered/official pipeline.config
: the main configuration file describing the modules used along the pipeline. If not set, it is automatically resolved from the configUrn
tag.module
: the name of the top level pipeline module, chosen from the list of modules defined in the main configuration file (default: "pipeline"
).A sample configuration script showing the organisation of the above parameters is shown below. Note that many of the parameters are commented out as they are generally note needed except for advanced use or testing.
The majority of scripts will just use the flreconstruct.plugins
and flreconstruct.pipeline
sections to select from the set of plugins and pipelines approved by the Calibration and Reconstruction Working Group. Output results from these can be used in the preparation of analyses.
In the case the flreconstruct.pipeline
section does not define an explicit pipeline configuration by tag (configUrn
) or configuration file path (config
), it is possible to provide a list of additional sections which define reconstruction modules. This technique is named the inline pipeline mode. Such a section uses the following format:
where ModuleName
is the unique name of the module and ModuleType
is the identifier of its registered class type (see below).
This mode should not be used for production runs. It is expected that only official registered pipeline setups are used in such a case, through the configUrn
properties in the flreconstruct.pipeline
section.