:orphan:

.. _starterkit_label:

``b2luigi`` Starter Kit
========================

This tutorial was developed as a workshop for the October 2024 Belle II General Meeting.
The idea of this tutorial is that each of the following examples is present in your current working directory and is executed by hand.
To get the example files...

1. ... eiher copy the Python code directly from this page into a blank file or...
2. ... download the examples directly from their respective pages or...
3. ... clone the ``b2luigi`` repository and move to the ``examples`` directory, e.g.

.. code-block:: bash

    git clone https://gitlab.desy.de/belle2/software/b2luigi.git
    cd b2luigi/examples

Alternatively, you can also clone directly from `GitHub <https://github.com/belle2/b2luigi>`_.

In any of these cases, it is recommended to run the tutorial in a virtual environment.
With access to the Belle II software stack, you can use the following commands to set up a virtual environment and install the necessary packages:

.. code-block:: bash

    source /cvmfs/belle.cern.ch/tools/b2setup
    b2venv release-09-00-00

.. hint::

    Use a full ``basf2`` release if you want to run the ``basf2`` examples with the reconstruction.

The ``b2venv`` command creates a ``venv`` directory in the repository with the name "venv".
The next step is to activate the environment:

.. code-block:: bash

    source venv/bin/activate

This environment will be based on the ``basf2`` environment.
However, Python packages that are not provided by the externals will be installed in the virtual environment.
To install the requirements with in a ``b2venv`` for this tutorial run:

.. code-block:: bash

    pip install b2luigi

If you use a clear ``venv``, you can install the requirements with:

.. code-block:: bash

    pip install b2luigi pandas pyarrow uproot matplotlib plothist

If you copied the examples from the Git repository, you can install the requirements with:

.. code-block:: bash

    pip install -r requirements.txt


.. raw:: html

  <div id='sg-tag-list' class='sphx-glr-tag-list'></div>


.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="(b2)luigi tasks are defined as classes that inherit from a luigi task class, for this example we will start with a basic class that inherits from b2luigi.Task. The task class should define the parameters, the run method, and the output method. The parameters are defined as class attributes, and the run method contains the actual computation. The output method defines the output of the task, which is used to determine if the task has been completed.">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex01_basics_b2luigi_task_thumb.png
    :alt:

  :doc:`/starterkit/Ex01_basics_b2luigi_task`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">A Simple b2luigi Task</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The requires method is used to define dependencies between different tasks. The method should return iterable instances of the required tasks. This task will be scheduled to run after the output of the required task is completed. The output is considered completed if the output files are present.">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex02_basics_b2luigi_require_thumb.png
    :alt:

  :doc:`/starterkit/Ex02_basics_b2luigi_require`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Building a Dependency Graph with b2luigi</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The b2luigi.WrapperTask class is used to define a task that requires all parameter combinations of another task to be executed. The combinations are determined by the requires method of the b2luigi.WrapperTask.">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex03_basics_b2luigi_wrappertask_thumb.png
    :alt:

  :doc:`/starterkit/Ex03_basics_b2luigi_wrappertask`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">b2luigi.WrapperTask and b2luigi Settings</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="b2luigi.WrapperTask is not the only way to require multiple tasks. Also normal tasks can require multiple tasks. This is useful if the wrapper task uses the inputs of the required tasks to compute its output.">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex04_basics_b2luigi_averagetask_thumb.png
    :alt:

  :doc:`/starterkit/Ex04_basics_b2luigi_averagetask`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Multiple Tasks and Introduction to Schedulers</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="For directly working with the Belle II Analysis Software Framework (``basf2``), b2luigi provides the Basf2PathTask task class. This class provides help for the user to create and execute a basf2 path. The Basf2PathTask also provides the following parameters that can be useful when executing basf2 processes:">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex05_basf2_simulation_thumb.png
    :alt:

  :doc:`/starterkit/Ex05_basf2_simulation`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Introduction to Basf2PathTask</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="We start with the usual imports. Note that we import the task SimulationTask class defined in the previous example exercise05_label: it&#x27;s generally recommended to keep your code as much modular as possible.">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex06_basf2_reconstruction_thumb.png
    :alt:

  :doc:`/starterkit/Ex06_basf2_reconstruction`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">The requires decorator</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Writing a basf2 analysis steering file">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex07_basf2_analysis_thumb.png
    :alt:

  :doc:`/starterkit/Ex07_basf2_analysis`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Writing a basf2 analysis steering file</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Scaling up your analysis">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex08_basf2_analysis_scaled_thumb.png
    :alt:

  :doc:`/starterkit/Ex08_basf2_analysis_scaled`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Scaling up your analysis</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="To define the batch system on which this task should run, the only change needed is to set the batch_system parameter to the desired batch system. The available options are:">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex09_basf2_analysis_LSF_thumb.png
    :alt:

  :doc:`/starterkit/Ex09_basf2_analysis_LSF`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Submitting a task to the LSF batch system</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="HTCondor specific settings that can be automatically provided to a htcondor task are:">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex10_basf2_analysis_HTCondor_thumb.png
    :alt:

  :doc:`/starterkit/Ex10_basf2_analysis_HTCondor`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Submitting a task to the HTCondor batch system</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="To submit jobs via gbasf2, the user will need to have a valid grid certificate. In principle, b2luigi will execute the gbasf2 setup with">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex11_basf2_analysis_gbasf2_thumb.png
    :alt:

  :doc:`/starterkit/Ex11_basf2_analysis_gbasf2`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">gbasf2 Analysis Task</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="One of the command line tools in b2luigi is the dry_run method. This will only execute the b2luigi.Task.dry_run method of each task. This can be useful when developing a pipeline and one wants to preview the output of the pipeline without actually running it.">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex12_merge_files_thumb.png
    :alt:

  :doc:`/starterkit/Ex12_merge_files`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Common Analysis Workflow</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In a separate module, we define the PlotInvariantMass class that performs the actual plotting of the invariant mass.">

.. only:: html

  .. image:: /starterkit/images/thumb/sphx_glr_Ex13_plot_invariant_mass_thumb.png
    :alt:

  :doc:`/starterkit/Ex13_plot_invariant_mass`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Common Analysis Workflow Continued</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /starterkit/Ex01_basics_b2luigi_task
   /starterkit/Ex02_basics_b2luigi_require
   /starterkit/Ex03_basics_b2luigi_wrappertask
   /starterkit/Ex04_basics_b2luigi_averagetask
   /starterkit/Ex05_basf2_simulation
   /starterkit/Ex06_basf2_reconstruction
   /starterkit/Ex07_basf2_analysis
   /starterkit/Ex08_basf2_analysis_scaled
   /starterkit/Ex09_basf2_analysis_LSF
   /starterkit/Ex10_basf2_analysis_HTCondor
   /starterkit/Ex11_basf2_analysis_gbasf2
   /starterkit/Ex12_merge_files
   /starterkit/Ex13_plot_invariant_mass



.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_