CLI Reference#
Command reference#
b2luigi#
Usage: b2luigi [OPTIONS] COMMAND [ARGS]...
Run user-defined b2luigi tasks
╭─ Options ─────────────────────────────────────────────────────╮
│ --version -V,-v Show the version and exit. │
│ --install-completion Install completion for the │
│ current shell. │
│ --show-completion Show completion for the │
│ current shell, to copy it │
│ or customize the │
│ installation. │
│ --help Show this message and │
│ exit. │
╰───────────────────────────────────────────────────────────────╯
╭─ Commands ────────────────────────────────────────────────────╮
│ about Show environment and b2luigi installation info. │
│ init Create starter tasks.py, parameters.py, and │
│ optional config. │
│ version Print version and exit. │
│ self-update Update b2luigi to the latest version in the │
│ current environment. │
│ run Run a task class from tasks.py │
│ show Show output files of task(s). Without a task │
│ name shows the full dependency tree. │
│ remove Remove output files of task(s). │
│ test Build a task from the given python script and │
│ execute it as b2luigi task. │
│ tasks List and inspect available task classes. │
│ graph Render the task dependency graph as a tree │
│ (terminal) or DOT (Graphviz). │
╰───────────────────────────────────────────────────────────────╯
about#
Usage: b2luigi about [OPTIONS]
Show environment and b2luigi installation info.
╭─ Options ─────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰───────────────────────────────────────────────────────────────╯
init#
Usage: b2luigi init [OPTIONS]
Create starter tasks.py, parameters.py, and optional config.
╭─ Options ─────────────────────────────────────────────────────╮
│ --force Overwrite existing files. │
│ --help Show this message and exit. │
╰───────────────────────────────────────────────────────────────╯
version#
Usage: b2luigi version [OPTIONS]
Print version and exit.
╭─ Options ─────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰───────────────────────────────────────────────────────────────╯
self-update#
Usage: b2luigi self-update [OPTIONS]
Update b2luigi to the latest version in the current
environment.
╭─ Options ─────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰───────────────────────────────────────────────────────────────╯
run#
Usage: b2luigi run [OPTIONS] {classname} COMMAND [ARGS]...
Run a task class from tasks.py
╭─ Arguments ───────────────────────────────────────────────────╮
│ * classname <str> The name of the task class to run. │
│ [required] │
╰───────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────╮
│ --task-file -f <str> Task definitions file (or │
│ $B2LUIGI_TASK_FILE) │
│ --params-file -p <str> Parameters file (or │
│ $B2LUIGI_PARAMS_FILE) │
│ --param -P <str> Override task parameters │
│ (repeatable): key=value. │
│ Values are parsed as JSON │
│ when possible. │
│ --dry -d Instead of running the │
│ task(s), write out which │
│ tasks will be executed. │
│ --batch -b Submit tasks to the │
│ configured batch system │
│ instead of running locally. │
│ --tui Show a live progress TUI │
│ while running tasks. │
│ Requires b2luigi[tui]. │
│ --scheduler-host <str> Host of a central luigi │
│ scheduler to connect to │
│ (instead of running locally) │
│ --scheduler-port <int> Port of a central luigi │
│ scheduler to connect to │
│ (instead of running locally) │
│ --workers <int> Number of parallel luigi │
│ workers to use. Overrides │
│ the 'workers' setting. │
│ --help Show this message and exit. │
╰───────────────────────────────────────────────────────────────╯
show#
Usage: b2luigi show [OPTIONS] [classnames]... COMMAND [ARGS]...
Show output files of task(s). Without a task name shows the
full dependency tree.
╭─ Arguments ───────────────────────────────────────────────────╮
│ classnames <str> Task class name(s) to show. Omit to │
│ show the full dependency tree for │
│ all tasks. │
╰───────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────╮
│ --task-file -f <str> Task definitions file (or │
│ $B2LUIGI_TASK_FILE) │
│ --params-file -p <str> Parameters file (or │
│ $B2LUIGI_PARAMS_FILE) │
│ --param -P <str> Override task parameters │
│ (repeatable): key=value. │
│ Values are parsed as JSON │
│ when possible. │
│ --direct Skip dependency graph │
│ traversal. Requires all │
│ target task parameters to │
│ be resolvable from │
│ parameters.py or --param. │
│ Useful for large graphs. │
│ --with-requirements Also show outputs of all │
│ tasks that the specified │
│ task(s) require. Requires │
│ positional task name(s). │
│ --details Show the Output key-name │
│ column, and (for tasks │
│ resolving to multiple │
│ parameter combinations) │
│ the Params column. Both │
│ are hidden by default. │
│ --paths Print one bare output │
│ path per line, with no │
│ table or styling. │
│ Suitable for piping. │
│ --links Make local output paths │
│ clickable via terminal │
│ hyperlinks. Ignored for │
│ remote targets, and note │
│ the link resolves on the │
│ machine your terminal │
│ runs on — not over SSH. │
│ --help Show this message and │
│ exit. │
╰───────────────────────────────────────────────────────────────╯
remove#
Usage: b2luigi remove [OPTIONS] [classnames]... COMMAND
[ARGS]...
Remove output files of task(s).
╭─ Arguments ───────────────────────────────────────────────────╮
│ classnames <str> Task class name(s) to remove. Omit │
│ to remove outputs for all project │
│ tasks (everything b2luigi tasks │
│ lists, not just names in tasks.py). │
╰───────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────╮
│ --task-file -f <str> Task definitions file (or │
│ $B2LUIGI_TASK_FILE) │
│ --params-file -p <str> Parameters file (or │
│ $B2LUIGI_PARAMS_FILE) │
│ --yes -y Skip confirmation prompt. │
│ --keep <str> Comma-separated task │
│ class names whose outputs │
│ should NOT be removed. │
│ Names are validated; │
│ dotted module.Class names │
│ are accepted. │
│ --param -P <str> Override task parameters │
│ (repeatable): key=value. │
│ Values are parsed as JSON │
│ when possible. │
│ --direct Skip dependency graph │
│ traversal. Requires all │
│ target task parameters to │
│ be resolvable from │
│ parameters.py or --param. │
│ Useful for large graphs. │
│ --with-requirements Also remove outputs of │
│ all tasks that the named │
│ task(s) require. │
│ --help Show this message and │
│ exit. │
╰───────────────────────────────────────────────────────────────╯
test#
Usage: b2luigi test [OPTIONS] [extra_args]... COMMAND [ARGS]...
Build a task from the given python script and execute it as
b2luigi task.
╭─ Arguments ───────────────────────────────────────────────────╮
│ extra_args <str> Extra arguments forwarded verbatim │
│ to the script subprocess. │
╰───────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────╮
│ * -s <str> Path to the │
│ Python script │
│ to execute as │
│ a b2luigi │
│ task. │
│ [required] │
│ * -o <str> Output │
│ filename for │
│ the task │
│ target. │
│ [required] │
│ -i <str> Optional input │
│ filename; │
│ creates a │
│ prerequisite │
│ task. │
│ --force Always re-run │
│ even if the │
│ output already │
│ exists. │
│ --batch Submit task │
│ via batch │
│ system │
│ (batch_system… │
│ --batch-syst… <str> Batch system │
│ to submit to │
│ (implies │
│ --batch). Use │
│ it to reach a │
│ system that │
│ cannot be │
│ auto-detected │
│ from PATH │
│ (gbasf2), or │
│ to choose │
│ between │
│ several that │
│ are installed. │
│ Equivalent to │
│ --setting │
│ batch_system=… │
│ --env-script <str> Path to an │
│ environment │
│ setup script, │
│ sourced by the │
│ batch │
│ submission │
│ wrapper. Only │
│ takes effect │
│ combined with │
│ --batch; a │
│ no-op │
│ otherwise. │
│ --setting <str> Override a │
│ b2luigi │
│ setting as │
│ key=value │
│ (JSON-aware, │
│ repeatable), │
│ e.g. --setting │
│ apptainer_ima… │
│ (must be │
│ combined with │
│ --env-script). │
│ Applied only │
│ on the │
│ submission │
│ host and NEVER │
│ forwarded to │
│ the batch │
│ worker (unlike │
│ settings.json… │
│ use it for │
│ submission-si… │
│ settings, and │
│ settings.json │
│ for anything │
│ (e.g. │
│ result_dir, │
│ log_dir) both │
│ sides must │
│ agree on. │
│ batch_system │
│ can be set │
│ this way (same │
│ as │
│ --batch-syste… │
│ env_script │
│ cannot, since │
│ FastTask sets │
│ it as a class │
│ attribute │
│ which takes │
│ priority over │
│ --setting. │
│ --literal-pa… --no-literal-… Write -o to │
│ the literal │
│ path given, │
│ exactly as -i │
│ already does │
│ (the default). │
│ Use │
│ --no-literal-… │
│ to nest the │
│ output under │
│ result_dir/pa… │
│ dirs via │
│ add_to_output… │
│ instead. No-op │
│ with --force │
│ (no output() │
│ is declared │
│ either way). │
│ [default: │
│ literal-path] │
│ --executable <str> Command used │
│ to run the │
│ script, split │
│ on shell rules │
│ (e.g. 'basf2', │
│ 'apptainer │
│ exec img.sif │
│ basf2'). │
│ Defaults to │
│ the current │
│ Python │
│ interpreter. │
│ Needed for │
│ basf2 steering │
│ files, whose │
│ -o/-i are │
│ handled by the │
│ basf2 wrapper │
│ rather than by │
│ the script. │
│ When set, -- │
│ is inserted │
│ before any │
│ extra │
│ arguments so │
│ the script's │
│ own args are │
│ not consumed │
│ by the │
│ wrapper. │
│ Unrelated to │
│ the │
│ 'executable' │
│ setting, which │
│ launches the │
│ b2luigi batch │
│ worker. │
│ --help Show this │
│ message and │
│ exit. │
╰───────────────────────────────────────────────────────────────╯
tasks#
Usage: b2luigi tasks [OPTIONS] COMMAND [ARGS]...
List and inspect available task classes.
╭─ Options ─────────────────────────────────────────────────────╮
│ --task-file -f <str> Task definitions file (or │
│ $B2LUIGI_TASK_FILE) │
│ --help Show this message and exit. │
╰───────────────────────────────────────────────────────────────╯
╭─ Commands ────────────────────────────────────────────────────╮
│ info Show docstring and parameters for a task class, or all │
│ task classes if omitted. │
╰───────────────────────────────────────────────────────────────╯
info#
Usage: b2luigi tasks info [OPTIONS] [classname]
Show docstring and parameters for a task class, or all task
classes if omitted.
:param classname: The task class name, or omit to show all.
:type classname: Optional
:param task_filename: Path to the task definitions file.
:type task_filename: Optional
╭─ Arguments ───────────────────────────────────────────────────╮
│ classname <str> Task class name to show info for. │
│ Omit to show info for all tasks. │
╰───────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────╮
│ --task-file -f <str> Task definitions file (or │
│ $B2LUIGI_TASK_FILE) │
│ --help Show this message and exit. │
╰───────────────────────────────────────────────────────────────╯
graph#
Usage: b2luigi graph [OPTIONS] [classnames]... COMMAND
[ARGS]...
Render the task dependency graph as a tree (terminal) or DOT
(Graphviz).
╭─ Arguments ───────────────────────────────────────────────────╮
│ classnames <str> Task class name(s) to scope the │
│ graph to. Omit for the full graph. │
╰───────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────╮
│ --task-file -f <str> Task definitions file (or │
│ $B2LUIGI_TASK_FILE) │
│ --params-file -p <str> Parameters file (or │
│ $B2LUIGI_PARAMS_FILE) │
│ --param -P <str> Override task parameters │
│ (repeatable): key=value. Values │
│ are parsed as JSON when │
│ possible. │
│ --format <str> Output format: 'tree' (Rich │
│ terminal, default) or 'dot' │
│ (Graphviz DOT to stdout). │
│ [default: tree] │
│ --params Include parameter values on │
│ each node. │
│ --status -s Check and display output │
│ completion status on each node. │
│ --summary Print per-class completion │
│ counts instead of the │
│ dependency tree. Cannot be │
│ combined with --format dot or │
│ --params. │
│ --help Show this message and exit. │
╰───────────────────────────────────────────────────────────────╯
Parameter generator classes#
- class b2luigi.ParameterGenerator(values: list[Any])[source]#
Declare a list of values for a single parameter to be expanded via cartesian product.
Place as a value in the
parameters.pyconfigdict. MultipleParameterGeneratorvalues in the same config are crossed with each other (and with anyZippedParameterGeneratorgroups) to produce all combinations.Example:
from b2luigi import ParameterGenerator config = {"split": ParameterGenerator([1, 2, 3])}
- Parameters:
values (list[Any]) – Non-empty list of concrete parameter values.
- Raises:
CliUserError – If
valuesis empty.
- class b2luigi.ZippedParameterGenerator(**kwargs: list[Any])[source]#
Declare multiple parameters to be expanded in lockstep (zip).
Place as a value in the
parameters.pyconfigdict under any sentinel key (the key is ignored; parameter names come from keyword arguments). MultipleZippedParameterGeneratorinstances are cartesian-crossed with each other and with anyParameterGeneratorvalues.Example:
from b2luigi import ZippedParameterGenerator config = { "zipped": ZippedParameterGenerator(alpha=[1, 2], beta=["a", "b"]), } # expands to: (alpha=1, beta="a"), (alpha=2, beta="b")
- Parameters:
kwargs (dict[str, list]) – Each keyword argument names a task parameter; its value is the list of concrete values for that parameter. All lists must have the same length.
- Raises:
CliUserError – If no keyword arguments are given, list lengths differ, or all lists are empty.