Parameters

As b2luigi automatically also imports luigi, you can use all the parameters from luigi you know and love. We have just added a single new flag called hashed to the parameters constructor. Turning it to true (it is turned off by default) will make b2luigi use a hashed version of the parameters value, when constructing output or log file paths. This is especially useful if you have parameters, which may include “dangerous” characters, like “/” or “{” (e.g. when using list or dictionary parameters). If you want to exclude certain parameters from the creation of the directory structure , you can use the significant flag and set it to False.

See also one of our FAQ.

For more information on luigi.Parameters and what types there are see the Luigi Documentation

b2luigi.wrap_parameter()

Monkey patch the parameter base class (and with it all other parameters( of luigi to include two additional parameters in its constructor: hashed and hash_function.

Enabling the hashed parameter will use a hashed version of the parameter value when creating file paths our of the parameters of a task instead of the value itself. By default an md5 hash is used. A custom hash function can be provided via the hash_function parameter. This function should take one input, the value of the parameter. It is up to the user to ensure a unique string is created from the input.

This is especially useful when you have list, string or dict parameters, where the resulting file path may include “/” or “{}”.