b2luigi Features

Once you have your first task tree setup, (b2)luigi has many features, that make your live easier. From leveraging batch systems, setting up different environments, keeping track over many parameters and files, there are many features to keep in mind. In the following sections some of the important features are explained in more detail. Also checkout our Examples and b2luigi Starter Kit for some working examples.

• luigi Features
  • The Central Scheduler
  • Parameters
  • Logging
  • Even more settings
  • Resource Management
• Run Modes
  • Start a Central Scheduler
• Batch Processing
  • Choosing the batch system
  • Choosing the Environment
  • File System
  • Drawbacks of the batch mode
  • Batch System Specific Settings
  • Add your own batch system
• Parameters (Advanced)
  • Unique Task outputs
  • Exclude Parameters from the output name
  • Do not use the parameter name in the output name
  • Do not use “=” to separate the parameter name and value
  • Hashing List and Dict Parameters
• Handling Input and Outputs
  • Outputs
  • Inputs
• Settings
  • How to set settings
  • Available settings
  • Custom settings