Python structure

In this section we will describe the basic structure of the wrapped tool python files.

Template class

Structure description for the Template class of the biobb_template/biobb_template/template/template.py file. The complete source code is available in the biobb_template repository.

Imports

import argparse
import shutil
from pathlib import PurePath
from biobb_common.generic.biobb_object import BiobbObject
from biobb_common.configuration import settings
from biobb_common.tools import file_utils as fu
from biobb_common.tools.file_utils import launchlogger

Libraries used for the execution of the Template class.

Class

Class template inherits from BiobbObject parent class.

# 1. Rename class as required
class Template(BiobbObject):

Docstrings

The automatic documentation generation system through sphinx is explained in detail in the Documentation section.

After the definition of the Template class we define all the arguments explained in the Arguments section inside a set of triple quotes:

"""
| biobb_template Template
| Short description for the `template <http://templatedocumentation.org>`_ module in Restructured Text (reST) syntax. Mandatory.
| Long description for the `template <http://templatedocumentation.org>`_ module in Restructured Text (reST) syntax. Optional.

Args:        
    input_file_path1 (str): Description for the first input file path. File type: input. `Sample file <https://urlto.sample>`_. Accepted formats: top (edam:format_3881).
    input_file_path2 (str) (Optional): Description for the second input file path (optional). File type: input. `Sample file <https://urlto.sample>`_. Accepted formats: dcd (edam:format_3878).
    output_file_path (str): Description for the output file path. File type: output. `Sample file <https://urlto.sample>`_. Accepted formats: zip (edam:format_3987).
    properties (dic):
        * **boolean_property** (*bool*) - (True) Example of boolean property.
        * **binary_path** (*str*) - ("zip") Example of executable binary property.
        * **remove_tmp** (*bool*) - (True) [WF property] Remove temporal files.
        * **restart** (*bool*) - (False) [WF property] Do not execute if output files exist.

Examples:
    This is a use example of how to use the building block from Python::

        from biobb_template.template.template import template

        prop = { 
            'boolean_property': True 
        }
        template(input_file_path1='/path/to/myTopology.top',
                output_file_path='/path/to/newCompressedFile.zip',
                input_file_path2='/path/to/mytrajectory.dcd',
                properties=prop)

Info:
    * wrapped_software:
        * name: Zip
        * version: >=3.0
        * license: BSD 3-Clause
    * ontology:
        * name: EDAM
        * schema: http://edamontology.org/EDAM.owl

"""

The docstrings syntax is broadly explained in the JSON Generator help of the utils_biobb repository. Following accurately this syntax, the JSON Generator Tool will automatically generate the JSON Schemas of the package.

All the multiline comments inside a set of triple quotes are used later as the function definition:

def launch(self) -> int:
    """Execute the :class:`Template <template.template.Template>` object."""

main() function

In the main() function we define the command line inputs and outputs and then we pass them to the launch function (described below) of the Template class.

def main():
    """Command line execution of this building block. Please check the command line documentation."""
    parser = argparse.ArgumentParser(description='Description for the template module.', formatter_class=lambda prog: argparse.RawTextHelpFormatter(prog, width=99999))
    parser.add_argument('--config', required=False, help='Configuration file')

    # 10. Include specific args of each building block following the examples. They should match step 2
    required_args = parser.add_argument_group('required arguments')
    required_args.add_argument('--input_file_path1', required=True, help='Description for the first input file path. Accepted formats: top.')
    parser.add_argument('--input_file_path2', required=False, help='Description for the second input file path (optional). Accepted formats: dcd.')
    required_args.add_argument('--output_file_path', required=True, help='Description for the output file path. Accepted formats: zip.')

    args = parser.parse_args()
    args.config = args.config or "{}"
    properties = settings.ConfReader(config=args.config).get_prop_dic()

    # 11. Adapt to match Class constructor (step 2)
    # Specific call of each building block
    template(input_file_path1=args.input_file_path1,
             output_file_path=args.output_file_path,
             input_file_path2=args.input_file_path2,
             properties=properties)

__init__() function

In the __init__() function initialises the Template class. In this function a dictionary with all the inputs and output (self.io_dict) is created, and all the properties are initialised.

# 2. Adapt input and output file paths as required. Include all files, even optional ones
def __init__(self, input_file_path1, output_file_path, input_file_path2=None, properties=None, **kwargs) -> None:
    properties = properties or {}

    # 2.0 Call parent class constructor
    super().__init__(properties)
    self.locals_var_dict = locals().copy()

    # 2.1 Modify to match constructor parameters
    # Input/Output files
    self.io_dict = {
        'in': {'input_file_path1': input_file_path1, 'input_file_path2': input_file_path2},
        'out': {'output_file_path': output_file_path}
    }

    # 3. Include all relevant properties here as
    # self.property_name = properties.get('property_name', property_default_value)

    # Properties specific for BB
    self.boolean_property = properties.get('boolean_property', True)
    self.binary_path = properties.get('binary_path', 'zip')
    self.properties = properties

    # Check the properties
    self.check_properties(properties)
    # Check the arguments
    self.check_arguments()

launch() function

In the function launch() we perform all the actions needed for the wrapping: creation of temporary folder(s), creation of command line, execution of command line and removing of the temporary folder(s). In this case we will comment all the blocks that shape this function separately.

@launchlogger decorator

Decorator used for wrapping the log.

@launchlogger
def launch(self) -> int:
    """Execute the :class:`Template <template.template.Template>` object."""

Setup BioBB

If restart property is enabled, skip this step. This property is only used for workflow purposes. Then, all files are staged.

# 4. Setup Biobb
if self.check_restart():
    return 0
self.stage_files()

Temporary folder

Creation of a temporary folder and copy the required input file inside it.

# Creating temporary folder
self.tmp_folder = fu.create_unique_dir()
fu.log('Creating %s temporary folder' % self.tmp_folder, self.out_log)

# 5. Include here all mandatory input files
# Copy input_file_path1 to temporary folder
shutil.copy(self.io_dict['in']['input_file_path1'], self.tmp_folder)

Command line

Creation of command line call. If boolean_property is enabled, append -v option to command line.

# 6. Prepare the command line parameters as instructions list
instructions = ['-j']
if self.boolean_property:
    instructions.append('-v')
    fu.log('Appending optional boolean property', self.out_log, self.global_log)

# 7. Build the actual command line as a list of items (elements order will be maintained)
self.cmd = [self.binary_path,
            ' '.join(instructions),
            self.io_dict['out']['output_file_path'],
            str(PurePath(self.tmp_folder).joinpath(PurePath(self.io_dict['in']['input_file_path1']).name))]
fu.log('Creating command line with instructions and required arguments', self.out_log, self.global_log)

Optional input file

If optional input file provided, copy it to the temporary folder and append it to the command line call.

# 8. Repeat for optional input files if provided
if self.io_dict['in']['input_file_path2']:
    # Copy input_file_path2 to temporary folder
    shutil.copy(self.io_dict['in']['input_file_path2'], self.tmp_folder)
    # Append optional input_file_path2 to cmd
    self.cmd.append(str(PurePath(self.tmp_folder).joinpath(PurePath(self.io_dict['in']['input_file_path2']).name)))
    fu.log('Appending optional argument to command line', self.out_log, self.global_log)

Launch execution

# Run Biobb block
self.run_biobb()

Remove temporary file(s)

Remove temporary file(s) created during the execution.

# Remove temporary file(s)
self.tmp_files.extend([
    self.stage_io_dict.get("unique_dir"),
    self.tmp_folder
])
self.remove_tmp_files()

Check arguments

Check output arguments and warn the user if some output is incorrect or not created.

# Check output arguments
self.check_arguments(output_files_created=True, raise_exception=False)

Return code

Finally, return self.return_code and finish the function.

return self.return_code

template() function

In the function template() we call the launch() function of the class Template. It will be used for external calls to the Template class such as Jupyter Notebooks and some adapters.

def template(input_file_path1: str, output_file_path: str, input_file_path2: str = None, properties: dict = None, **kwargs) -> int:
"""Create :class:`Template <template.template.Template>` class and
execute the :meth:`launch() <template.template.Template.launch>` method."""

return Template(input_file_path1=input_file_path1,
                output_file_path=output_file_path,
                input_file_path2=input_file_path2,
                properties=properties, **kwargs).launch()

TemplateContainer class

Structure description for the TemplateContainer class of the biobb_template/biobb_template/template/template_container.py file. The complete source code is available in the biobb_template repository.

Imports

import argparse
from biobb_common.generic.biobb_object import BiobbObject
from biobb_common.configuration import settings
from biobb_common.tools import file_utils as fu
from biobb_common.tools.file_utils import launchlogger

Libraries used for the execution of the TemplateContainer class.

Class

Class template inherits from BiobbObject parent class.

# 1. Rename class as required
class TemplateContainer(BiobbObject):

Docstrings

The automatic documentation generation system through sphinx is explained in detail in the Documentation section.

In the first line of the Template class we define all the arguments explained in the Arguments section inside a set of triple quotes:

"""
| biobb_template TemplateContainer
| Short description for the `template container <http://templatedocumentation.org>`_ module in Restructured Text (reST) syntax. Mandatory.
| Long description for the `template container <http://templatedocumentation.org>`_ module in Restructured Text (reST) syntax. Optional.

Args:
    input_file_path1 (str): Description for the first input file path. File type: input. `Sample file <https://urlto.sample>`_. Accepted formats: top (edam:format_3881).
    input_file_path2 (str) (Optional): Description for the second input file path (optional). File type: input. `Sample file <https://urlto.sample>`_. Accepted formats: dcd (edam:format_3878).
    output_file_path (str): Description for the output file path. File type: output. `Sample file <https://urlto.sample>`_. Accepted formats: zip (edam:format_3987).
    properties (dic):
        * **boolean_property** (*bool*) - (True) Example of boolean property.
        * **binary_path** (*str*) - ("zip") Example of executable binary property.
        * **remove_tmp** (*bool*) - (True) [WF property] Remove temporal files.
        * **restart** (*bool*) - (False) [WF property] Do not execute if output files exist.
        * **container_path** (*str*) - (None) Container path definition.
        * **container_image** (*str*) - ('mmbirb/zip:latest') Container image definition.
        * **container_volume_path** (*str*) - ('/tmp') Container volume path definition.
        * **container_working_dir** (*str*) - (None) Container working directory definition.
        * **container_user_id** (*str*) - (None) Container user_id definition.
        * **container_shell_path** (*str*) - ('/bin/bash') Path to default shell inside the container.

Examples:
    This is a use example of how to use the building block from Python::

        from biobb_template.template.template_container import template_container

        prop = { 
            'boolean_property': True,
            'container_path': 'docker',
            'container_image': 'mmbirb/zip:latest',
            'container_volume_path': '/tmp'
        }
        template_container(input_file_path1='/path/to/myTopology.top',
                        output_file_path='/path/to/newCompressedFile.zip',
                        input_file_path2='/path/to/mytrajectory.dcd',
                        properties=prop)

Info:
    * wrapped_software:
        * name: Zip
        * version: >=3.0
        * license: BSD 3-Clause
    * ontology:
        * name: EDAM
        * schema: http://edamontology.org/EDAM.owl

"""

The docstrings syntax is broadly explained in the JSON Generator help of the utils_biobb repository. Following accurately this syntax, the JSON Generator Tool will automatically generate the JSON Schemas of the package.

All the multiline comments inside a set of triple quotes are used later as the function definition:

def launch(self) -> int:
    """Execute the :class:`TemplateContainer <template.template_container.TemplateContainer>` object."""

main() function

In the main() function we define the command line inputs and outputs and then we pass them to the launch function (described below) of the TemplateContainer class.

def main():
    """Command line execution of this building block. Please check the command line documentation."""
    parser = argparse.ArgumentParser(description='Description for the template container module.', formatter_class=lambda prog: argparse.RawTextHelpFormatter(prog, width=99999))
    parser.add_argument('--config', required=False, help='Configuration file')

    # 10. Include specific args of each building block following the examples. They should match step 2
    required_args = parser.add_argument_group('required arguments')
    required_args.add_argument('--input_file_path1', required=True, help='Description for the first input file path. Accepted formats: top.')
    parser.add_argument('--input_file_path2', required=False, help='Description for the second input file path (optional). Accepted formats: dcd.')
    required_args.add_argument('--output_file_path', required=True, help='Description for the output file path. Accepted formats: zip.')

    args = parser.parse_args()
    args.config = args.config or "{}"
    properties = settings.ConfReader(config=args.config).get_prop_dic()

    # 11. Adapt to match Class constructor (step 2)
    # Specific call of each building block
    template_container(input_file_path1=args.input_file_path1,
                       output_file_path=args.output_file_path,
                       input_file_path2=args.input_file_path2,
                       properties=properties)

__init__() function

In the __init__() function initialises the TemplateContainer class. In this function a dictionary with all the inputs and output (self.io_dict) is created, and all the properties are initialised.

# 2. Adapt input and output file paths as required. Include all files, even optional ones
def __init__(self, input_file_path1, output_file_path, input_file_path2=None, properties=None, **kwargs) -> None:
    properties = properties or {}

    # 2.0 Call parent class constructor
    super().__init__(properties)
    self.locals_var_dict = locals().copy()

    # 2.1 Modify to match constructor parameters
    # Input/Output files
    self.io_dict = {
        'in': {'input_file_path1': input_file_path1, 'input_file_path2': input_file_path2},
        'out': {'output_file_path': output_file_path}
    }

    # 3. Include all relevant properties here as
    # self.property_name = properties.get('property_name', property_default_value)

    # Properties specific for BB
    self.boolean_property = properties.get('boolean_property', True)
    self.binary_path = properties.get('binary_path', 'zip')
    self.properties = properties

    # Check the properties
    self.check_properties(properties)
    # Check the arguments
    self.check_arguments()

launch() function

In the function launch() we perform all the actions needed for the wrapping: creation of temporary folder(s), mapping of these temporary folder(s) to the container, creation of command line, execution of command line, retrieve of data from the container and removing of the temporary folder(s). In this case we will comment all the blocks that shape this function separately.

@launchlogger decorator

Decorator used for wrapping the log.

@launchlogger
def launch(self) -> int:
    """Execute the :class:`TemplateContainer <template.template_container.TemplateContainer>` object."""

Setup BioBB

If restart property is enabled, skip this step. This property is only used for workflow purposes. Then, all files are staged.

# 4. Setup Biobb
if self.check_restart():
    return 0
self.stage_files()

Command line

Creation of command line call. If boolean_property is enabled, append -v option to command line.

# 5. Prepare the command line parameters as instructions list
instructions = ['-j']
if self.boolean_property:
    instructions.append('-v')
    fu.log('Appending optional boolean property', self.out_log, self.global_log)

# 6. Build the actual command line as a list of items (elements order will be maintained)
self.cmd = [self.binary_path,
            ' '.join(instructions),
            self.stage_io_dict['out']['output_file_path'],
            self.stage_io_dict['in']['input_file_path1']]
fu.log('Creating command line with instructions and required arguments', self.out_log, self.global_log)

Optional input file

If optional input file provided append it to the command line call.

# 7. Repeat for optional input files if provided
if self.stage_io_dict['in']['input_file_path2']:
    # Append optional input_file_path2 to cmd
    self.cmd.append(self.stage_io_dict['in']['input_file_path2'])
    fu.log('Appending optional argument to command line', self.out_log, self.global_log)

Launch execution

Creation of command line according to the container_path property and the rest of container-related properties. Then, launch execution.

# Run Biobb block
self.run_biobb()

Output data retrieval

Copy output file(s) from the mapped container_volume_path inside the container to the definitive output path defined by the user in the arguments.

# Copy files to host
self.copy_to_host()

Remove temporary file(s)

Remove temporary file(s) created during the execution.

# Remove temporary file(s)
self.tmp_files.extend([
    self.stage_io_dict.get("unique_dir"),
    self.tmp_folder
])
self.remove_tmp_files()

Check arguments

Check output arguments and warn the user if some output is incorrect or not created.

# Check output arguments
self.check_arguments(output_files_created=True, raise_exception=False)

Return code

Finally, return self.return_code and finish the function.

return self.return_code

template_container() function

In the function template_container() we call the launch() function of the class TemplateContainer. It will be used for external calls to the Template class such as Jupyter Notebooks and some adapters.

def template_container(input_file_path1: str, output_file_path: str, input_file_path2: str = None, properties: dict = None, **kwargs) -> int:
"""Create :class:`TemplateContainer <template.template_container.TemplateContainer>` class and
execute the :meth:`launch() <template.template_container.TemplateContainer.launch>` method."""

return TemplateContainer(input_file_path1=input_file_path1,
                            output_file_path=output_file_path,
                            input_file_path2=input_file_path2,
                            properties=properties, **kwargs).launch()