Welcome to pytest-wdl’s documentation!

User manual

pytest-wdl is a plugin for the pytest unit testing framework that enables testing of workflows written in Workflow Description Language. Test workflow inputs and expected outputs are configured in a test_data.json file. Workflows are run by one or more executors. By default, actual and expected outputs are compared by MD5 hash, but data type-specific comparisons are provided. Data types and executors are pluggable and can be provided via third-party packages.

Fixtures

All functionality of pytest-wdl is provided via fixtures. As long as pytest-wdl is in your PYTHONPATH, its fixtures will be discovered and made available when you run pytest.

The two most important fixtures are:

• workflow_data: Provides access to data files for use as inputs to a workflow, and for comparing to workflow output.

• workflow_runner: Given a WDL workflow, inputs, and expected outputs, runs the workflow using one or more executors and compares actual and expected outputs.

There are also several additional fixtures used for configuration of the two main fixtures. In most cases, the default values returned by these fixtures “just work.” However, if you need to override the defaults, you may do so either directly within your test modules, or in a conftest.py file.

Test data

Typically, workflows require inputs and generate outputs. Beyond simply ensuring that a workflow runs successfully, we often want to additionally test that it reproducibly generates the same results given the same inputs.

Test inputs and outputs are configured in a test_data.json file that is stored in the same directory as the test module. This file has one entry for each input/output. Primitive types map as expected from JSON to Python to WDL. For example, the following test_data.json file defines an integer input that is loaded as a Python int and then maps to the WDL Integer type when passed as an input parameter to a workflow:

{
  "input_int": 42
}

Files

For file inputs and outputs, pytest-wdl offers several different options. Test data files may be located remotely (identified by a URL), located within the test directory (using the folder hierarchy established by the datadir-ng plugin), located at an arbitrary local path, or defined by specifying the file contents directly within the JSON file. Files that do not already exist locally are localized on-demand and stored in the cache directory.

Some additional options are available only for expected outputs, in order to specify how they should be compared to the actual outputs.

Below is an example test_data.json file that demonstrates different ways to define input and output data files:

{
  "bam": {
    "url": "http://example.com/my.bam",
    "http_headers": {
      "auth_token": "TOKEN"
    }
  },
  "reference": {
    "path": "${REFERENCE_DIR}/chr22.fa"
  },
  "sample": {
    "path": "samples.vcf",
    "contents": "sample1\nsample2"
  },
  "output_vcf": {
    "name": "output.vcf",
    "type": "vcf",
    "allowed_diff_lines": 2
  }
}

The available keys for configuring file inputs/outputs are:

• name: Filename to use when localizing the file; when none of url, path, or contents are defined, name is also used to search for the data file within the tests directory, using the same directory structure defined by the datadir-ng fixture.

• path: The local path to the file. If the path does not already exist, the file will be localized to this path. Typically, this is defined as a relative path that will be prefixed with the cache directory path. Environment variables can be used to enable the user to configure an environment-specific path.

• env: The name of an environment variable in which to look up the local path of the file.

• url: A URL that can be resolved by urllib.

  • http_headers: Optional dict mapping header names to values. These headers are used for file download requests. Keys are header names and values are either strings (environment variable name) or mappings with the following keys:

    • env: The name of an environment variable in which to look up the header value.

    • value: The header value; only used if an environment variable is not specified or is unset.

• contents: The contents of the file, specified as a string. The file is written to path the first time it is requested.

In addition, the following keys are recognized for output files only:

• type: The file type. This is optional and only needs to be provided for certain types of files that are handled specially for the sake of comparison.

• allowed_diff_lines: Optional and only used for outputs comparison. If ‘0’ or not specified, it is assumed that the expected and actual outputs are identical.

Data Types

When comparing actual and expected outputs, the “type” of the expected output is used to determine how the files are compared. If no type is specified, then the type is assumed to be “default.”

Available types:

• default: The default type if one is not specified.

  • It can handle raw text files, as well as gzip compressed files.

  • If allowed_diff_lines is 0 or not specified, then the files are compared by their MD5 hashes.

  • If allowed_diff_lines is > 0, the files are converted to text and compared using the linux diff tool.

• vcf: During comparison, headers are ignored, as are the QUAL, INFO, and FORMAT columns; for sample columns, only the first sample column is compared between files, and only the genotype values for that sample.

• bam*:

  • BAM is converted to SAM and headers are ignored.

  • Replaces random UNSET-\w*\b type IDs that samtools often adds.

* requires extra dependencies to be installed, see Installing Data Type Plugins

Executors

An Executor is a wrapper around a WDL workflow execution engine that prepares inputs, runs the tool, captures outputs, and handles errors. Currently, Cromwell is the only supported executor, but aternative executors can be implemented as plugins.

The workflow_runner fixture is a callable that runs the workflow using the executor. It takes one required arguments and several additional optional arguments:

• wdl_script: Required; the WDL script to execute. The path must either be absolute or (more commonly) relative to the project root.

• workflow_name: The name of the workflow to execute in the WDL script. If not specified, it is assumed that the workflow has the same name as the WDL file (without the “.wdl” extension).

• inputs: Dict that will be serialized to JSON and provided to Cromwell as the workflow inputs. If not specified, the workflow must not have any required inputs.

• expected: Dict mapping output parameter names to expected values. Any workflow outputs that are not specified are ignored. This is an optional parameter and can be omitted if, for example, you only want to test that the workflow completes successfully.

Executor-specific options

Cromwell

• inputs_file: Specify the inputs.json file to use, or the path to the inputs.json file to write, instead of a temp file.

• imports_file: Specify the imports file to use, or the path to the imports zip file to write, instead of a temp file.

• java_args: Override the default Java arguments.

• cromwell_args: Override the default Cromwell arguments.

Configuration

pytest-wdl has two levels of configuration:

• Project-specific configuration, which generally deals with the structure of the project, and may require customization if the structure of your project differs substantially from what is expected, but also encompases executor-specific configuration.

• Environment-specific configuration, which generally deals with idiosyncrasies of the local environment.

Project-specific configuration

Configuration at the project level is handled by overriding fixtures, either in the test module or in a top-level conftest.py file. The following fixtures may be overridden:

fixture name	scope	description	default
project_root_files	module	List of filenames that are found in the project root directory.	["setup.py", "pyproject.toml", ".git"]
project_root	module	The root directory of the project. Relative paths are relative to this directory, unless specified otherwise.	Starting in the current test directory/module, scan up the directory hierarchy until one of the project_root_files are located.
workflow_data_descriptor_file	module	Path to the JSON file that describes the test data.	tests/test_data.json
workflow_data_descriptors	module	Mapping of workflow input/output names to values (as described in the Files section).	Loaded from the workflow_data_descriptor_file
workflow_data_resolver	module	Provides the DataResolver object that resolves test data; this should only need to be overridden for testing/debugging purposes	DataResolver created from workflow_data_descriptors
import_paths	module	Provides the path to the file that lists the directories from which to import WDL dependencies	"import_paths.txt"
import_dirs	module	Provides the directories from which to import WDL dependencies	Loaded from import_paths file, if any

Environment-specific configuration

There are several aspects of pytest-wdl that can be configured to the local environment, for example to enable the same tests to run both on a user’s development machine and in a continuous integration environment.

Environment-specific configuration is specified either or both of two places: a JSON configuration file and environment variables. Environment variables always take precendence over values in the configuration file. Keep in mind that (on a *nix system), environment variables can be set (semi-)permanently (using export) or temporarily (using env):

# Set environment variable durably
$ export FOO=bar

# Set environment variable only in the context of a single command
$ env FOO=bar echo "foo is $FOO"

Configuration file

The pytest-wdl configuration file is a JSON-format file. Its default location is $HOME/pytest_wdl_config.json. Here is an example.

The available configuration options are listed in the following table:

configuration file key	environment variable	description	default	recommendation
cache_dir	PYTEST_WDL_CACHE_DIR	Directory to use for localizing test data files.	Temporary directory; a separate directory is used for each test module	pro: saves time when multiple tests rely on the same test data files; con: can cause conflicts, if tests use different files with the same name
execution_dir	PYTEST_WDL_EXECUTION_DIR	Directory in which tests are executed	Temporary directory; a separate directory is used for each test function	Only use for debugging; use an absolute path
proxies	Configurable	Proxy server information; see details below	None	Use environment variable(s) to configure your proxy server(s), if any
http_headers	Configurable	HTTP header configuration that applies to all URLs matching a given pattern; see details below	None	Configure headers by URL pattern; configure headers for specific URLs in the test_data.json file
show_progress	N/A	Whether to show progress bars when downloading files	False
executors	Executor-dependent	Configuration options specific to each executor; see below	None
N/A	LOGLEVEL	Level of detail to log; can set to 'DEBUG', 'INFO', 'WARNING', or 'ERROR'	'WARNING'	Use 'DEBUG' when developing plugins/fixtures/etc., otherwise 'WARNING'

Proxies

In the proxies section of the configuration file, you can define the proxy servers for schemes used in data file URLs. The keys are scheme names and the values are either strings - environment variable names - or mappings with the following keys:

• env: The name of an environment variable in which to look for the proxy server address.

• value: The value to use for the proxy server address, if the environment variable is not defined or is unset.

{
  "proxies": {
    "http": {
      "env": "HTTP_PROXY"
    },
    "https": {
      "value": "https://foo.com/proxy",
      "env": "HTTPS_PROXY"
    }
  }
}

HTTP(S) Headers

In the http_headers section of the configuration file, you can define a list of headers to use when downloading data files. In addition to env and value keys (which are interpreted the same as for proxies, two additional keys are allowed:

• name: Required; the header name

• pattern: A regular expression used to match the URL; if not specified, the header is used with all URLs.

{
  "http_headers": [
    {
      "name": "X-JFrog-Art-Api",
      "pattern": "http://my.company.com/artifactory/*",
      "env": "TOKEN"
    }
  ]
}

Executor-specific configuration

Cromwell

configuration file key	environment variable	description	default
java_bin	JAVA_HOME	Path to java executable; If not specified, then Java executable is expected to be in $JAVA_HOME/bin/java	None
java_args	JAVA_ARGS	Arguments to add to the java command	-Dconfig.file=<cromwell_config_file> (if cromwell_config_file is specified
cromwell_jar	CROMWELL_JAR	Path to Cromwell JAR file	None
N/A	CLASSPATH	Java classpath; searched for a file matching "cromwell*.jar" if cromwell_jar is not specified	None
cromwell_config_file	CROMWELL_CONFIG	Path to Cromwell configuration file	None
cromwell_args	CROMWELL_ARGS	Arguments to add to the cromwell run command	None; recommended to use -Ddocker.hash-lookup.enabled=false to disable Docker lookup by hash

Fixtures

There are two fixtures that control the loading of the user configuration:

fixture name	scope	description	default
user_config_file	session	The location of the user configuration file	The value of the PYTEST_WDL_CONFIG environment variable if set, otherwise $HOME/pytest_wdl_config.json
user_config	session	Provides a UserConfiguration object that is used by other fixtures to access configuration values	Default values are loaded from user_config_file, but most values can be overridden via environment variables (see below)

Plugins

pytest-wdl provides the ability to implement 3rd-party plugins for data types and executors. When two plugins with the same name are present, the third-party plugin takes precedence over the built-in plugin (however, if there are two conflicting third-party plugins, an exception is raised).

Creating new data types

To create a new data type plugin, add a module in the data_types package of pytest-wdl, or create it in your own 3rd party package.

Your plugin should subclass the pytest_wdl.core.DataFile class and override its methods for _assert_contents_equal() and/or _diff() to define the behavior for this file type.

Next, add an entry point in setup.py. If the data type requires more dependencies to be installed, make sure to use a try/except ImportError to warn about this and add the extra dependencies under the setup.py’s extras_require. For example:

# plugin.py
try:
    import mylib
except ImportError:
    logger.warning(
        "mytype is not available because the mylib library is not "
        "installed"
    )

setup(
    ...,
    entry_points={
        "pytest_wdl.data_types": [
            "mydata = pytest_wdl.data_types.mytype:MyDataFile"
        ]
    },
    extras_require={
        "mydata": ["mylib"]
    }
)

In this example, the extra dependencies can be installed with pip install pytest-wdl[mydata].

Creating new executors

To create a new executor, add a module in the executors package, or in your own 3rd party package.

Your plugin should subclass pytest_wdl.core.Executor and implement the run_workflow() method.

Next, add an entry point in setup.py. If the data type requires more dependencies to be installed, make sure to use a try/except ImportError to warn about this and add the extra dependencies under the setup.py’s extras_require (see example under Creating new data types). For example:

setup(
    ...,
    entry_points={
        "pytest_wdl.executors": [
            "myexec = pytest_wdl.executors.myexec:MyExecutor"
        ]
    },
    extras_require={
        "myexec": ["mylib"]
    }
)

pytest_wdl

pytest_wdl package

Subpackages

pytest_wdl.data_types package

Submodules

pytest_wdl.data_types.bam module

Convert BAM to SAM for diff.

class pytest_wdl.data_types.bam.BamDataFile(local_path: pathlib.Path, localizer: Optional[pytest_wdl.core.Localizer] = None, allowed_diff_lines: Optional[int] = 0)

Bases: pytest_wdl.core.DataFile

Supports comparing output of BAM file. This uses pysam to convert BAM to SAM, so that DataFile can carry out a regular diff on the SAM files.

pytest_wdl.data_types.vcf module

Some tools that generate VCF (callers) will result in very slightly different qual scores and other floating-point-valued fields when run on different hardware. This handler ignores the QUAL and INFO columns and only compares the genotype (GT) field of sample columns. Only works for single-sample VCFs.

class pytest_wdl.data_types.vcf.VcfDataFile(local_path: pathlib.Path, localizer: Optional[pytest_wdl.core.Localizer] = None, allowed_diff_lines: Optional[int] = 0)

Bases: pytest_wdl.core.DataFile

Module contents

pytest_wdl.executors package

Submodules

pytest_wdl.executors.cromwell module

class pytest_wdl.executors.cromwell.CromwellExecutor(project_root: pathlib.Path, import_dirs: Optional[List[pathlib.Path]] = None, java_bin: Union[str, pathlib.Path, None] = None, java_args: Optional[str] = None, cromwell_jar_file: Union[str, pathlib.Path, None] = None, cromwell_config_file: Union[str, pathlib.Path, None] = None, cromwell_args: Optional[str] = None)

Bases: pytest_wdl.core.Executor

Manages the running of WDL workflows using Cromwell.

Parameters

• project_root – The root path to which non-absolute WDL script paths are relative.

• import_dirs – Relative or absolute paths to directories containing WDL scripts that should be available as imports.

• java_bin – Path to the java executable.

• java_args – Default Java arguments to use; can be overidden by passing java_args=… to run_workflow.

• cromwell_jar_file – Path to the Cromwell JAR file.

• cromwell_args – Default Cromwell arguments to use; can be overridden by passing cromwell_args=… to run_workflow.

static get_cromwell_outputs(output)

run_workflow(wdl_script: Union[str, pathlib.Path], workflow_name: Optional[str] = None, inputs: Optional[dict] = None, expected: Optional[dict] = None, **kwargs) → dict

Run a WDL workflow on given inputs, and check that the output matches given expected values.

Parameters

• wdl_script – The WDL script to execute.

• workflow_name – The name of the workflow in the WDL script. If None, the name of the WDL script is used (without the .wdl extension).

• inputs – Object that will be serialized to JSON and provided to Cromwell as the workflow inputs.

• expected – Dict mapping output parameter names to expected values.

• kwargs –

  Additional keyword arguments, mostly for debugging: * inputs_file: Path to the Cromwell inputs file to use. Inputs are

  written to this file only if it doesn’t exist.

  • imports_file: Path to the WDL imports file to use. Imports are

    written to this file only if it doesn’t exist.

  • java_args: Additional arguments to pass to Java runtime.

  • cromwell_args: Additional arguments to pass to cromwell run.

Returns

Dict of outputs.

Raises

• Exception – if there was an error executing Cromwell

• AssertionError – if the actual outputs don’t match the expected outputs

Module contents

pytest_wdl.executors.get_workflow(project_root: pathlib.Path, wdl_file: Union[str, pathlib.Path], workflow_name: Optional[str] = None) → Tuple[pathlib.Path, str]

Resolve the WDL file and workflow name.

TODO: if workflow_name is None, parse the WDL file and extract the name

of the workflow.

Parameters

• project_root – The root directory to which wdl_file might be relative.

• wdl_file – Path to the WDL file.

• workflow_name – The workflow name; if None, the filename without “.wdl” extension is used.

Returns

A tuple (wdl_path, workflow_name)

pytest_wdl.executors.get_workflow_imports(import_dirs: Optional[List[pathlib.Path]] = None, imports_file: Optional[pathlib.Path] = None) → pathlib.Path

Creates a ZIP file with all WDL files to be imported.

Parameters

• import_dirs – Directories from which to import WDL files.

• imports_file – Text file naming import directories/files - one per line.

Returns

Path to the ZIP file.

pytest_wdl.executors.get_workflow_inputs(workflow_name: str, inputs_dict: Optional[dict] = None, inputs_file: Optional[pathlib.Path] = None) → Tuple[dict, pathlib.Path]

Persist workflow inputs to a file, or load workflow inputs from a file.

Parameters

• workflow_name – Name of the workflow; used to prefix the input parameters when creating the inputs file from the inputs dict.

• inputs_dict – Dict of input names/values.

• inputs_file – JSON file with workflow inputs.

Returns

A tuple (inputs_dict, inputs_file)

Submodules

pytest_wdl.core module

pytest_wdl.core.DATA_TYPES = {'bam': <pytest_wdl.utils.PluginFactory object>, 'vcf': <pytest_wdl.utils.PluginFactory object>}

Data type plugin modules from the discovered entry points.

class pytest_wdl.core.DataDirs(basedir: pathlib.Path, module, function: Callable, cls: Optional[Type] = None)

Bases: object

Provides data files from test data directory structure as defined by the datadir and datadir-ng plugins. Paths are resolved lazily upon first request.

property paths

class pytest_wdl.core.DataFile(local_path: pathlib.Path, localizer: Optional[pytest_wdl.core.Localizer] = None, allowed_diff_lines: Optional[int] = 0)

Bases: object

A data file, which may be local, remote, or represented as a string.

Parameters

• local_path – Path where the data file should exist after being localized.

• localizer – Localizer object, for persisting the file on the local disk.

• allowed_diff_lines – Number of lines by which the file is allowed to differ from another and still be considered equal.

assert_contents_equal(other: Union[str, pathlib.Path, DataFile]) → None

Assert the contents of two files are equal.

If allowed_diff_lines == 0, files are compared using MD5 hashes, otherwise their contents are compared using the linux diff command.

Parameters

other – A DataFile or string file path.

Raises

AssertionError if the files are different. –

property path

class pytest_wdl.core.DataManager(data_resolver: pytest_wdl.core.DataResolver, datadirs: pytest_wdl.core.DataDirs)

Bases: object

Manages test data, which is defined in a test_data.json file.

Parameters

• data_resolver – Module-level config.

• datadirs – Data directories to search for the data file.

get_dict(*names: str, **params) → dict

Creates a dict with one or more entries from this DataManager.

Parameters

• *names – Names of test data entries to add to the dict.

• **params – Mapping of workflow parameter names to test data entry names.

Returns

Dict mapping parameter names to test data entries for all specified names.

class pytest_wdl.core.DataResolver(data_descriptors: dict, user_config: pytest_wdl.core.UserConfiguration)

Bases: object

Resolves data files that may need to be localized.

create_data_file(type: Optional[str] = 'default', name: Optional[str] = None, path: Optional[str] = None, url: Optional[str] = None, contents: Optional[str] = None, env: Optional[str] = None, datadirs: Optional[pytest_wdl.core.DataDirs] = None, http_headers: Optional[dict] = None, **kwargs) → pytest_wdl.core.DataFile

resolve(name: str, datadirs: Optional[pytest_wdl.core.DataDirs] = None) → pytest_wdl.core.DataFile

pytest_wdl.core.EXECUTORS = {'cromwell': <pytest_wdl.utils.PluginFactory object>}

Executor plugin modules from the discovered entry points.

class pytest_wdl.core.Executor

Bases: object

Base class for WDL workflow executors.

abstract run_workflow(wdl_script: Union[str, pathlib.Path], workflow_name: Optional[str] = None, inputs: Optional[dict] = None, expected: Optional[dict] = None, **kwargs) → dict

Run a WDL workflow on given inputs, and check that the output matches given expected values.

Parameters

• wdl_script – The WDL script to execute.

• workflow_name – The name of the workflow in the WDL script. If None, the name of the WDL script is used (without the .wdl extension).

• inputs – Object that will be serialized to JSON and provided to Cromwell as the workflow inputs.

• expected – Dict mapping output parameter names to expected values.

• kwargs –

  Additional keyword arguments, mostly for debugging: * execution_dir: DEPRECATED * inputs_file: Path to the Cromwell inputs file to use. Inputs are

  written to this file only if it doesn’t exist.

  • imports_file: Path to the WDL imports file to use. Imports are

    written to this file only if it doesn’t exist.

  • java_args: Additional arguments to pass to Java runtime.

  • cromwell_args: Additional arguments to pass to cromwell run.

Returns

Dict of outputs.

Raises

• Exception – if there was an error executing the workflow

• AssertionError – if the actual outputs don’t match the expected outputs

class pytest_wdl.core.LinkLocalizer(source: pathlib.Path)

Bases: pytest_wdl.core.Localizer

Localizes a file to another destination using a symlink.

localize(destination: pathlib.Path)

Localize a resource to destination.

Parameters

destination – Path to file where the non-local resource is to be localized.

class pytest_wdl.core.Localizer

Bases: object

Abstract base of classes that implement file localization.

abstract localize(destination: pathlib.Path) → None

Localize a resource to destination.

Parameters

destination – Path to file where the non-local resource is to be localized.

class pytest_wdl.core.StringLocalizer(contents: str)

Bases: pytest_wdl.core.Localizer

Localizes a string by writing it to a file.

localize(destination: pathlib.Path)

Localize a resource to destination.

Parameters

destination – Path to file where the non-local resource is to be localized.

class pytest_wdl.core.UrlLocalizer(url: str, user_config: pytest_wdl.core.UserConfiguration, http_headers: Optional[dict] = None)

Bases: pytest_wdl.core.Localizer

Localizes a file specified by a URL.

property http_headers

localize(destination: pathlib.Path)

Localize a resource to destination.

Parameters

destination – Path to file where the non-local resource is to be localized.

property proxies

class pytest_wdl.core.UserConfiguration(config_file: Optional[pathlib.Path] = None, cache_dir: Optional[pathlib.Path] = None, remove_cache_dir: Optional[bool] = None, execution_dir: Optional[pathlib.Path] = None, proxies: Optional[Dict[str, Union[str, Dict[str, str]]]] = None, http_headers: Optional[List[dict]] = None, show_progress: Optional[bool] = None, executor_defaults: Optional[Dict[str, dict]] = None)

Bases: object

Stores pytest-wdl configuration. If configuration options are specified both in the config file and as arguments to the constructor, the latter take precedence.

Parameters

• config_file – JSON file from which to load default values.

• cache_dir – The directory in which to cache localized files; defaults to using a temporary directory that is specific to each module and deleted afterwards.

• remove_cache_dir – Whether to remove the cache directory; if None, takes the value True if a temp directory is used for caching, and False, if a value for cache_dir is specified.

• execution_dir – The directory in which to run workflows. Defaults to None, which signals that a different temporary directory should be used for each workflow run.

• proxies – Mapping of proxy type (typically ‘http’ or ‘https’ to either an environment variable, or a dict with either/both keys ‘env’ and ‘value’, where the value is taken from the environment variable (‘env’) first, and from ‘value’ if the environment variable is not specified or is unset.

• http_headers – A list of dicts, each of which defines a header. The allowed keys are ‘pattern’, ‘name’, ‘env’, and ‘value’, where pattern is a URL pattern to match, ‘name’ is the header name and ‘env’ and ‘value’ are interpreted the same as for proxies. If no pattern is provided, the header is used for all URLs.

• show_progress – Whether to show progress bars when downloading remote test data files.

• executor_defaults – Mapping of executor name to dict of executor-specific configuration options.

cleanup() → None

Preforms cleanup operations, such as deleting the cache directory if self.remove_cache_dir is True.

get_executor_defaults(executor_name: str) → dict

Get default configuration values for the given executor.

Parameters

executor_name – The executor name

Returns

A dict with the executor configuration values, if any.

pytest_wdl.fixtures module

Fixtures for writing tests that execute WDL workflows using Cromwell.

Note: This library is being transitioned to python3 only, and to use `pathlib.Path`s instead of string paths. For backward compatibility fixtures that produce a path may still return string paths, but this support will be dropped in a future version.

pytest_wdl.fixtures.import_dirs(project_root: Union[str, pathlib.Path], import_paths: Union[str, pathlib.Path, None]) → List[Union[str, pathlib.Path]]

Fixture that provides a list of directories containing WDL scripts to make avaialble as imports. Uses the file provided by import_paths fixture if it is not None, otherwise returns a list containing the parent directory of the test module.

Parameters

• project_root – Project root directory

• import_paths – File listing paths to imports, one per line

pytest_wdl.fixtures.import_paths(request: _pytest.fixtures.FixtureRequest) → Union[str, pathlib.Path, None]

Fixture that provides the path to a file that lists directories containing WDL scripts to make available as imports. This looks for the file at “tests/import_paths.txt” by default, and returns None if that file doesn’t exist.

pytest_wdl.fixtures.project_root(request: _pytest.fixtures.FixtureRequest, project_root_files: List[str]) → Union[str, pathlib.Path]

Fixture that provides the root directory of the project. By default, this assumes that the project has one subdirectory per task, and that this framework is being run from the test subdirectory of a task diretory, and therefore looks for the project root two directories up.

pytest_wdl.fixtures.project_root_files() → List[str]

Fixture that provides a list of filenames that are found in the project root directory. Used by the project_root fixture to locate the project root directory.

pytest_wdl.fixtures.user_config(user_config_file: Optional[pathlib.Path]) → pytest_wdl.core.UserConfiguration

pytest_wdl.fixtures.user_config_file() → Optional[pathlib.Path]

Fixture that provides the value of ‘user_config’ environment variable. If not specified, looks in the default location ($HOME/pytest_user_config.json).

Returns

Path to the confif file, or None if not specified.

pytest_wdl.fixtures.workflow_data(request: _pytest.fixtures.FixtureRequest, workflow_data_resolver: pytest_wdl.core.DataResolver) → pytest_wdl.core.DataManager

Provides an accessor for test data files, which may be local or in a remote repository.

Parameters

• request – FixtureRequest object

• workflow_data_resolver – Module-level test data configuration

Examples

def workflow_data_descriptor_file():

return “tests/test_data.json”

def test_workflow(workflow_data):

print(workflow_data[“myfile”])

pytest_wdl.fixtures.workflow_data_descriptor_file() → Union[str, pathlib.Path]

Fixture that provides the path to the JSON file that describes test data files.

pytest_wdl.fixtures.workflow_data_descriptors(workflow_data_descriptor_file: Union[str, pathlib.Path]) → dict

Fixture that provides a mapping of test data names to values.

Parameters

workflow_data_descriptor_file – Path to the data descriptor JSON file.

Returns

A dict with keys as test data names and each value either a primitive, a map describing a data file, or a DataFile object.

pytest_wdl.fixtures.workflow_data_resolver(workflow_data_descriptors: dict, user_config: pytest_wdl.core.UserConfiguration) → pytest_wdl.core.DataResolver

Provides access to test data files for tests in a module.

Parameters

• workflow_data_descriptors – workflow_data_descriptors fixture.

• user_config –

pytest_wdl.fixtures.workflow_runner(project_root: Union[str, pathlib.Path], import_dirs: List[Union[str, pathlib.Path]], user_config: pytest_wdl.core.UserConfiguration)

Provides a callable that runs a workflow (with the same signature as Executor.run_workflow).

Parameters

• project_root – Project root directory.

• import_dirs – Directories from which to import WDL scripts.

• user_config –

pytest_wdl.utils module

Utility functions for pytest-wdl.

class pytest_wdl.utils.PluginFactory(entry_point: pkg_resources.EntryPoint, return_type: Type[T])

Bases: typing.Generic

Lazily loads a plugin class associated with a data type.

pytest_wdl.utils.chdir(todir: pathlib.Path)

Context manager that temporarily changes directories.

Parameters

todir – The directory to change to.

pytest_wdl.utils.context_dir(path: Optional[pathlib.Path] = None, change_dir: bool = False, cleanup: Optional[bool] = None) → pathlib.Path

Context manager that looks for a specific environment variable to specify a directory. If the environment variable is not set, a temporary directory is created and cleaned up upon return from the yield.

Parameters

• path – The environment variable to look for.

• change_dir – Whether to change to the directory.

• cleanup – Whether to delete the directory when exiting the context. If None, the directory is only deleted if a temporary directory is created.

Yields

A directory path.

pytest_wdl.utils.download_file(url: str, destination: pathlib.Path, http_headers: Optional[dict] = None, proxies: Optional[dict] = None, show_progress: bool = True)

pytest_wdl.utils.ensure_path(path: Union[str, py._path.local.LocalPath, pathlib.Path], root: Optional[pathlib.Path] = None, canonicalize: bool = True, exists: Optional[bool] = None, is_file: Optional[bool] = None, executable: Optional[bool] = None, create: bool = False) → pathlib.Path

Converts a string path or py.path.local.LocalPath to a pathlib.Path.

Parameters

• path – The path to convert.

• root – Root directory to use to make path absolute if it is not already.

• canonicalize – Whether to return the canonicalized version of the path - expand home directory shortcut (~), make absolute, and resolve symlinks.

• exists – If True, raise an exception if the path does not exist; if False, raise an exception if the path does exist.

• is_file – If True, raise an exception if the path is not a file; if False, raise an exception if the path is not a directory.

• executable – If True and is_file is True and the file exists, raise an exception if it is not executable.

• create – Create the directory (or parent, if path is an existing file) if it does not exist. Ignored if exists is True.

Returns

A pathlib.Path object.

pytest_wdl.utils.env_map(d: dict) → dict

Given a mapping of keys to value descriptors, creates a mapping of the keys to the described values.

pytest_wdl.utils.find_executable_path(executable: str, search_path: Optional[Sequence[pathlib.Path]] = None) → Optional[pathlib.Path]

Finds ‘executable’ in search_path.

Parameters

• executable – The name of the executable to find.

• search_path – The list of directories to search. If None, the system search path (defined by the $PATH environment variable) is used.

Returns

Absolute path of the executable, or None if no matching executable was found.

pytest_wdl.utils.find_in_classpath(glob: str) → Optional[pathlib.Path]

Attempts to find a .jar file matching the specified glob pattern in the Java classpath.

Parameters

glob – JAR filename pattern

Returns

Path to the JAR file, or None if a matching file is not found.

pytest_wdl.utils.find_project_path(*filenames: Union[str, pathlib.Path], start: Optional[pathlib.Path] = None, return_parent: bool = False, assert_exists: bool = False) → Optional[pathlib.Path]

Starting from path folder and moving upwards, search for any of filenames and return the first path containing any one of them.

Parameters

• *filenames – Filenames to search. Either a string filename, or a sequence of string path elements.

• start – Starting folder

• return_parent – Whether to return the containing folder or the discovered file.

• assert_exists – Whether to raise an exception if a file cannot be found.

Returns

A Path, or None if no folder is found that contains any of filenames. If return_parent is False and more than one of the files is found one of the files is randomly selected for return.

Raises

FileNotFoundError if the file cannot be found and assert_exists is True. –

pytest_wdl.utils.is_executable(path: pathlib.Path) → bool

Checks if a path is executable.

Parameters

path – The path to check

Returns

True if path exists and is executable by the user, otherwise False.

pytest_wdl.utils.plugin_factory_map(return_type: Type[T], group: Optional[str] = None, entry_points: Optional[Iterable[pkg_resources.EntryPoint]] = None) → Dict[str, pytest_wdl.utils.PluginFactory[~T][T]]

Creates a mapping of entry point name to PluginFactory for all discovered entry points in the specified group.

Parameters

• group – Entry point group name

• return_type – Expected return type

Returns

Dict mapping entry point name to PluginFactory instances

pytest_wdl.utils.resolve_file(filename: Union[str, pathlib.Path], project_root: pathlib.Path, assert_exists: bool = True) → Optional[pathlib.Path]

Finds filename under project_root or in the project path.

Parameters

• filename – The filename, relative path, or absolute path to resolve.

• project_root – The project root dir.

• assert_exists – Whether to raise an error if the file cannot be found.

Returns

A pathlib.Path object, or None if the file cannot be found and assert_exists is False.

Raises

FileNotFoundError if the file cannot be found and assert_exists is True. –

pytest_wdl.utils.resolve_value_descriptor(value_descriptor: Union[str, dict]) → Optional

Resolves the value of a value descriptor, which may be an environment variable name, or a map with keys env (the environment variable name) and value (the value to use if env is not specified or if the environment variable is unset.

Parameters

value_descriptor –

Returns:

pytest_wdl.utils.safe_string(s: str, replacement: str = '_') → str

Makes a string safe by replacing non-word characters.

Parameters

• s – The string to make safe

• replacement – The replacement stringj

Returns

The safe string

pytest_wdl.utils.tempdir(change_dir: bool = False) → pathlib.Path

Context manager that creates a temporary directory, yields it, and then deletes it after return from the yield.

Parameters

change_dir – Whether to temporarily change to the temp dir.

Module contents

Fixtures for writing tests that execute WDL workflows using Cromwell. For testability purposes, the implementaiton of these fixtures is done in the pytest_wdl.fixtures module.

Indices and tables

• Index

• Module Index

• Search Page