cwltool.job

Attributes

CollectOutputsType

needs_shell_quoting_re

FORCE_SHELLED_POPEN

SHELL_COMMAND_TEMPLATE

CONTROL_CODE_RE

Classes

JobBase

Base class for get_requirement().

CommandLineJob

Base class for get_requirement().

ContainerCommandLineJob

Commandline job using containers.

Functions

relink_initialworkdir(pathmapper, host_outdir, ...[, ...])

neverquote(string[, pos, endpos])

No-op.

Module Contents

cwltool.job.CollectOutputsType
cwltool.job.needs_shell_quoting_re
cwltool.job.FORCE_SHELLED_POPEN
cwltool.job.SHELL_COMMAND_TEMPLATE = Multiline-String
Show Value
"""#!/bin/bash
python3 "run_job.py" "job.json"
"""
Parameters:
Return type:

None

cwltool.job.neverquote(string, pos=0, endpos=0)

No-op.

Parameters:
Return type:

re.Match[str] | None

class cwltool.job.JobBase(builder, joborder, make_path_mapper, requirements, hints, name)

Bases: cwltool.utils.HasReqsHints

Inheritance diagram of cwltool.job.JobBase

Base class for get_requirement().

Parameters:
builder
joborder
stdin: str | None = None
stderr: str | None = None
stdout: str | None = None
successCodes: collections.abc.Iterable[int] = []
temporaryFailCodes: collections.abc.Iterable[int] = []
permanentFailCodes: collections.abc.Iterable[int] = []
requirements
hints
name
command_line: list[str] = []
pathmapper
make_path_mapper
generatemapper: cwltool.pathmapper.PathMapper | None = None
collect_outputs = None
output_callback: cwltool.utils.OutputCallbackType | None = None
outdir = ''
tmpdir = ''
environment: collections.abc.MutableMapping[str, str]
generatefiles
stagedir: str | None = None
inplace_update = False
timelimit: int | None = None
networkaccess: bool = False
mpi_procs: int | None = None
__repr__()

Represent this Job object.

Return type:

str

abstractmethod run(runtimeContext, tmpdir_lock=None)
Parameters:
Return type:

None

static extract_environment(runtimeContext, envVarReq)

Extract environment variables that should be preserved.

Parameters:
Return type:

collections.abc.Mapping[str, str]

prepare_environment(runtimeContext, envVarReq)

Set up environment variables.

Here we prepare the environment for the job, based on any preserved variables and EnvVarRequirement. Later, changes due to MPIRequirement, Secrets, or SoftwareRequirement are applied (in that order).

Parameters:
Return type:

None

process_monitor(sproc)

Watch a process, logging its max memory usage.

Parameters:

sproc (subprocess.Popen[str])

Return type:

None

class cwltool.job.CommandLineJob(builder, joborder, make_path_mapper, requirements, hints, name)

Bases: JobBase

Inheritance diagram of cwltool.job.CommandLineJob

Base class for get_requirement().

Parameters:
run(runtimeContext, tmpdir_lock=None)
Parameters:
Return type:

None

cwltool.job.CONTROL_CODE_RE = '\\x1b\\[[0-9;]*[a-zA-Z]'
class cwltool.job.ContainerCommandLineJob(builder, joborder, make_path_mapper, requirements, hints, name)

Bases: JobBase

Inheritance diagram of cwltool.job.ContainerCommandLineJob

Commandline job using containers.

Parameters:
CONTAINER_TMPDIR: str = '/tmp'
abstractmethod get_from_requirements(r, pull_image, force_pull, tmp_outdir_prefix, image_base_path=None)
Parameters:

  • r (cwl_utils.types.CWLObjectType)

  • pull_image (bool)

  • force_pull (bool)

  • tmp_outdir_prefix (str)

  • image_base_path (str | None)

Return type:

str | None

abstractmethod create_runtime(env, runtime_context)

Return the list of commands to run the selected container engine.

Parameters:
Return type:

tuple[list[str], str | None]

static append_volume(runtime, source, target, writable=False)
Abstractmethod:

Parameters:
Return type:

None

Add binding arguments to the runtime list.

abstractmethod add_file_or_directory_volume(runtime, volume, host_outdir_tgt)

Append volume a file/dir mapping to the runtime option list.

Parameters:
Return type:

None

abstractmethod add_writable_file_volume(runtime, volume, host_outdir_tgt, tmpdir_prefix)

Append a writable file mapping to the runtime option list.

Parameters:
Return type:

None

abstractmethod add_writable_directory_volume(runtime, volume, host_outdir_tgt, tmpdir_prefix)

Append a writable directory mapping to the runtime option list.

Parameters:
Return type:

None

create_file_and_add_volume(runtime, volume, host_outdir_tgt, secret_store, tmpdir_prefix)

Create the file and add a mapping.

Parameters:
Return type:

str

add_volumes(pathmapper, runtime, tmpdir_prefix, secret_store=None, any_path_okay=False)

Append volume mappings to the runtime option list.

Parameters:
Return type:

None

run(runtimeContext, tmpdir_lock=None)
Parameters:
Return type:

None

docker_monitor(cidfile, tmpdir_prefix, cleanup_cidfile, docker_exe, process)

Record memory usage of the running Docker container.

Parameters:
Return type:

None