parsl.executors.ThreadPoolExecutor

class parsl.executors.ThreadPoolExecutor(label: str = 'threads', max_threads: int | None = 2, thread_name_prefix: str | None = None, storage_access: List[Staging] | None = None, working_dir: str | None = None, remote_monitoring_radio: RadioConfig | None = None)

A thread-based executor.

Parameters:

• max_threads (Optional[int]) – Number of threads. Default is 2.

• thread_name_prefix (string) – Thread name prefix

• storage_access (list of Staging) – Specifications for accessing data this executor remotely.

__init__(label: str = 'threads', max_threads: int | None = 2, thread_name_prefix: str | None = None, storage_access: List[Staging] | None = None, working_dir: str | None = None, remote_monitoring_radio: RadioConfig | None = None)

Methods

__init__([label, max_threads, ...])
monitor_resources()	Resource monitoring does not make sense when using the ThreadPoolExecutor, as there is no per-task process tree: all tasks run inside the same single submitting process.
shutdown([block])	Shutdown the ThreadPool.
start()	Start the executor.
submit(func, resource_specification, *args, ...)	Submits work to the thread pool.

Attributes

label
run_dir	Path to the run directory.
run_id	UUID for the enclosing DFK.

monitor_resources()

Resource monitoring does not make sense when using the ThreadPoolExecutor, as there is no per-task process tree: all tasks run inside the same single submitting process.

In addition, the use of fork-based multiprocessing in the remote wrapper in parsl/monitoring/remote.py was especially prone to deadlock with this executor.

shutdown(block=True)

Shutdown the ThreadPool. The underlying concurrent.futures thread pool implementation will not terminate tasks that are being executed, because it does not provide a mechanism to do that. With block set to false, this will return immediately and it will appear as if the DFK is shut down, but the python process will not be able to exit until the thread pool has emptied out by task completions. In either case, this can be a very long wait.

Kwargs:

• block (Bool): To block for confirmations or not

start()

Start the executor.

By default, this does nothing, but this method should be overridden to perform any spin-up operations (for example: starting thread pools).

submit(func, resource_specification, *args, **kwargs)

Submits work to the thread pool.

This method is simply pass through and behaves like a submit call as described here Python docs: