executorlib.task_scheduler.file.shared.execute_tasks_h5

executorlib.task_scheduler.file.shared.execute_tasks_h5#

executorlib.task_scheduler.file.shared.execute_tasks_h5(future_queue: Queue, execute_function: Callable, executor_kwargs: dict, terminate_function: Callable | None = None, pysqa_config_directory: str | None = None, backend: str | None = None, disable_dependencies: bool = False, pmi_mode: str | None = None, wait: bool = True, refresh_rate: float = 0.01) → None[source]#

Execute tasks stored in a queue using HDF5 files.

Parameters:

  • future_queue (queue.Queue) – The queue containing the tasks.

  • executor_kwargs (dict) – A dictionary of executor arguments required by the task. With the following keys: - cores (int): number of MPI cores to be used for each function call - cwd (str/None): current working directory where the parallel python task is executed

  • execute_function (Callable) – The function to execute the tasks.

  • terminate_function (Callable) – The function to terminate the tasks.

  • pysqa_config_directory (str, optional) – path to the pysqa config directory (only for pysqa based backend).

  • backend (str, optional) – name of the backend used to spawn tasks.

  • disable_dependencies (boolean) – Disable resolving future objects during the submission.

  • pmi_mode (str) – PMI interface to use (OpenMPI v5 requires pmix) default is None (Flux only)

  • wait (bool) – Whether to wait for the completion of all tasks before shutting down the executor.

  • refresh_rate (float) – The rate at which to refresh the result. Defaults to 0.01.

Returns:

None