API¶

SlurmJob¶

class slurmpter.SlurmJob(name, executable, error=None, output=None, submit=None, nodes=None, ntasks_per_node=None, cpus_per_task=None, mem_per_node=None, extra_sbatch_options=None, extra_srun_options=['ntasks=1', 'exclusive'], extra_lines=None, modules=None, slurm=None, arguments=None, verbose=0)¶

A class for handling Slurm job.

Methods:

`__init__`(name, executable[, error, output, …])	Constructor.
`add_arg`(arg[, name, retry])	Add argument to Job
`add_args`(args)	Adds multiple arguments to Job
`add_child`(node)	Adds node to children list
`add_children`(nodes)	Adds nodes to the children list
`add_parent`(node)	Adds node to parents list
`add_parents`(nodes)	Adds nodes to the parents list
`build`([makedirs, fancyname])	Build and save the submit file for Job.
`build_submit`([makedirs, fancyname, …])	Build and submit sequentially.
`haschildren`()	Checks for any children nodes
`hasparents`()	Checks for any parent nodes
`submit_job`([submit_options])	Submit Job to Slurm.

__init__(name, executable, error=None, output=None, submit=None, nodes=None, ntasks_per_node=None, cpus_per_task=None, mem_per_node=None, extra_sbatch_options=None, extra_srun_options=['ntasks=1', 'exclusive'], extra_lines=None, modules=None, slurm=None, arguments=None, verbose=0)¶

Constructor.

Parameters

name (str) – Name of the job.
executable (str) – Path of the executable for the job.
error (str, optional) – Directory of error files.
output (str, optional) – Directory of output files.
submit (str, optional) – Directory of submit files.
nodes (str, optional) – <minnodes[-maxnodes]> Request that a minimum of minnodes nodes be allocated to this job. A maximum node count may also be specified with maxnodes. If only one number is specified, this is used as both the minimum and maximum node count.
ntasks_per_node (str, optional) – <ntasks> Request that ntasks be invoked on each node.
cpus_per_task (str, optional) – <ncpus> Request that ncpus be allocated per process.
mem_per_node (str, optional) – <size[units]> Specify the real memory required per node.
extra_sbatch_options (array-like str, optionl) – An array of extra options to append after ‘#SBATCH ‘.
extra_srun_option (array-like str, optional) – An array of extra options to append after ‘srun’.
extra_lines (array-like str, optional) – An array of extra lines to add before srun.
modules (array-like str, optional) – An array of modules to append after ‘module load ‘.
slurm (Slurm, optional) – If specified, SlurmJob will be added to Slurm.
arguments (str or iterable, optional) – Arguments to initialize the job list.
verbose (int, optional) – Level of logging verbosity option are 0-warning, 1-info, 2-debugging (default is 0).

add_arg(arg, name=None, retry=None)¶

Add argument to Job

Parameters

arg (str) – Argument to append to Job args list.
name (str or None, optional) –
Option to specify a name related to this argument. If a name is specified, then a separate set of log, output, and error files will be generated for this particular argument (default is None).

New in version 0.1.2.
retry (int or None, optional) –
Option to specify the number of times to retry this node. Default number of retries is 0. Note: this feature is only available to Jobs that are submitted via a Dagman.

New in version 0.1.2.

Returns

self – Returns self.

Return type

object

add_args(args)¶

Adds multiple arguments to Job

Parameters: args (iterable) – Iterable of arguments to append to the arguments list
Returns: self – Returns self.
Return type: object

add_child(node)¶

Adds node to children list

Parameters: node (BaseNode) – Job or Dagman to append to the children list.
Returns: self – Returns self.
Return type: object

add_children(nodes)¶

Adds nodes to the children list

Parameters: nodes (list or tuple) – List of nodes to append to the children list
Returns: self – Returns self.
Return type: object

add_parent(node)¶

Adds node to parents list

Parameters: node (BaseNode) – Job or Dagman to append to the parents list.
Returns: self – Returns self.
Return type: object

add_parents(nodes)¶

Adds nodes to the parents list

Parameters: nodes (list or tuple) – List of nodes to append to the parents list
Returns: self – Returns self.
Return type: object

build(makedirs=True, fancyname=True)¶

Build and save the submit file for Job.

Parameters

makedirs (bool, optional) – Create job directories if not exist.
fancyname (bool, optional) – Append the name with date and unique id.

Returns

self – Self object.

Return type

object

build_submit(makedirs=True, fancyname=True, submit_options=None)¶

Build and submit sequentially.

Parameters

makedirs (bool, optional) – Create directories if not exist.
fancyname (bool, optional) – Append date of unique id to submit.
submit_options (str, optional) – Options to be passed to ‘sbatch’.

Returns

self – Self object.

Return type

object

haschildren()¶

Checks for any children nodes

Returns: Returns whether or not this node has any child nodes.
Return type: bool

hasparents()¶

Checks for any parent nodes

Returns: Returns whether or not this node has any parent nodes.
Return type: bool

submit_job(submit_options=None)¶

Submit Job to Slurm.

Parameters: submit_options (str, optional) – Submit options appends after sbatch.
Returns: self – Self object.
Return type: object

Slurm¶

class slurmpter.Slurm(name, submit=None, extra_lines=None, verbose=0)¶

Slurm object manages the workflow of a series of SlurmJobs.

Methods:

`__init__`(name[, submit, extra_lines, verbose])	Constructor.
`add_job`(job)	Add job to Slurm.
`build`([makedirs, fancyname])	Build slurm submit files.
`build_submit`([makedirs, fancyname, …])	Build and submit sequentially.
`submit_slurm`([submit_options])	Submit to slurm.
`visualize`([filename])	Visualize Slurm graph.

__init__(name, submit=None, extra_lines=None, verbose=0)¶

Constructor.

Parameters

name (str) – Name of the Dagman instance.
submit (str) – Directory to write submit files.
extra_lines (array-like str) – Extra lines to add into the submit file.
verbose (int) – Level of logging verbosity.

add_job(job)¶

Add job to Slurm.

Parameters: job (SlurmJob) – SlurmJob to append to Slurm job list.

build(makedirs=True, fancyname=True)¶

Build slurm submit files.

Parameters

makedirs (bool, optional) – Create job directories if do not exist.
fancyname (bool, optional) – Append the date and unique id number to error, log, output and submit files.

build_submit(makedirs=True, fancyname=True, submit_options=None)¶

Build and submit sequentially.

Parameters

makedirs (bool, optional) – Create directories if not exist.
fancyname (bool, optional) – Append date of unique id to submit.
submit_options (str, optional) – Options to be passed to ‘sbatch’.

Returns

self – Self object.

Return type

object

submit_slurm(submit_options=None)¶

Submit to slurm.

Parameters: submit_options (str, optional) – Submit options appends after sbatch.

visualize(filename=None)¶

Visualize Slurm graph.

Parameters: filename (str or None, optional) – File to save graph diagram to. If None then no file is saved. Valid file extensions are ‘png’, ‘pdf’, ‘dot’, ‘svg’, ‘jpeg’, ‘jpg’.