Class: OodCore::Job::Adapters::PBSPro::Batch Private

Inherits:
Object
  • Object
show all
Defined in:
lib/ood_core/job/adapters/pbspro.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

Object used for simplified communication with a PBS Pro batch server

Defined Under Namespace

Classes: Error

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(host: nil, submit_host: "", strict_host_checking: true, pbs_exec: nil, bin_overrides: {}) ⇒ Batch

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Batch.

Parameters:

  • host (#to_s, nil) (defaults to: nil) —

    the batch server host

  • submit_host (#to_s, nil) (defaults to: "") —

    the login node to ssh to

  • strict_host_checking (bool, true) (defaults to: true) —

    wheter to use strict host checking when ssh to submit_host

  • exec (#to_s, nil) —

    path to pbs executables



81
82
83
84
85
86
87
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 81

def initialize(host: nil, submit_host: "", strict_host_checking: true, pbs_exec: nil, bin_overrides: {})
  @host                 = host && host.to_s
  @submit_host          = submit_host && submit_host.to_s
  @strict_host_checking = strict_host_checking
  @pbs_exec             = pbs_exec && Pathname.new(pbs_exec.to_s)
  @bin_overrides        = bin_overrides
end

Instance Attribute Details

#bin_overrides ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Optional overrides for PBS Pro client executables

Examples:

{'qsub' => '/usr/local/bin/qsub'}


71
72
73
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 71

def bin_overrides
  @bin_overrides
end

#host ⇒ String? (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

The host of the PBS Pro batch server

Examples:

my_batch.host #=> "my_batch.server.edu"

Returns:

  • (String, nil) —

    the batch server host



47
48
49
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 47

def host
  @host
end

#pbs_exec ⇒ Pathname? (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

The path containing the PBS executables

Examples:

my_batch.pbs_exec.to_s #=> "/usr/local/pbspro/10.0.0

Returns:

  • (Pathname, nil) —

    path to pbs executables



65
66
67
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 65

def pbs_exec
  @pbs_exec
end

#strict_host_checking ⇒ Bool, true (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Whether to use strict host checking when ssh to submit_host

Examples:

my_batch.strict_host_checking #=> "false"

Returns:

  • (Bool, true) —

    the login node; true if not present



59
60
61
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 59

def strict_host_checking
  @strict_host_checking
end

#submit_host ⇒ String? (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

The login node to submit the job via ssh

Examples:

my_batch.submit_host #=> "my_batch.server.edu"

Returns:

  • (String, nil) —

    the login node



53
54
55
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 53

def submit_host
  @submit_host
end

Instance Method Details

#delete_job(id) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Delete a specified job from batch server

Examples:

Delete job “1234”

my_batch.delete_job("1234")

Parameters:

  • id (#to_s) —

    the id of the job

Raises:

  • (Error) —

    if ‘qdel` command exited unsuccessfully



207
208
209
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 207

def delete_job(id)
  call("qdel", id.to_s)
end

#get_cluster_info ⇒ ClusterInfo

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Get a ClusterInfo object containing information about the given cluster

Returns:



91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 91

def get_cluster_info
  args = ["-a", "-F", "json"]
  stdout = call("pbsnodes", *args)
  node_info = JSON.parse(stdout)
    
  # Initialize cluster info values
  total_nodes     = 0
  allocated_nodes = 0
  total_cpus      = 0
  allocated_cpus  = 0
  total_gpus      = 0
  allocated_gpus  = 0 

  nodes = node_info.fetch('nodes', {})

  nodes.each do |_node_name, node|
    total_nodes += 1
    resources_avail = node.fetch('resources_available', {})
    total_cpus += get_node_resource(resources_avail, 'ncpus')
    total_gpus += get_node_resource(resources_avail, 'ngpus')

    # Resources assigned (currently allocated to jobs)
    resources_assigned = node.fetch('resources_assigned', {})
    ncpus_assigned     = get_node_resource(resources_assigned, 'ncpus')
    ngpus_assigned     = get_node_resource(resources_assigned, 'ngpus')

    allocated_cpus += ncpus_assigned
    allocated_gpus += ngpus_assigned

    # A node is allocated if at least one CPU has been assigned to a job
    allocated_nodes += 1 if ncpus_assigned > 0
  end

  ClusterInfo.new(active_nodes: allocated_nodes,
                  total_nodes: total_nodes,
                  active_processors: allocated_cpus,
                  total_processors: total_cpus,
                  active_gpus: allocated_gpus,
                  total_gpus: total_gpus
  )
end

#get_jobs(id: "") ⇒ Array<Hash>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Get a list of hashes detailing each of the jobs on the batch server

Examples:

Status info for all jobs

my_batch.get_jobs
#=>
#[
#  {
#    :account => "account",
#    :job_id => "my_job",
#    ...
#  },
#  {
#    :account => "account",
#    :job_id => "my_other_job",
#    ...
#  },
#  ...
#]

Parameters:

  • id (#to_s) (defaults to: "") —

    the id of the job

Returns:

  • (Array<Hash>) —

    list of details for jobs

Raises:

  • (Error) —

    if ‘qstat` command exited unsuccessfully



153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 153

def get_jobs(id: "")
  args = ["-f", "-t"]   # display all information
  args.concat [id.to_s] unless id.to_s.empty?
  lines = call("qstat", *args).gsub("\n\t", "").split("\n").map(&:strip)

  jobs = []
  lines.each do |line|
    if /^Job Id: (?<job_id>.+)$/ =~ line
      jobs << { job_id: job_id }
    elsif /^(?<key>[^\s]+) = (?<value>.+)$/ =~ line
      hsh = jobs.last
      k1, k2 = key.split(".").map(&:to_sym)
      k2 ? ( hsh[k1] ||= {} and hsh[k1][k2] = value ) : ( hsh[k1] = value )
    end
  end

  jobs
end

#hold_job(id) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Put a specified job on hold

Examples:

Put job “1234” on hold

my_batch.hold_job("1234")

Parameters:

  • id (#to_s) —

    the id of the job

Raises:

  • (Error) —

    if ‘qhold` command exited unsuccessfully



187
188
189
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 187

def hold_job(id)
  call("qhold", id.to_s)
end

#release_job(id) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Release a specified job that is on hold

Examples:

Release job “1234” from on hold

my_batch.release_job("1234")

Parameters:

  • id (#to_s) —

    the id of the job

Raises:

  • (Error) —

    if ‘qrls` command exited unsuccessfully



197
198
199
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 197

def release_job(id)
  call("qrls", id.to_s)
end

#select_jobs(args: []) ⇒ Array<String>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Select batch jobs from the batch server

Parameters:

  • args (Array<#to_s>) (defaults to: []) —

    arguments passed to ‘qselect` command

Returns:

  • (Array<String>) —

    list of job ids that match selection criteria

Raises:

  • (Error) —

    if ‘qselect` command exited unsuccessfully



177
178
179
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 177

def select_jobs(args: [])
  call("qselect", *args).split("\n").map(&:strip)
end

#submit_string(str, args: [], chdir: nil) ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Submit a script expanded as a string to the batch server

Parameters:

  • str (#to_s) —

    script as a string

  • args (Array<#to_s>) (defaults to: []) —

    arguments passed to ‘qsub` command

  • chdir (#to_s, nil) (defaults to: nil) —

    working directory where ‘qsub` is called

Returns:

  • (String) —

    the id of the job that was created

Raises:

  • (Error) —

    if ‘qsub` command exited unsuccessfully



217
218
219
 
# File 'lib/ood_core/job/adapters/pbspro.rb', line 217

def submit_string(str, args: [], chdir: nil)
  call("qsub", *args, stdin: str.to_s, chdir: chdir).strip
end