Class: Cloudtasker::Config

Inherits:
Object
  • Object
show all
Defined in:
lib/cloudtasker/config.rb

Overview

Holds cloudtasker configuration. See Cloudtasker#configure

Constant Summary collapse

MAX_TASK_SIZE =

Max Cloud Task size in bytes

100 * 1024
RETRY_HEADER =

Retry header in Cloud Task responses

Definitions:

X-CloudTasks-TaskRetryCount: total number of retries (including 504 "instance unreachable")
X-CloudTasks-TaskExecutionCount: number of non-503 retries (= actual number of job failures)

'X-Cloudtasks-Taskexecutioncount'
TASK_ID_HEADER =

Cloud Task ID header

'X-CloudTasks-TaskName'
ENCODING_HEADER =

Content-Transfer-Encoding header in Cloud Task responses

'Content-Transfer-Encoding'
CONTENT_TYPE_HEADER =

Content Type

'Content-Type'
OIDC_AUTHORIZATION_HEADER =

OIDC Authorization header

'Authorization'
CT_AUTHORIZATION_HEADER =

Custom authentication header that does not conflict with OIDC authorization header

'X-Cloudtasker-Authorization'
CT_SIGNATURE_HEADER = 
'X-Cloudtasker-Signature'
DEFAULT_LOCATION_ID =

Default values

'us-east1'
DEFAULT_PROCESSOR_PATH = 
'/cloudtasker/run'
DEFAULT_LOCAL_SERVER_SSL_VERIFY_MODE = 
true
DEFAULT_JOB_QUEUE =

Default queue values

'default'
DEFAULT_QUEUE_CONCURRENCY = 
10
DEFAULT_QUEUE_RETRIES =

unlimited

-1 # unlimited
DEFAULT_DISPATCH_DEADLINE =

Job timeout configuration for Cloud Tasks

10 * 60
MIN_DISPATCH_DEADLINE =

10 minutes

15
MAX_DISPATCH_DEADLINE =

seconds

30 * 60
DEFAULT_ON_ERROR =

Default on_error Proc

->(error, worker) {}
WORKER_STORE_PREFIX =

Cache key prefix used to store workers in cache and retrieve them later.

'worker_store'
DEFAULT_MAX_RETRY_ATTEMPTS =

The number of times jobs will be attempted before declaring them dead.

With the default retry configuration (maxDoublings = 16 and minBackoff = 0.100s) it means that jobs will be declared dead after 20h of consecutive failing.

Note that this configuration parameter is internal to Cloudtasker and does not affect the Cloud Task queue configuration. The number of retries configured on the Cloud Task queue should be higher than the number below to also cover failures due to the instance being unreachable.

25
PROCESSOR_HOST_MISSING = 
<<~DOC
  Missing host for processing.
  Please specify a processor hostname in form of `https://some-public-dns.example.com`'
DOC
PROJECT_ID_MISSING_ERROR = 
<<~DOC
  Missing GCP project ID.
  Please specify a project ID in the cloudtasker configurator.
DOC
SECRET_MISSING_ERROR = 
<<~DOC
  Missing cloudtasker secret.
  Please specify a secret in the cloudtasker initializer or add Rails secret_key_base in your credentials
DOC
OIDC_EMAIL_MISSING_ERROR = 
<<~DOC
  Missing OpenID Connect (OIDC) service account email.
  You specified an OIDC configuration hash but the :service_account_email property is missing.
DOC

Instance Attribute Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#dispatch_deadline ⇒ Integer

Return the Dispatch deadline duration. Cloud Tasks will timeout the job after this duration is elapsed.

Returns:

  • (Integer) —

    The value in seconds.



220
221
222
 
# File 'lib/cloudtasker/config.rb', line 220

def dispatch_deadline
  @dispatch_deadline || DEFAULT_DISPATCH_DEADLINE
end

#gcp_location_id ⇒ String

Return the GCP location ID. Default to ‘us-east1’

Returns:

  • (String) —

    The location ID where tasks will be processed.



210
211
212
 
# File 'lib/cloudtasker/config.rb', line 210

def gcp_location_id
  @gcp_location_id || DEFAULT_LOCATION_ID
end

#gcp_project_id ⇒ String

Return the GCP project ID.

Returns:

  • (String) —

    The ID of the project for which tasks will be processed.



201
202
203
 
# File 'lib/cloudtasker/config.rb', line 201

def gcp_project_id
  @gcp_project_id || raise(StandardError, PROJECT_ID_MISSING_ERROR)
end

#gcp_queue_prefix ⇒ Object

Returns the value of attribute gcp_queue_prefix.



8
9
10
 
# File 'lib/cloudtasker/config.rb', line 8

def gcp_queue_prefix
  @gcp_queue_prefix
end

#local_server_ssl_verify ⇒ Boolean

Return the ssl verify mode for the Cloudtasker local server.

Returns:

  • (Boolean) —

    The ssl verify mode for the Cloudtasker local server.



298
299
300
 
# File 'lib/cloudtasker/config.rb', line 298

def local_server_ssl_verify
  @local_server_ssl_verify.nil? ? DEFAULT_LOCAL_SERVER_SSL_VERIFY_MODE : @local_server_ssl_verify
end

#logger ⇒ Logger, any

Return the Cloudtasker logger.

Returns:

  • (Logger, any) —

    The cloudtasker logger.



142
143
144
 
# File 'lib/cloudtasker/config.rb', line 142

def logger
  @logger ||= defined?(Rails) ? Rails.logger : ::Logger.new($stdout)
end

#max_retries ⇒ Integer

The number of times jobs will be retried. This number of retries does not include failures due to the application being unreachable.

Returns:

  • (Integer) —

    The number of retries



113
114
115
 
# File 'lib/cloudtasker/config.rb', line 113

def max_retries
  @max_retries ||= DEFAULT_MAX_RETRY_ATTEMPTS
end

#mode ⇒ <Type>

The operating mode.

- :production => process tasks via GCP Cloud Task.
- :development => process tasks locally via Redis.

Returns:

  • (<Type>) —

    <description>



124
125
126
 
# File 'lib/cloudtasker/config.rb', line 124

def mode
  @mode ||= environment == 'development' ? :development : :production
end

#oidc ⇒ Hash

Return the Open ID Connect configuration to use for tasks.

Returns:

  • (Hash) —

    The OIDC configuration

Raises:

  • (StandardError) 


261
262
263
264
265
266
267
268
269
 
# File 'lib/cloudtasker/config.rb', line 261

def oidc
  return unless @oidc
  raise(StandardError, OIDC_EMAIL_MISSING_ERROR) unless @oidc[:service_account_email]

  {
    service_account_email: @oidc[:service_account_email],
    audience: @oidc[:audience] || processor_host
  }
end

#on_dead ⇒ Proc

Return a Proc invoked whenever a worker DeadWorkerError is raised. See Cloudtasker::WorkerHandler.with_worker_handling

Returns:

  • (Proc) —

    A Proc handler



252
253
254
 
# File 'lib/cloudtasker/config.rb', line 252

def on_dead
  @on_dead || DEFAULT_ON_ERROR
end

#on_error ⇒ Proc

Return a Proc invoked whenever a worker runtime error is raised. See Cloudtasker::WorkerHandler.with_worker_handling

Returns:

  • (Proc) —

    A Proc handler



242
243
244
 
# File 'lib/cloudtasker/config.rb', line 242

def on_error
  @on_error || DEFAULT_ON_ERROR
end

#processor_path ⇒ String

The path on the host when worker payloads will be sent. Default to ‘/cloudtasker/run`

Returns:

  • (String) —

    The processor path



192
193
194
 
# File 'lib/cloudtasker/config.rb', line 192

def processor_path
  @processor_path || DEFAULT_PROCESSOR_PATH
end

#redis ⇒ Object

Returns the value of attribute redis.



8
9
10
 
# File 'lib/cloudtasker/config.rb', line 8

def redis
  @redis
end

#secret ⇒ String

Return the secret to use to sign the verification tokens attached to tasks.

Returns:

  • (String) —

    The cloudtasker secret



230
231
232
233
234
 
# File 'lib/cloudtasker/config.rb', line 230

def secret
  @secret ||= (
    defined?(Rails) && Rails.application.credentials&.dig(:secret_key_base)
  ) || raise(StandardError, SECRET_MISSING_ERROR)
end

#store_payloads_in_redis ⇒ Object

Returns the value of attribute store_payloads_in_redis.



8
9
10
 
# File 'lib/cloudtasker/config.rb', line 8

def store_payloads_in_redis
  @store_payloads_in_redis
end

Instance Method Details

#client_middleware {|@client_middleware| ... } ⇒ Cloudtasker::Middleware::Chain

Return the chain of client middlewares.

Yields:

Returns:



276
277
278
279
280
 
# File 'lib/cloudtasker/config.rb', line 276

def client_middleware
  @client_middleware ||= Middleware::Chain.new
  yield @client_middleware if block_given?
  @client_middleware
end

#environment ⇒ String

Return the current environment.

Returns:

  • (String) —

    The environment name.



133
134
135
 
# File 'lib/cloudtasker/config.rb', line 133

def environment
  ENV['CLOUDTASKER_ENV'] || ENV['RAILS_ENV'] || ENV['RACK_ENV'] || 'development'
end

#processor_host ⇒ String

The hostname of the application processing the workers. The hostname must be reachable from Cloud Task.

Returns:

  • (String) —

    The processor host.



181
182
183
 
# File 'lib/cloudtasker/config.rb', line 181

def processor_host
  @processor_host || raise(StandardError, PROCESSOR_HOST_MISSING)
end

#processor_host=(val) ⇒ Object

Set the processor host. In the context of Rails the host will also be added to the list of authorized Rails hosts.

Parameters:

  • val (String) —

    The processor host to set.



162
163
164
165
166
167
168
169
170
171
172
173
 
# File 'lib/cloudtasker/config.rb', line 162

def processor_host=(val)
  @processor_host = val

  # Check if Rails supports host filtering
  return unless val &&
                defined?(Rails) &&
                Rails.application.config.respond_to?(:hosts) &&
                Rails.application.config.hosts&.any?

  # Add processor host to the list of authorized hosts
  Rails.application.config.hosts << val.gsub(%r{https?://}, '')
end

#processor_url ⇒ String

Return the full URL of the processor. Worker payloads will be sent to this URL.

Returns:

  • (String) —

    The processor URL.



152
153
154
 
# File 'lib/cloudtasker/config.rb', line 152

def processor_url
  File.join(processor_host, processor_path)
end

#redis_payload_storage_threshold ⇒ Integer?

Return the threshold above which job arguments must be stored in Redis instead of being sent to the backend as part of the job payload.

Return nil if redis payload storage is disabled.

Returns:

  • (Integer, nil) —

    The threshold above which payloads will be stored in Redis.



100
101
102
103
104
 
# File 'lib/cloudtasker/config.rb', line 100

def redis_payload_storage_threshold
  return nil unless store_payloads_in_redis

  store_payloads_in_redis.respond_to?(:to_i) ? store_payloads_in_redis.to_i : 0
end

#server_middleware {|@server_middleware| ... } ⇒ Cloudtasker::Middleware::Chain

Return the chain of server middlewares.

Yields:

Returns:



287
288
289
290
291
 
# File 'lib/cloudtasker/config.rb', line 287

def server_middleware
  @server_middleware ||= Middleware::Chain.new
  yield @server_middleware if block_given?
  @server_middleware
end