Class: Cloudtasker::Config

Inherits:
Object
  • Object
show all
Defined in:
lib/cloudtasker/config.rb

Overview

Holds cloudtasker configuration. See Cloudtasker#configure

Constant Summary collapse

MAX_TASK_SIZE =

Max Cloud Task size in bytes The GCP limit is 1MiB, which is 1049KB. The task formatting and headers add about 20KB.

1000 * 1000
RETRY_HEADER =

Retry header in Cloud Task responses

Definitions:

X-CloudTasks-TaskRetryCount: total number of retries (including 50x errors)
X-CloudTasks-TaskExecutionCount: number of non-50x retries (= actual number of job failures)

'X-Cloudtasks-Taskexecutioncount'
ATTEMPT_HEADER = 
'X-CloudTasks-TaskRetryCount'
TASK_ID_HEADER =

Cloud Task ID header

'X-CloudTasks-TaskName'
ENCODING_HEADER =

Content-Transfer-Encoding header in Cloud Task responses

'Content-Transfer-Encoding'
CONTENT_TYPE_HEADER =

Content Type

'Content-Type'
OIDC_AUTHORIZATION_HEADER =

OIDC Authorization header

'Authorization'
CT_AUTHORIZATION_HEADER =

Custom authentication header that does not conflict with OIDC authorization header

'X-Cloudtasker-Authorization'
CT_SIGNATURE_HEADER = 
'X-Cloudtasker-Signature'
DEFAULT_LOCATION_ID =

Default values

'us-east1'
DEFAULT_PROCESSOR_PATH = 
'/cloudtasker/run'
DEFAULT_LOCAL_SERVER_SSL_VERIFY_MODE = 
true
DEFAULT_JOB_QUEUE =

Default queue values

'default'
DEFAULT_QUEUE_CONCURRENCY = 
10
DEFAULT_QUEUE_RETRIES =

unlimited

-1 # unlimited
DEFAULT_DISPATCH_DEADLINE =

Job timeout configuration for Cloud Tasks

10 * 60
MIN_DISPATCH_DEADLINE =

10 minutes

15
MAX_DISPATCH_DEADLINE =

seconds

30 * 60
DEFAULT_ON_ERROR =

Default on_error Proc

->(error, worker) {}
DEFAULT_BASE64_ENCODE_BODY =

Default base64 encoding flag

true
WORKER_STORE_PREFIX =

Cache key prefix used to store workers in cache and retrieve them later.

'worker_store'
DEFAULT_MAX_RETRY_ATTEMPTS =

The number of times jobs will be attempted before declaring them dead.

With the default retry configuration (maxDoublings = 16 and minBackoff = 0.100s) it means that jobs will be declared dead after 20h of consecutive failing.

Note that this configuration parameter is internal to Cloudtasker and does not affect the Cloud Task queue configuration. The number of retries configured on the Cloud Task queue should be higher than the number below to also cover failures due to the instance being unreachable.

25
LOCAL_SERVER_RETRY_DELAY =

How long to wait between retries in local server mode

(ENV['CLOUDTASKER_LOCAL_RETRY_DELAY'] || 20).to_i
PROCESSOR_HOST_MISSING = 
<<~DOC
  Missing host for processing.
  Please specify a processor hostname in form of `https://some-public-dns.example.com`'
DOC
PROJECT_ID_MISSING_ERROR = 
<<~DOC
  Missing GCP project ID.
  Please specify a project ID in the cloudtasker configurator.
DOC
SECRET_MISSING_ERROR = 
<<~DOC
  Missing cloudtasker secret.
  Please specify a secret in the cloudtasker initializer or add Rails secret_key_base in your credentials
DOC
OIDC_EMAIL_MISSING_ERROR = 
<<~DOC
  Missing OpenID Connect (OIDC) service account email.
  You specified an OIDC configuration hash but the :service_account_email property is missing.
DOC

Instance Attribute Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#base64_encode_body ⇒ Boolean

Return whether to base64 encode the task body when sending to Cloud Tasks. Encoding is enabled by default to support UTF-8 content.

Returns:

  • (Boolean) —

    Whether to base64 encode the body.



318
319
320
 
# File 'lib/cloudtasker/config.rb', line 318

def base64_encode_body
  @base64_encode_body.nil? ? DEFAULT_BASE64_ENCODE_BODY : @base64_encode_body
end

#dispatch_deadline ⇒ Integer

Return the Dispatch deadline duration. Cloud Tasks will timeout the job after this duration is elapsed.

Returns:

  • (Integer) —

    The value in seconds.



230
231
232
 
# File 'lib/cloudtasker/config.rb', line 230

def dispatch_deadline
  @dispatch_deadline || DEFAULT_DISPATCH_DEADLINE
end

#gcp_location_id ⇒ String

Return the GCP location ID. Default to ‘us-east1’

Returns:

  • (String) —

    The location ID where tasks will be processed.



220
221
222
 
# File 'lib/cloudtasker/config.rb', line 220

def gcp_location_id
  @gcp_location_id || DEFAULT_LOCATION_ID
end

#gcp_project_id ⇒ String

Return the GCP project ID.

Returns:

  • (String) —

    The ID of the project for which tasks will be processed.



211
212
213
 
# File 'lib/cloudtasker/config.rb', line 211

def gcp_project_id
  @gcp_project_id || raise(StandardError, PROJECT_ID_MISSING_ERROR)
end

#gcp_queue_prefix ⇒ Object

Returns the value of attribute gcp_queue_prefix.



8
9
10
 
# File 'lib/cloudtasker/config.rb', line 8

def gcp_queue_prefix
  @gcp_queue_prefix
end

#local_server_ssl_verify ⇒ Boolean

Return the ssl verify mode for the Cloudtasker local server.

Returns:

  • (Boolean) —

    The ssl verify mode for the Cloudtasker local server.



308
309
310
 
# File 'lib/cloudtasker/config.rb', line 308

def local_server_ssl_verify
  @local_server_ssl_verify.nil? ? DEFAULT_LOCAL_SERVER_SSL_VERIFY_MODE : @local_server_ssl_verify
end

#logger ⇒ Logger, any

Return the Cloudtasker logger.

Returns:

  • (Logger, any) —

    The cloudtasker logger.



152
153
154
 
# File 'lib/cloudtasker/config.rb', line 152

def logger
  @logger ||= defined?(Rails) ? Rails.logger : ::Logger.new($stdout)
end

#max_retries ⇒ Integer

The number of times jobs will be retried. This number of retries does not include failures due to the application being unreachable.

Returns:

  • (Integer) —

    The number of retries



123
124
125
 
# File 'lib/cloudtasker/config.rb', line 123

def max_retries
  @max_retries ||= DEFAULT_MAX_RETRY_ATTEMPTS
end

#mode ⇒ <Type>

The operating mode.

- :production => process tasks via GCP Cloud Task.
- :development => process tasks locally via Redis.

Returns:

  • (<Type>) —

    <description>



134
135
136
 
# File 'lib/cloudtasker/config.rb', line 134

def mode
  @mode ||= environment == 'development' ? :development : :production
end

#oidc ⇒ Hash

Return the Open ID Connect configuration to use for tasks.

Returns:

  • (Hash) —

    The OIDC configuration

Raises:

  • (StandardError) 


271
272
273
274
275
276
277
278
279
 
# File 'lib/cloudtasker/config.rb', line 271

def oidc
  return unless @oidc
  raise(StandardError, OIDC_EMAIL_MISSING_ERROR) unless @oidc[:service_account_email]

  {
    service_account_email: @oidc[:service_account_email],
    audience: @oidc[:audience] || processor_host
  }
end

#on_dead ⇒ Proc

Return a Proc invoked whenever a worker DeadWorkerError is raised. See Cloudtasker::WorkerHandler.with_worker_handling

Returns:

  • (Proc) —

    A Proc handler



262
263
264
 
# File 'lib/cloudtasker/config.rb', line 262

def on_dead
  @on_dead || DEFAULT_ON_ERROR
end

#on_error ⇒ Proc

Return a Proc invoked whenever a worker runtime error is raised. See Cloudtasker::WorkerHandler.with_worker_handling

Returns:

  • (Proc) —

    A Proc handler



252
253
254
 
# File 'lib/cloudtasker/config.rb', line 252

def on_error
  @on_error || DEFAULT_ON_ERROR
end

#processor_path ⇒ String

The path on the host when worker payloads will be sent. Default to ‘/cloudtasker/run`

Returns:

  • (String) —

    The processor path



202
203
204
 
# File 'lib/cloudtasker/config.rb', line 202

def processor_path
  @processor_path || DEFAULT_PROCESSOR_PATH
end

#redis ⇒ Object

Returns the value of attribute redis.



8
9
10
 
# File 'lib/cloudtasker/config.rb', line 8

def redis
  @redis
end

#secret ⇒ String

Return the secret to use to sign the verification tokens attached to tasks.

Returns:

  • (String) —

    The cloudtasker secret



240
241
242
243
244
 
# File 'lib/cloudtasker/config.rb', line 240

def secret
  @secret ||= (
    defined?(Rails) && Rails.application.credentials&.dig(:secret_key_base)
  ) || raise(StandardError, SECRET_MISSING_ERROR)
end

#store_payloads_in_redis ⇒ Object

Returns the value of attribute store_payloads_in_redis.



8
9
10
 
# File 'lib/cloudtasker/config.rb', line 8

def store_payloads_in_redis
  @store_payloads_in_redis
end

Instance Method Details

#client_middleware {|@client_middleware| ... } ⇒ Cloudtasker::Middleware::Chain

Return the chain of client middlewares.

Yields:

Returns:



286
287
288
289
290
 
# File 'lib/cloudtasker/config.rb', line 286

def client_middleware
  @client_middleware ||= Middleware::Chain.new
  yield @client_middleware if block_given?
  @client_middleware
end

#environment ⇒ String

Return the current environment.

Returns:

  • (String) —

    The environment name.



143
144
145
 
# File 'lib/cloudtasker/config.rb', line 143

def environment
  ENV['CLOUDTASKER_ENV'] || ENV['RAILS_ENV'] || ENV['RACK_ENV'] || 'development'
end

#processor_host ⇒ String

The hostname of the application processing the workers. The hostname must be reachable from Cloud Task.

Returns:

  • (String) —

    The processor host.



191
192
193
 
# File 'lib/cloudtasker/config.rb', line 191

def processor_host
  @processor_host || raise(StandardError, PROCESSOR_HOST_MISSING)
end

#processor_host=(val) ⇒ Object

Set the processor host. In the context of Rails the host will also be added to the list of authorized Rails hosts.

Parameters:

  • val (String) —

    The processor host to set.



172
173
174
175
176
177
178
179
180
181
182
183
 
# File 'lib/cloudtasker/config.rb', line 172

def processor_host=(val)
  @processor_host = val

  # Check if Rails supports host filtering
  return unless val &&
                defined?(Rails) &&
                Rails.application.config.respond_to?(:hosts) &&
                Rails.application.config.hosts&.any?

  # Add processor host to the list of authorized hosts
  Rails.application.config.hosts << val.gsub(%r{https?://}, '')
end

#processor_url ⇒ String

Return the full URL of the processor. Worker payloads will be sent to this URL.

Returns:

  • (String) —

    The processor URL.



162
163
164
 
# File 'lib/cloudtasker/config.rb', line 162

def processor_url
  File.join(processor_host, processor_path)
end

#redis_payload_storage_threshold ⇒ Integer?

Return the threshold above which job arguments must be stored in Redis instead of being sent to the backend as part of the job payload.

Return nil if redis payload storage is disabled.

Returns:

  • (Integer, nil) —

    The threshold above which payloads will be stored in Redis.



110
111
112
113
114
 
# File 'lib/cloudtasker/config.rb', line 110

def redis_payload_storage_threshold
  return nil unless store_payloads_in_redis

  store_payloads_in_redis.respond_to?(:to_i) ? store_payloads_in_redis.to_i : 0
end

#server_middleware {|@server_middleware| ... } ⇒ Cloudtasker::Middleware::Chain

Return the chain of server middlewares.

Yields:

Returns:



297
298
299
300
301
 
# File 'lib/cloudtasker/config.rb', line 297

def server_middleware
  @server_middleware ||= Middleware::Chain.new
  yield @server_middleware if block_given?
  @server_middleware
end