Class: DataDrain::Storage::Local

Inherits:
Base
  • Object
show all
Defined in:
lib/data_drain/storage/local.rb

Overview

Implementación del adaptador de almacenamiento para el disco local.

Instance Attribute Summary

Attributes inherited from Base

#config

Instance Method Summary collapse

Methods inherited from Base

#initialize

Constructor Details

This class inherits a constructor from DataDrain::Storage::Base

Instance Method Details

#build_path(bucket, folder_name, partition_path) ⇒ String

Parameters:

  • bucket (String)
  • folder_name (String)
  • partition_path (String, nil)

Returns:

  • (String) 


26
27
28
 
# File 'lib/data_drain/storage/local.rb', line 26

def build_path(bucket, folder_name, partition_path)
  "#{build_path_base(bucket, folder_name, partition_path)}/**/*.parquet"
end

#destroy_partitions(bucket, folder_name, partition_keys, partitions) ⇒ Integer

Parameters:

  • bucket (String)
  • folder_name (String)
  • partition_keys (Array<Symbol>)
  • partitions (Hash)

Returns:

  • (Integer) 


48
49
50
51
52
53
54
55
56
57
58
59
60
61
 
# File 'lib/data_drain/storage/local.rb', line 48

def destroy_partitions(bucket, folder_name, partition_keys, partitions)
  path_parts = partition_keys.map do |key|
    val = partitions[key]
    val.nil? || val.to_s.empty? ? "#{key}=*" : "#{key}=#{val}"
  end

  pattern = File.join(bucket, folder_name, path_parts.join("/"))
  folders_to_delete = Dir.glob(pattern)

  return 0 if folders_to_delete.empty?

  folders_to_delete.each { |folder| FileUtils.rm_rf(folder) }
  folders_to_delete.size
end

#prepare_export_path(bucket, folder_name) ⇒ Object

Crea la jerarquía de carpetas en el disco si no existe.

Parameters:

  • bucket (String)
  • folder_name (String) 


18
19
20
 
# File 'lib/data_drain/storage/local.rb', line 18

def prepare_export_path(bucket, folder_name)
  FileUtils.mkdir_p(File.join(bucket, folder_name))
end

#setup_duckdb(connection) ⇒ Object

(DuckDB ya soporta archivos locales de forma nativa, no requiere extensiones extras)

Parameters:

  • connection (DuckDB::Connection) 


11
12
13
 
# File 'lib/data_drain/storage/local.rb', line 11

def setup_duckdb(connection)
  # No-op
end

#upload_file(local_path, bucket, s3_key, content_type: nil) ⇒ String

Returns Path absoluto al archivo destino.

Parameters:

  • local_path (String)
  • bucket (String) —

    Directorio destino

  • s3_key (String) —

    Path relativo dentro del bucket

  • content_type (String, nil) (defaults to: nil) —

    Ignorado en modo local

Returns:

  • (String) —

    Path absoluto al archivo destino



35
36
37
38
39
40
41
 
# File 'lib/data_drain/storage/local.rb', line 35

def upload_file(local_path, bucket, s3_key, content_type: nil)
  _ = content_type
  dest_path = File.join(bucket, s3_key)
  FileUtils.mkdir_p(File.dirname(dest_path))
  FileUtils.cp(local_path, dest_path)
  dest_path
end