Class: Zon::Zig::Hasher

Inherits:
Object
  • Object
show all
Defined in:
lib/zon/hasher.rb

Overview

Produces a package hash of the format: $name-$semver-$hashplus

Along with 200 bits of a SHA-256, a package hash includes:

  • package name
  • package version
  • id component of the package fingerprint
  • total unpacked size on disk

The following steps are taken to calculate the hash:

  1. Read the list of included paths from the manifest, i.e. files that are part of the package.
  2. Resolve the paths:
    • for directories: include all files that are within the directory or a child dir.
    • for files: include the file
    • for symlinks: TODO
  3. Sort the paths by:
    1. Lexographical order
    2. by length
  4. For every included file:
    • Calculate a SHA-256 over: RELATIVE_PATH_FROM_PKG_ROOT || 0x0000 || DATA
    • Record the file size in bytes
    • TODO: the 0x0000 will probably change in the future
  5. Calculate a SHA-256 sum over all calculated file hashes
    • respecting the previously mentioned sorting rules.
  6. Sum up the sizes of all files into a u32 (sizes that don't fit into a 32-bit unsingned integer will be saturated with the value 2**32 - 1)
  7. Produce the package hash

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(package_path, manifest_name: "build.zig.zon", debug: false) ⇒ Hasher

Returns a new instance of Hasher.

Raises:

  • (ArgumentError) 


36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/zon/hasher.rb', line 36

def initialize(package_path, manifest_name: "build.zig.zon", debug: false)
  # Make sure we have a valid path to work with
  @package_path = package_path
  raise ArgumentError, "not a directory '#{@package_path}'" if not Dir.exist? @package_path

  manifest_path = File.join(@package_path, manifest_name)
  raise ArgumentError, "no manifest '#{manifest_name}' in '#{@package_path}'" if not File.exist? manifest_path

  @manifest = Zon.parse(File.open(manifest_path))
  raise ArgumentError, "no 'paths' field in ZON manifest '#{manifest_path}'" if not @manifest.key? :paths
  raise ArgumentError, "no 'name' field in ZON manifest '#{manifest_path}'" if not @manifest.key? :name
  raise ArgumentError, "no 'version' field in ZON manifest '#{manifest_path}'" if not @manifest.key? :version
  raise ArgumentError, "no 'fingerprint' field in ZON manifest '#{manifest_path}'" if not @manifest.key? :fingerprint
  
  @debug = debug
  @paths = []
  @results = {}
  @total_size = 0
  @digest = nil

  @manifest[:paths].each do |path|
    full_path = File.join(@package_path, path)

    if File.directory? full_path
      f = Find.find(full_path)
      f.each do |p|
        # TODO: it seems like when "" is used as path, hidden folders
        # like .git are not considered but hidden files like .gitignore
        # are. Make this more robust by removing all hidden folders that
        # are not explicitely listed in the manifest.
        next if File.directory? p or p.include? ".git/"
        
        pstr = p.delete_prefix(@package_path)[1..]
        @paths.append(pstr) if not @paths.include? pstr
      end
    elsif File.file? full_path or File.symlink? full_path
      pstr = full_path.delete_prefix(@package_path)[1..]
      @paths.append(pstr) if not @paths.include? pstr
    end
  end

  @paths = @paths.sort_by { |str| [str, -str.length] }
end

Instance Attribute Details

#digest ⇒ Object (readonly)

Returns the value of attribute digest.



34
35
36
 
# File 'lib/zon/hasher.rb', line 34

def digest
  @digest
end

#paths ⇒ Object (readonly)

Returns the value of attribute paths.



34
35
36
 
# File 'lib/zon/hasher.rb', line 34

def paths
  @paths
end

#total_size ⇒ Object (readonly)

Returns the value of attribute total_size.



34
35
36
 
# File 'lib/zon/hasher.rb', line 34

def total_size
  @total_size
end

Class Method Details

.hash_file(package_path, path) ⇒ Object 



153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
 
# File 'lib/zon/hasher.rb', line 153

def self.hash_file(package_path, path)
  sha2 = Digest::SHA2.new    
  sha2.update path

  full_path = File.join(package_path, path)

  if File.file? full_path
    f = File.open full_path
    data = f.read
    file_size = data.bytesize
    
    # Hard-coded false executable bit: https://github.com/ziglang/zig/issues/17463
    sha2.update "\x00\x00"
    sha2.update data
  elsif File.symlink? full_path
    link_name = File.readlink full_path
    sha2.update link_name
  end

  # TODO: symlink

  {
    digest: sha2.hexdigest,
    size: file_size,
  }
end

.make_hash(digest, name, semver, id, size) ⇒ Object

Produces $name-$semver-$hashplus

  • name is the name field from a build.zig.zon manifest. It is expected to be at most 32 bytes long and a valid Zig identifier.
  • semver is the version field from a build.zig.zon manifest. It is expected to be at most 32 bytes long.
  • hashplus is a base64 (urlsafe and nopad) encoded, 33-byte string: 
    - Pacakge ID (4) || Decompressed Size (4) || Digest0, Digest1, ..., Digest24

Raises:

  • (ArgumentError) 


142
143
144
145
146
147
148
149
150
151
 
# File 'lib/zon/hasher.rb', line 142

def self.make_hash(digest, name, semver, id, size)
  raise ArgumentError, "name '#{name}' is expected to be at most 32 bytes long but it is #{name.bytesize} bytes" if name.bytesize > 32
  raise ArgumentError, "semver '#{semver}' is expected to be at most 32 bytes long but it is #{semver.bytesize} bytes" if semver.bytesize > 32

  hash_plus = [id].pack("V")
  hash_plus += [size].pack("V")
  hash_plus += [digest].pack('H*')[0..24]

  "#{name}-#{semver}-#{Base64.urlsafe_encode64(hash_plus, padding: false)}"
end

Instance Method Details

#get_id ⇒ Object

Get the ID part of the fingerprint



113
114
115
 
# File 'lib/zon/hasher.rb', line 113

def get_id
  @manifest[:fingerprint] & 0xffffffff
end

#get_saturated_size ⇒ Object

Get the calculated, total size of the unpacked package.



119
120
121
122
123
124
125
126
127
 
# File 'lib/zon/hasher.rb', line 119

def get_saturated_size
  max_uint32 = 2**32 - 1

  if @total_size > max_uint32
    max_uint32
  else
    @total_size
  end
end

#hash ⇒ Object 



88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
 
# File 'lib/zon/hasher.rb', line 88

def hash
  @total_size = 0
  @digest = nil
  
  # Hash all files individually  
  @paths.each do |str|
    @results[str] = Zon::Zig::Hasher::hash_file(@package_path, str)

    puts "file: #{@results[str][:digest]}: #{str}" if @debug
  end 

  sha2 = Digest::SHA2.new

  @paths.each do |str|
    res = @results[str]

    @total_size += res[:size]
    sha2.update [res[:digest]].pack('H*')
  end

  @digest = sha2.hexdigest
end

#name ⇒ Object 



80
81
82
 
# File 'lib/zon/hasher.rb', line 80

def name
  String(@manifest[:name])
end

#result ⇒ Object 



129
130
131
 
# File 'lib/zon/hasher.rb', line 129

def result
  Zon::Zig::Hasher::make_hash(@digest, name, version, get_id, get_saturated_size)
end

#version ⇒ Object 



84
85
86
 
# File 'lib/zon/hasher.rb', line 84

def version
  @manifest[:version]
end