Class: Arachni::RPC::Server::Framework

Inherits:
Framework show all
Includes:
MultiInstance, Utilities
Defined in:
lib/arachni/rpc/server/framework.rb,
lib/arachni/rpc/server/framework/slave.rb,
lib/arachni/rpc/server/framework/master.rb,
lib/arachni/rpc/server/framework/distributor.rb,
lib/arachni/rpc/server/framework/multi_instance.rb

Overview

Note:

Ignore:

  • Inherited methods and attributes – only public methods of this class are

    accessible over RPC.

  • `block` parameters, they are an RPC implementation detail for methods which

    perform asynchronous operations.

Wraps the framework of the local instance and the frameworks of all its slaves (when it is a Master in multi-Instance mode) into a neat, easy to handle package.

Author:

  • Tasos “Zapotek” Laskos <tasos.laskos@arachni-scanner.com>

Defined Under Namespace

Modules: Distributor, Master, MultiInstance, Slave Classes: Error

Constant Summary

Constants included from Distributor

Distributor::MAX_CONCURRENCY

Constants inherited from Framework

Framework::AUDIT_PAGE_MAX_TRIES

Instance Attribute Summary

Attributes inherited from Framework

#options

Attributes included from Framework::Parts::Audit

#failures, #http, #session, #trainer

Attributes included from Framework::Parts::Check

#checks

Attributes included from Framework::Parts::Plugin

#plugins

Attributes included from Framework::Parts::Report

#reporters

Instance Method Summary collapse

Methods included from MultiInstance

#errors, #multi_self_url, #progress, #solo?, #update_page_queue

Methods included from Master

#enslave, #has_slaves?, #master?, #set_as_master, #slave_sitrep

Methods included from Slave

#process_pages, #set_master, #slave?

Methods included from Distributor

#connect_to_instance, #each_slave, #iterator_for, #map_slaves, #slave_iterator

Methods included from Utilities

#available_port, available_port_mutex, #bytes_to_kilobytes, #bytes_to_megabytes, #caller_name, #caller_path, #cookie_decode, #cookie_encode, #cookies_from_file, #cookies_from_parser, #cookies_from_response, #exception_jail, #exclude_path?, #follow_protocol?, #form_decode, #form_encode, #forms_from_parser, #forms_from_response, #full_and_absolute_url?, #generate_token, #get_path, #hms_to_seconds, #html_decode, #html_encode, #include_path?, #links_from_parser, #links_from_response, #normalize_url, #page_from_response, #page_from_url, #parse_set_cookie, #path_in_domain?, #path_too_deep?, #port_available?, #rand_port, #random_seed, #redundant_path?, #regexp_array_match, #remove_constants, #request_parse_body, #seconds_to_hms, #skip_page?, #skip_path?, #skip_resource?, #skip_response?, #to_absolute, #uri_decode, #uri_encode, #uri_parse, #uri_parse_query, #uri_parser, #uri_rewrite

Methods inherited from Framework

#inspect, #statistics, #version

Methods included from Framework::Parts::State

#abort, #abort?, #aborted?, #aborting?, #done?, included, #pause, #pause?, #paused?, #pausing?, #reset, #reset_trainer, #restore, #resume, #running?, #scanning?, #snapshot_path, #state, #status, #status_messages, #suspend, #suspend?, #suspended?

Methods included from Framework::Parts::Data

#data, #page_queue_total_size, #push_to_page_queue, #push_to_url_queue, #sitemap, #url_queue_total_size

Methods included from Framework::Parts::Audit

#after_page_audit, #audit_page, #on_effective_page_audit, #on_page_audit

Methods included from Support::Mixins::Observable

included

Methods included from UI::Output

#debug?, #debug_level_1?, #debug_level_2?, #debug_level_3?, #debug_level_4?, #debug_off, #debug_on, #disable_only_positives, #included, #mute, #muted?, #only_positives, #only_positives?, #print_bad, #print_debug, #print_debug_backtrace, #print_debug_level_1, #print_debug_level_2, #print_debug_level_3, #print_debug_level_4, #print_error, #print_error_backtrace, #print_exception, #print_info, #print_line, #print_ok, #print_status, #print_verbose, #reroute_to_file, #reroute_to_file?, reset_output_options, #unmute, #verbose?, #verbose_on

Methods included from Framework::Parts::Platform

#list_platforms

Methods included from Framework::Parts::Report

#report_as

Methods included from Framework::Parts::Browser

#browser_cluster, #browser_cluster_job_skip_states, #host_has_browser?, #use_browsers?, #wait_for_browser_cluster?

Methods included from Framework::Parts::Scope

#accepts_more_pages?, #crawl?, #page_limit_reached?

Constructor Details

#initialize ⇒ Framework

Returns a new instance of Framework.



69
70
71
72
73
74
75
 
# File 'lib/arachni/rpc/server/framework.rb', line 69

def initialize( * )
    super

    # Override standard framework components with their RPC-server counterparts.
    @checks  = Check::Manager.new( self )
    @plugins = Plugin::Manager.new( self )
end

Instance Method Details

#busy?(&block) ⇒ Bool

Returns `true` If the system is scanning, `false` if #run hasn't been called yet or if the scan has finished.

Returns:

  • (Bool) —

    `true` If the system is scanning, `false` if #run hasn't been called yet or if the scan has finished.



116
117
118
119
120
121
122
123
124
125
 
# File 'lib/arachni/rpc/server/framework.rb', line 116

def busy?( &block )
    # If we have a block it means that it was called via RPC, so use the
    # status variable to determine if the scan is done.
    if block_given?
        block.call @prepared && status != :done
        return
    end

    !!@extended_running
end

#clean_up(&block) ⇒ Object

If the scan needs to be aborted abruptly this method takes care of any unfinished business (like signaling running plug-ins to finish).

Should be called before grabbing the #report, especially when running in multi-Instance mode, as it will take care of merging the plug-in results of all instances.

You don't need to call this if you've let the scan complete.



171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
 
# File 'lib/arachni/rpc/server/framework.rb', line 171

def clean_up( &block )
    if @rpc_cleaned_up
        # Don't shutdown the BrowserCluster here, its termination will be
        # handled by Instance#shutdown.
        block.call false if block_given?
        return false
    end

    @rpc_cleaned_up   = true
    @extended_running = false

    r = super( false )

    if !block_given?
        state.status = :done
        return r
    end

    if !has_slaves?
        state.status = :done
        block.call r
        return
    end

    foreach = proc do |instance, iter|
        instance.framework.clean_up do
            instance.plugins.results do |res|
                iter.return( !res.rpc_exception? ? res : nil )
            end
        end
    end
    after = proc do |results|
        @plugins.merge_results( results.compact )
        state.status = :done
        block.call true
    end
    map_slaves( foreach, after )
end

#error_test(str, &block) ⇒ Object 



245
246
247
248
249
250
251
 
# File 'lib/arachni/rpc/server/framework.rb', line 245

def error_test( str, &block )
    print_error str.to_s
    return block.call if !has_slaves?

    each = proc { |instance, iter| instance.framework.error_test( str ) { iter.next } }
    each_slave( each, &block )
end

#issues ⇒ Array<Hash>

Returns Issues as RPC data.

Returns:



214
215
216
 
# File 'lib/arachni/rpc/server/framework.rb', line 214

def issues
    Data.issues.sort.map(&:to_rpc_data)
end

#issues_as_hash ⇒ Array<Hash>

Returns #issues as an array of Hashes.

Returns:

See Also:



222
223
224
 
# File 'lib/arachni/rpc/server/framework.rb', line 222

def issues_as_hash
    Data.issues.sort.map(&:to_h)
end

#list_checks ⇒ Object 



106
107
108
109
110
111
 
# File 'lib/arachni/rpc/server/framework.rb', line 106

def list_checks
    super.map do |check|
        check[:issue][:severity] = check[:issue][:severity].to_s
        check
    end
end

#list_plugins ⇒ Object 



90
91
92
93
94
95
 
# File 'lib/arachni/rpc/server/framework.rb', line 90

def list_plugins
    super.map do |plugin|
        plugin[:options] = plugin[:options].map(&:to_h)
        plugin
    end
end

#list_reporters ⇒ Object 



98
99
100
101
102
103
 
# File 'lib/arachni/rpc/server/framework.rb', line 98

def list_reporters
    super.map do |reporter|
        reporter[:options] = reporter[:options].map(&:to_h)
        reporter
    end
end

#report(&block) ⇒ Report

Returns Arachni::Report#to_rpc_data.

Returns:



79
80
81
82
83
84
85
86
87
 
# File 'lib/arachni/rpc/server/framework.rb', line 79

def report( &block )
    # If a block is given it means the call was form an RPC client.
    if block_given?
        block.call super.to_rpc_data
        return
    end

    super
end

#run ⇒ Bool

Starts the scan.

Returns:

  • (Bool) —

    `false` if already running, `true` otherwise.



141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
 
# File 'lib/arachni/rpc/server/framework.rb', line 141

def run
    # Return if we're already running.
    return false if busy?

    @extended_running = true

    # Prepare the local instance (runs plugins and starts the timer).
    prepare

    # Start the scan  -- we can't block the RPC server so we're using a Thread.
    # Thread.abort_on_exception = true
    Thread.new do
        if !solo?
            multi_run
        else
            super
        end
    end

    true
end

#self_url ⇒ String

Returns URL of this instance.

Returns:

  • (String) —

    URL of this instance.



230
231
232
233
234
235
236
 
# File 'lib/arachni/rpc/server/framework.rb', line 230

def self_url
    options.dispatcher.external_address ||= options.rpc.server_address

    @self_url ||= options.dispatcher.external_address ?
        "#{options.dispatcher.external_address }:#{options.rpc.server_port}" :
        options.rpc.server_socket
end

#sitemap_entries(from_index = 0) ⇒ Hash<String=>Integer>

Parameters:

  • from_index (Integer) (defaults to: 0) —

    Get sitemap entries after this index.

Returns:



131
132
133
134
135
 
# File 'lib/arachni/rpc/server/framework.rb', line 131

def sitemap_entries( from_index = 0 )
    return {} if sitemap.size <= from_index + 1

    Hash[sitemap.to_a[from_index..-1] || {}]
end

#token ⇒ String

Returns This instance's RPC token.

Returns:

  • (String) —

    This instance's RPC token.



240
241
242
 
# File 'lib/arachni/rpc/server/framework.rb', line 240

def token
    options.datastore.token
end