Module: Beaker::DSL::InstallUtils::PEUtils

Includes:
AIODefaults, PEDefaults, PuppetUtils, WindowsUtils
Included in:
PE
Defined in:
lib/beaker-pe/install/pe_utils.rb

Overview

This module contains methods to help installing/upgrading PE builds - including Higgs installs

To mix this is into a class you need the following:

  • a method hosts that yields any hosts implementing Host‘s interface to act upon.

  • a method options that provides an options hash, see Options::OptionsHash

  • the module Roles that provides access to the various hosts implementing Host‘s interface to act upon

  • the module Wrappers the provides convenience methods for Command creation

Constant Summary collapse

MEEP_CUTOVER_VERSION =

Version of PE when we switched from legacy installer to MEEP.

'2016.2.0'
MEEP_CLASSIFICATION_VERSION =

Version of PE when we switched to using meep for classification instead of PE node groups

'2018.2.0'
DEFAULT_MEEP_CLASSIFICATION =

PE-18799 temporary default used for meep classification check while we navigate the switchover. PE-18718 switch flag to true once beaker-pe, beaker-answers, beaker-pe-large-environments and pe_acceptance_tests are ready

false
MANAGE_PUPPET_SERVICE_VERSION =

Version of PE in which PE is managing the agent service

'2018.1.0'
MEEP_DATA_DIR = 
'/etc/puppetlabs/enterprise'
PE_CONF_FILE = 
"#{MEEP_DATA_DIR}/conf.d/pe.conf"
NODE_CONF_PATH = 
"#{MEEP_DATA_DIR}/conf.d/nodes"
BEAKER_MEEP_TMP = 
"pe_conf"

Constants included from PEDefaults

Beaker::DSL::InstallUtils::PEDefaults::PE_DEFAULTS

Instance Method Summary collapse

Methods included from PEDefaults

#add_pe_defaults_on, #add_platform_pe_defaults, #remove_pe_defaults_on, #remove_platform_pe_defaults

Instance Method Details

#check_console_status_endpoint(host) ⇒ Object

Note:

Uses the global option’s :pe_console_status_attempts value to determine how many times it’s going to retry the check with fibonacci back offs.

Checks Console Status Endpoint, failing the test if the endpoints don’t report a running state.

Parameters:

  • host (Host) —

    Host to check status on

Returns:

  • nil



1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1747

def check_console_status_endpoint(host)
  return true if version_is_less(host['pe_ver'], '2015.2.0')

  attempts_limit = options[:pe_console_status_attempts] || 9
  # Workaround for PE-14857. The classifier status service at the
  # default level is broken in 2016.1.1. Instead we need to query
  # the classifier service at critical level and check for service
  # status
  query_params = (host['pe_ver'] == '2016.1.1' ? '?level=critical' : '')
  step 'Check Console Status Endpoint' do
    match = repeat_fibonacci_style_for(attempts_limit) do
      output = on(host, "curl -s -k https://localhost:4433/status/v1/services#{query_params} --cert /etc/puppetlabs/puppet/ssl/certs/#{host}.pem --key /etc/puppetlabs/puppet/ssl/private_keys/#{host}.pem --cacert /etc/puppetlabs/puppet/ssl/certs/ca.pem", :accept_all_exit_codes => true)
      begin
        output = JSON.parse(output.stdout)
        match = output['classifier-service']['state'] == 'running'
        match = match && output['rbac-service']['state'] == 'running'
        match && output['activity-service']['state'] == 'running'
      rescue JSON::ParserError
        false
      end
    end
    fail_test 'Console services took too long to start' if !match
  end
end

#check_puppetdb_status_endpoint(host) ⇒ Object 



1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1721

def check_puppetdb_status_endpoint(host)
  if version_is_less(host['pe_ver'], '2016.1.0')
    return true
  end
  Timeout.timeout(60) do
    match = nil
    while not match
      output = on(host, "curl -s http://localhost:8080/pdb/meta/v1/version", :accept_all_exit_codes => true)
      match = output.stdout =~ /version.*\d+\.\d+\.\d+/
      sleep 1
    end
  end
rescue Timeout::Error
  fail_test "PuppetDB took too long to start"
end

#config_hosts_for_proxy_access(hosts) ⇒ Object

Configure the master to use a proxy and drop unproxied connections



808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 808

def config_hosts_for_proxy_access hosts
  hosts.each do |host|
    step "Configuring #{host} to use proxy" do
      @osmirror_host = "osmirror.delivery.puppetlabs.net"
      puts "Grabbing IP for osmirror.delivery.puppetlabs.net"
      @osmirror_host_ip = IPSocket.getaddress(@osmirror_host)
      @delivery_host = "artifactory.delivery.puppetlabs.net"
      puts "Grabbing IP for artifactory.delivery.puppetlabs.net"
      @delivery_host_ip = IPSocket.getaddress(@delivery_host)
      @test_forge_host = "api-forge-aio02-petest.puppet.com"
      puts "Grabbing IP for api-forge-aio02-petest.puppet.com"
      @test_forge_host_ip = IPSocket.getaddress(@test_forge_host)
      @github_host = "github.com"
      puts "Grabbing IP for github.com"
      @github_host_ip = IPSocket.getaddress(@github_host)
      @proxy_ip = @options[:proxy_ip]
      @proxy_hostname = @options[:proxy_hostname]

      #sles does not support the -I all-ip-addresses flag
      hostname_flag = host.host_hash[:platform].include?("sles") ? '-i' : '-I'
      @master_ip = on master, "hostname #{hostname_flag} | tr '\n' ' '"

      on host, "echo \"#{@proxy_ip}  #{@proxy_hostname}\" >> /etc/hosts"
      on host, "echo \"#{@master_ip.stdout}  #{master.connection.vmhostname}\" >> /etc/hosts"
      on host, "echo \"#{@osmirror_host_ip}    #{@osmirror_host}\" >> /etc/hosts"
      on host, "echo \"#{@delivery_host_ip}    #{@delivery_host}\" >> /etc/hosts"
      on host, "echo \"#{@test_forge_host_ip}    #{@test_forge_host}\" >> /etc/hosts"
      on host, "echo \"#{@github_host_ip}    #{@github_host}\" >> /etc/hosts"

      on host, "iptables -A OUTPUT -p tcp -d #{master.connection.vmhostname} -j ACCEPT"
      # Treat these hosts as if they were outside the puppet lan
      on host, "iptables -A OUTPUT -p tcp -d #{@osmirror_host_ip} -j DROP"
      on host, "iptables -A OUTPUT -p tcp -d #{@delivery_host_ip} -j DROP"
      on host, "iptables -A OUTPUT -p tcp -d #{@test_forge_host_ip} -j DROP"
      # The next two lines are for our production and test k8s test runners
      on host, "iptables -A OUTPUT -p tcp -d 10.236.112.0/20 -j ACCEPT"
      on host, "iptables -A OUTPUT -p tcp -d 10.220.0.0/16 -j ACCEPT"
      # The next two lines clear the rest of the internal puppet lan
      on host, "iptables -A OUTPUT -p tcp -d 10.16.0.0/16 -j ACCEPT"
      on host, "iptables -A OUTPUT -p tcp -d 10.32.0.0/16 -j ACCEPT"
      # And for GCP scratchpad hosts
      on host, "iptables -A OUTPUT -p tcp -d 10.253.0.0/16 -j ACCEPT"
      # This allows udp on a port bundler requires
      on host, 'iptables -A OUTPUT -p udp -m udp --dport 53 -j ACCEPT'
      # Next two lines allow host to access itself via localhost or 127.0.0.1
      on host, 'iptables -A INPUT -i lo -j ACCEPT'
      on host, 'iptables -A OUTPUT -o lo -j ACCEPT'
      #Opens up port that git uses
      on host, "iptables -A OUTPUT -p tcp -d #{@github_host_ip} -j ACCEPT"
      on host, "iptables -A INPUT -p tcp -d #{@github_host_ip} --dport 9143 -j ACCEPT"

      #Platform9
      on host, "iptables -A OUTPUT -p tcp -d 10.234.0.0/16 -j ACCEPT"
      #enterprise.delivery.puppetlabs.net network, required if running from your work laptop over the network
      on host, "iptables -A OUTPUT -p tcp -d 10.0.25.0/16 -j ACCEPT"

      on host, "iptables -A OUTPUT -p tcp --dport 3128 -d #{@proxy_hostname} -j ACCEPT"
      on host, "iptables -P OUTPUT DROP"
      # Verify we can reach osmirror via the proxy
      on host, "curl --proxy #{@proxy_hostname}:3128 http://#{@osmirror_host}", :acceptable_exit_codes => [0]
      # Verify we can't reach it without the proxy
      on host, "curl -k http://#{@osmirror_host} -m 5", :acceptable_exit_codes => [28]
      # For ubuntu we configure Apt to use a proxy globally
      if host.host_hash[:platform].include?("ubuntu")
        on host, "echo 'Acquire::http::Proxy \"http://'#{@proxy_hostname}':3128/\";' >> /etc/apt/apt.conf"
        on host, "echo 'Acquire::https::Proxy \"http://'#{@proxy_hostname}':3128/\";' >> /etc/apt/apt.conf"
      # For SLES we configure ENV variables to use a proxy, then set no_proxy on master and possible CM
      elsif host.host_hash[:platform].include?("sles")
        on host, 'rm /etc/sysconfig/proxy'
        on host, 'echo "PROXY_ENABLED=\"yes\"" >> /etc/sysconfig/proxy'
        on host, "echo 'HTTP_PROXY=\"http://#{@proxy_hostname}:3128\"' >> /etc/sysconfig/proxy"
        on host, "echo 'HTTPS_PROXY=\"http://#{@proxy_hostname}:3128\"' >> /etc/sysconfig/proxy"
        #Needs to not use proxy on the host itself, and master (in order to download the agent)
        no_proxy_list="localhost,127.0.0.1,#{host.hostname},#{master.hostname}"
        if any_hosts_as?('compile_master')
          no_proxy_list.concat(",#{compile_master}")
        end
        on host, "echo \"NO_PROXY='#{no_proxy_list}'\" >> /etc/sysconfig/proxy"
      # For Redhat/Centos we configre Yum globally to use a proxy
      else
        on host, "echo 'proxy=http://#{@proxy_hostname}:3128' >> /etc/yum.conf"
      end
    end
  end
end

#configure_puppet_agent_service(parameters) ⇒ Object

In PE versions >= 2018.1.0, allows you to configure the puppet agent service for all nodes.

Parameters:

  • parameters (Hash) —

    • agent profile parameters

Options Hash (parameters):

  • :managed (Boolean) —

    • whether or not to manage the

    agent resource at all (Optional, defaults to true).

  • :ensure (String) —

    • ‘stopped’, ‘running’

  • :enabled (Boolean) —

    • whether the service will be

    enabled (for restarts)

Raises:

  • (StandardError) —

    if master version is less than 2017.1.0



2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2127

def configure_puppet_agent_service(parameters)
  raise(StandardError, "Can only manage puppet service in PE versions >= #{MANAGE_PUPPET_SERVICE_VERSION}; tried for #{master['pe_ver']}") if version_is_less(master['pe_ver'], MANAGE_PUPPET_SERVICE_VERSION)
  puppet_managed = parameters.include?(:managed) ? parameters[:managed] : true
  puppet_ensure = parameters[:ensure]
  puppet_enabled = parameters[:enabled]

  msg = puppet_managed ?
    "Configure agents '#{puppet_ensure}' and #{puppet_enabled ? 'enabled' : 'disabled'}" :
    "Do not manage agents"

  step msg do
    # PE-18799 and remove this conditional
    if use_meep_for_classification?(master[:pe_ver], options)
      class_name = 'pe_infrastructure::agent'
    else
      class_name = 'puppet_enterprise::profile::agent'
    end

    # update pe conf
    update_pe_conf({
      "#{class_name}::puppet_service_managed" => puppet_managed,
      "#{class_name}::puppet_service_ensure" => puppet_ensure,
      "#{class_name}::puppet_service_enabled" => puppet_enabled,
    })
  end
end

#create_agent_specified_arrays(hosts) ⇒ Array<Host>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Note:

should only be called against versions 4.0+, as this method assumes AIO packages will be required.

Note:

agent_only hosts with the :pe_ver setting < 4.0 will not be included in the agent_only array, as AIO install can only happen in versions > 4.0

Builds the agent_only and not_agent_only arrays needed for installation.

Parameters:

  • hosts (Array<Host>) —

    hosts to split up into the arrays

Returns:

  • (Array<Host>, Array<Host>) —

    the array of hosts to do an agent_only install on and the array of hosts to do our usual install methods on



1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1681

def create_agent_specified_arrays(hosts)
  hosts_agent_only = []
  hosts_not_agent_only = []
  non_agent_only_roles = %w(master database dashboard console frictionless)
  hosts.each do |host|
    if host['roles'].none? {|role| non_agent_only_roles.include?(role) }
      if !aio_version?(host)
        hosts_not_agent_only << host
      else
        hosts_agent_only << host
      end
    else
      hosts_not_agent_only << host
    end
  end
  return hosts_agent_only, hosts_not_agent_only
end

#create_or_update_node_conf(host, parameters, node_conf_path = NODE_CONF_PATH) ⇒ Object

Creates a new /etc/puppetlabs/enterprise/conf.d/nodes/*.conf file for the given host’s certname, and adds the passed parameters, or updates with the passed parameters if the file already exists.

Does not remove an empty file.

Parameters:

  • host (Beaker::Host) —

    to create a node file for

  • parameters (Hash) —

    of key value pairs to add to the nodes conf file

  • node_conf_path (String) (defaults to: NODE_CONF_PATH) —

    defaults to /etc/puppetlabs/enterprise/conf.d/nodes



2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2292

def create_or_update_node_conf(host, parameters, node_conf_path = NODE_CONF_PATH)
  node_conf_file = "#{node_conf_path}/#{host.node_name}.conf"
  step "Create or Update #{node_conf_file} with #{parameters}" do
    if !master.file_exist?(node_conf_file)
      if !master.file_exist?(node_conf_path)
        # potentially create the nodes directory
        on(master, "mkdir #{node_conf_path}")
      end
      # The hocon gem will create a list of comma separated parameters
      # on the same line unless we start with something in the file.
      create_remote_file(master, node_conf_file, %Q|{\n}\n|)
      on(master, "chown pe-puppet #{node_conf_file}")
    end
    update_pe_conf(parameters, node_conf_file)
  end
end

#deploy_frictionless_to_master(host) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Classify the master so that it can deploy frictionless packages for a given host. This function does nothing when using meep for classification.

Parameters:

  • host (Host) —

    The host to install pacakges for



510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 510

def deploy_frictionless_to_master(host)
  return if use_meep_for_classification?(master[:pe_ver], options)

  # For some platforms (e.g, redhatfips), packaging_platfrom is set and should
  # be used as the primary source of truth for the platform string.
  platform = host['packaging_platform'] || host['platform']
  klass = platform.gsub(/-/, '_').gsub(/\./,'')
  if host['platform'] =~ /windows/
    if host['template'] =~ /i386/
      klass = "pe_repo::platform::windows_i386"
    else
      klass = "pe_repo::platform::windows_x86_64"
    end
  elsif platform =~ /aix-.*-power/ && !version_is_less(master[:pe_ver], '2019.0.3')
    klass = "pe_repo::platform::aix_power"
  else
    klass = "pe_repo::platform::#{klass}"
  end
  if version_is_less(host['pe_ver'], '3.8')
    # use the old rake tasks
    on dashboard, "cd /opt/puppet/share/puppet-dashboard && /opt/puppet/bin/bundle exec /opt/puppet/bin/rake nodeclass:add[#{klass},skip]"
    on dashboard, "cd /opt/puppet/share/puppet-dashboard && /opt/puppet/bin/bundle exec /opt/puppet/bin/rake node:add[#{master},,,skip]"
    on dashboard, "cd /opt/puppet/share/puppet-dashboard && /opt/puppet/bin/bundle exec /opt/puppet/bin/rake node:addclass[#{master},#{klass}]"
    on master, puppet("agent -t"), :acceptable_exit_codes => [0,2]
  else
    _console_dispatcher = get_console_dispatcher_for_beaker_pe!

    # Add pe_repo packages to 'PE Master' group
    node_group = _console_dispatcher.get_node_group_by_name('PE Master')

    # add the pe_repo platform class if it's not already present
    if node_group
      if !node_group['classes'].include?(klass)
        node_group['classes'][klass] = {}
        _console_dispatcher.create_new_node_group_model(node_group)

        # The puppet agent run that will download the agent tarballs to the master can sometimes fail with
        # curl errors if there is a network hiccup. Use beakers `retry_on` method to retry up to
        # three times to avoid failing the entire test pipeline due to a network blip
        retry_opts = {
          :desired_exit_codes => [0,2],
          :max_retries => 3,
          # Beakers retry_on method wants the verbose value to be a string, not a bool.
          :verbose => 'true'
        }
        retry_on(master, puppet("agent -t"), retry_opts)

        # If we are connecting through loadbalancer, download the agent tarballs to compile masters
        if lb_connect_loadbalancer_exists?
          hosts.each do |h|
            if h['roles'].include?('compile_master') || h['roles'].include?('pe_compiler')
              retry_on(h, puppet("agent -t"), retry_opts)
            end
          end
        end
      end
    else
      raise "Failed to add pe_repo packages, PE Master node group is not available"
    end
  end
end

#determine_higgs_answer(pe_ver) ⇒ String

Determines the answer to supply to the command line installer in order to load up Higgs

Returns:

  • (String) —

    One of, ‘Y’, ‘1’, ‘2’

    'Y'
  Pre-meep install of Higgs (Before PE 2016.2.0)
'1'
  meep before PE 2018.1.3 (PE 2016.2.0 -> PE 2018.1.2)
'2'
  Any meep PE 2018.1.3 or greater



1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1894

def determine_higgs_answer(pe_ver)
  if(use_meep?(pe_ver))
    if(version_is_less(pe_ver, '2018.1.3'))
      return '1'
    elsif(version_is_less(pe_ver, '2019.0.2'))
      return '2'
    else
      return '3'
    end
  else
    return 'Y'
  end
end

#determine_install_type(hosts, opts) ⇒ Symbol

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Determine what kind of install is being performed

Examples:

determine_install_type(hosts, {:type => :install, :pe_ver => '2017.2.0'})

Parameters:

  • hosts (Array<Host>) —

    The sorted hosts to install or upgrade PE on

  • opts (Hash{Symbol=>Symbol, String}) —

    The options for how to install or upgrade PE

Returns:

  • (Symbol) —

    One of :generic, :simple_monolithic, :simple_split, :pe_managed_postgres :simple_monolithic

    returned when installing >=2016.4 with a monolithic master and
any number of frictionless agents

    :simple_split

    returned when installing >=2016.4 with a split install and any
number of frictionless agents

    :pe_managed_postgres

    returned when installing PE with postgres being managed on a node
that is different then the database node

    :generic

    returned for any other install or upgrade



703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 703

def determine_install_type(hosts, opts)
  # Do a generic install if this is masterless, not all the same PE version, an upgrade, or earlier than 2016.4
  return :generic if opts[:masterless]
  return :generic if hosts.map {|host| host['pe_ver']}.uniq.length > 1
  return :generic if (opts[:type] == :upgrade) && (hosts.none? {|host| host['roles'].include?('pe_postgres')})
  return :generic if version_is_less(opts[:pe_ver] || hosts.first['pe_ver'], '2016.4')
  #PE-20610 Do a generic install for old versions on windows that needs msi install because of PE-18351
  return :generic if hosts.any? {|host| host['platform'] =~ /windows/ && install_via_msi?(host)}

  mono_roles = ['master', 'database', 'dashboard']
  if has_all_roles?(hosts.first, mono_roles) && hosts.drop(1).all? {|host| host['roles'].include?('frictionless')}
    if hosts.first['template'] =~ /-preload/ && opts[:type] != :upgrade
      :simple_monolithic_install_with_preload
    else
      :simple_monolithic
    end
  elsif hosts[0]['roles'].include?('master') && hosts[1]['roles'].include?('database') && hosts[2]['roles'].include?('dashboard') && hosts.drop(3).all? {|host| host['roles'].include?('frictionless')}
    :simple_split
  elsif hosts.any? {|host| host['roles'].include?('pe_postgres')}
    :pe_managed_postgres
  else
    :generic
  end
end

#do_higgs_install(host, opts) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Perform a Puppet Enterprise Higgs install up until web browser interaction is required, runs on linux hosts only.

Examples:

do_higgs_install(master, {:pe_dir => path, :pe_ver => version})

Parameters:

  • host (Host) —

    The host to install higgs on

  • opts (Hash{Symbol=>Symbol, String}) —

    The options

Options Hash (opts):

  • :pe_dir (String) —

    Default directory or URL to pull PE package from (Otherwise uses individual hosts pe_dir)

  • :pe_ver (String) —

    Default PE version to install (Otherwise uses individual hosts pe_ver)

  • :fetch_local_then_push_to_host (Boolean) —

    determines whether you use Beaker as the middleman for this (true), or curl the file from the host (false; default behavior)

Raises:

  • (StandardError) —

    When installation times out



1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1925

def do_higgs_install host, opts
  use_all_tar = ENV['PE_USE_ALL_TAR'] == 'true'
  platform = use_all_tar ? 'all' : host['platform']
  version = host['pe_ver'] || opts[:pe_ver]
  host['dist'] = "puppet-enterprise-#{version}-#{platform}"

  use_all_tar = ENV['PE_USE_ALL_TAR'] == 'true'
  host['pe_installer'] ||= 'puppet-enterprise-installer'
  host['working_dir'] = host.tmpdir(Time.new.strftime("%Y-%m-%d_%H.%M.%S"))

  fetch_pe([host], opts)

  host['higgs_file'] = "higgs_#{File.basename(host['working_dir'])}.log"

  prepare_host_installer_options(host)
  on host, higgs_installer_cmd(host), opts

  #wait for output to host['higgs_file']
  #we're all done when we find this line in the PE installation log
  if version_is_less(opts[:pe_ver] || host['pe_ver'], '2016.3')
    higgs_re = /Please\s+go\s+to\s+https:\/\/.*\s+in\s+your\s+browser\s+to\s+continue\s+installation/m
  else
    higgs_re = /o\s+to\s+https:\/\/.*\s+in\s+your\s+browser\s+to\s+continue\s+installation/m
  end
  res = Result.new(host, 'tmp cmd')
  tries = 10
  attempts = 0
  prev_sleep = 0
  cur_sleep = 1
  while (res.stdout !~ higgs_re) and (attempts < tries)
    res = on host, "cd #{host['working_dir']}/#{host['dist']} && cat #{host['higgs_file']}", :accept_all_exit_codes => true
    attempts += 1
    sleep( cur_sleep )
    prev_sleep = cur_sleep
    cur_sleep = cur_sleep + prev_sleep
  end

  if attempts >= tries
    raise "Failed to kick off PE (Higgs) web installation"
  end
end

#do_install(hosts, opts = {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Note:

on windows, the :ruby_arch host parameter can determine in addition

Note:

for puppet-agent install options, refer to FOSSUtils#install_puppet_agent_pe_promoted_repo_on

Perform a Puppet Enterprise upgrade or install to other settings whether the 32 or 64bit install is used

Examples:

do_install(hosts, {:type => :upgrade, :pe_dir => path, :pe_ver => version, :pe_ver_win =>  version_win})

Parameters:

  • hosts (Array<Host>) —

    The hosts to install or upgrade PE on

  • opts (Hash{Symbol=>Symbol, String}) (defaults to: {}) —

    The options

Options Hash (opts):

  • :pe_dir (String) —

    Default directory or URL to pull PE package from (Otherwise uses individual hosts pe_dir)

  • :pe_ver (String) —

    Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)

  • :pe_ver_win (String) —

    Default PE version to install or upgrade to on Windows hosts (Otherwise uses individual Windows hosts pe_ver)

  • :type (Symbol) — default: :install —

    One of :upgrade or :install

  • :set_console_password (Boolean) —

    Should we set the PE console password in the answers file? Used during upgrade only.

  • :answers (Hash<String>) —

    Pre-set answers based upon ENV vars and defaults (See Options::Presets.env_vars)

  • :fetch_local_then_push_to_host (Boolean) —

    determines whether you use Beaker as the middleman for this (true), or curl the file from the host (false; default behavior)

  • :masterless (Boolean) —

    Are we performing a masterless installation?



653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 653

def do_install hosts, opts = {}
  # detect the kind of install we're doing
  install_type = determine_install_type(hosts, opts)
  verify_network_resources(hosts, options[:net_diag_hosts])
  verify_vm_resources(hosts)
  if opts[:use_proxy]
    config_hosts_for_proxy_access(hosts - hosts_as('proxy'))
  end
  case install_type
  when :pe_managed_postgres
    do_install_pe_with_pe_managed_external_postgres(hosts,opts)
  when :simple_monolithic
    simple_monolithic_install(hosts.first, hosts.drop(1), opts)
  when :simple_monolithic_install_with_preload
    simple_monolithic_install_with_preload(hosts.first, hosts.drop(1), opts)
  when :simple_split
    # This isn't implemented yet, so just do a generic install instead
    #simple_split_install(hosts, opts)
    generic_install(hosts, opts)
  else
    generic_install(hosts, opts)
  end
end

#do_install_pe_with_pe_managed_external_postgres(hosts, opts) ⇒ Object

Installs PE with a PE managed external postgres



1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1989

def do_install_pe_with_pe_managed_external_postgres(hosts, opts)
  pe_infrastructure = select_hosts({:roles => ['master', 'dashboard', 'database', 'pe_postgres']}, hosts)
  non_infrastructure = hosts.reject{|host| pe_infrastructure.include? host}

  is_upgrade = (original_pe_ver(hosts[0]) != hosts[0][:pe_ver])
  step "Setup tmp installer directory and pe.conf" do

    prepare_hosts(pe_infrastructure,opts)
    register_feature_flags!(opts)
    fetch_pe(pe_infrastructure,opts)

    [master, database, dashboard, pe_postgres].uniq.each do |host|
      configure_type_defaults_on(host)
      prepare_host_installer_options(host)

      unless is_upgrade
        setup_pe_conf(host, hosts, opts)
      end
    end
  end

  unless is_upgrade
    step "Initial master install, expected to fail due to RBAC database not being initialized" do
      begin
        execute_installer_cmd(master, opts)
      rescue Beaker::Host::CommandFailure => e
        unless is_expected_pe_postgres_failure?(master)
          raise "Install on master failed in an unexpected manner"
        end
      end
    end
  end

  step "Install/Upgrade postgres service on pe-postgres node" do
    execute_installer_cmd(pe_postgres, opts)
  end

  step "Finish install/upgrade on infrastructure" do
      [master, database, dashboard].uniq.each do |host|
        execute_installer_cmd(host, opts)
      end
  end

  step "Stop agent service on infrastructure nodes" do
    stop_agent_on(pe_infrastructure, :run_in_parallel => true)
  end

  step "First puppet run on infrastructure + postgres node" do
    [master, database, dashboard, pe_postgres].uniq.each do |host|
      on host, 'puppet agent -t', :acceptable_exit_codes => [0,2]
    end
  end

  if(non_infrastructure.size > 0)
    install_agents_only_on(non_infrastructure, opts)

    step "Run puppet to setup mcollective and pxp-agent" do
      on master, 'puppet agent -t', :acceptable_exit_codes => [0,2]
      run_puppet_on_non_infrastructure_nodes(non_infrastructure)
    end

  end
  step "Run puppet a second time on the primary to populate services.conf (PE-19054)" do
    on master, 'puppet agent -t', :acceptable_exit_codes => [0,2]
  end
end

#download_pe_conf_if_master(host) ⇒ Object 



2336
2337
2338
2339
2340
2341
2342
2343
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2336

def download_pe_conf_if_master(host)
  if host['roles'].include?('master')
    step "Downloading generated #{MEEP_DATA_DIR}/conf.d locally" do
      # scp conf.d over from master
      scp_from(host, "#{MEEP_DATA_DIR}/conf.d", BEAKER_MEEP_TMP)
    end
  end
end

#execute_installer_cmd(host, opts) ⇒ Object

This calls the installer command on the host in question



291
292
293
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 291

def execute_installer_cmd(host, opts)
  on host, installer_cmd(host, opts)
end

#feature_flag?(flag, opts) ⇒ Boolean

Tests if a feature flag has been set in the answers hash provided to beaker options. Assumes a ‘feature_flags’ hash is present in the answers and looks for flag within it.

Parameters:

  • flag —

    String flag to lookup

  • opts —

    Hash options hash to inspect

Returns:

  • (Boolean) —

    true if flag is true or ‘true’ in the feature_flags hash, false otherwise. However, returns nil if there is no flag in the answers hash at all



1403
1404
1405
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1403

def feature_flag?(flag, opts)
  Beaker::DSL::InstallUtils::FeatureFlags.new(opts).flag?(flag)
end

#fetch_and_push_pe(host, path, filename, extension, local_dir = 'tmp/pe') ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Grabs the pe file from a remote host to the machine running Beaker, then scp’s the file out to the host.

Parameters:

  • host (Host) —

    The host to install on

  • path (String) —

    path to the install file

  • filename (String) —

    the filename of the pe file (without the extension)

  • extension (String) —

    the extension of the pe file

  • local_dir (String) (defaults to: 'tmp/pe') —

    the directory to store the pe file in on the Beaker-running-machine

Returns:

  • nil



2087
2088
2089
2090
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2087

def fetch_and_push_pe(host, path, filename, extension, local_dir='tmp/pe')
  fetch_http_file("#{path}", "#{filename}#{extension}", local_dir)
  scp_to host, "#{local_dir}/#{filename}#{extension}", host['working_dir']
end

#fetch_pe(hosts, opts) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Determine the PE package to download/upload per-host, download/upload that package onto the host and unpack it.

Parameters:

  • hosts (Array<Host>) —

    The hosts to download/upload and unpack PE onto

  • opts (Hash{Symbol=>Symbol, String}) —

    The options

Options Hash (opts):

  • :pe_dir (String) —

    Default directory or URL to pull PE package from (Otherwise uses individual hosts pe_dir)

  • :pe_ver (String) —

    Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)

  • :pe_ver_win (String) —

    Default PE version to install or upgrade to on Windows hosts (Otherwise uses individual Windows hosts pe_ver)

  • :fetch_local_then_push_to_host (Boolean) —

    determines whether you use Beaker as the middleman for this (true), or curl the file from the host (false; default behavior)



491
492
493
494
495
496
497
498
499
500
501
502
503
504
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 491

def fetch_pe(hosts, opts)
  hosts.each do |host|
    # We install Puppet from the master for frictionless installs, so we don't need to *fetch* anything
    next if host['roles'].include?('frictionless') && (! version_is_less(opts[:pe_ver] || host['pe_ver'], '3.2.0'))

    if host['platform'] =~ /windows/
      fetch_pe_on_windows(host, opts)
    elsif host['platform'] =~ /osx/
      fetch_pe_on_mac(host, opts)
    else
      fetch_pe_on_unix(host, opts)
    end
  end
end

#fetch_pe_on_mac(host, opts) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Determine the PE package to download/upload on a mac host, download/upload that package onto the host. Assumed file name format: puppet-enterprise-3.3.0-rc1-559-g97f0833-osx-10.9-x86_64.dmg.

Parameters:

  • host (Host) —

    The mac host to download/upload and unpack PE onto

  • opts (Hash{Symbol=>Symbol, String}) —

    The options

Options Hash (opts):

  • :pe_dir (String) —

    Default directory or URL to pull PE package from (Otherwise uses individual hosts pe_dir)

  • :fetch_local_then_push_to_host (Boolean) —

    determines whether you use Beaker as the middleman for this (true), or curl the file from the host (false; default behavior)



305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 305

def fetch_pe_on_mac(host, opts)
  path = host['pe_dir'] || opts[:pe_dir]
  local = File.directory?(path)
  filename = "#{host['dist']}"
  extension = ".dmg"
  if local
    if not File.exists?("#{path}/#{filename}#{extension}")
      raise "attempting installation on #{host}, #{path}/#{filename}#{extension} does not exist"
    end
    scp_to host, "#{path}/#{filename}#{extension}", "#{host['working_dir']}/#{filename}#{extension}"
  else
    if not link_exists?("#{path}/#{filename}#{extension}")
      raise "attempting installation on #{host}, #{path}/#{filename}#{extension} does not exist"
    end
    if opts[:fetch_local_then_push_to_host]
      fetch_and_push_pe(host, path, filename, extension)
    else
      curlopts = opts[:use_proxy] ? " --proxy #{opts[:proxy_hostname]}:3128" : ""
      on host, "cd #{host['working_dir']}; curl -L -O #{path}/#{filename}#{extension}#{curlopts}"
    end
  end
end

#fetch_pe_on_unix(host, opts) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Determine the PE package to download/upload on a unix style host, download/upload that package onto the host and unpack it.

Parameters:

  • host (Host) —

    The unix style host to download/upload and unpack PE onto

  • opts (Hash{Symbol=>Symbol, String}) —

    The options

Options Hash (opts):

  • :pe_dir (String) —

    Default directory or URL to pull PE package from (Otherwise uses individual hosts pe_dir)

  • :fetch_local_then_push_to_host (Boolean) —

    determines whether you use Beaker as the middleman for this (true), or curl the file from the host (false; default behavior)



394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 394

def fetch_pe_on_unix(host, opts)
  path = host['pe_dir'] || opts[:pe_dir]
  local = File.directory?(path)
  filename = "#{host['dist']}"
  if local
    extension = File.exist?("#{path}/#{filename}.tar.gz") ? ".tar.gz" : ".tar"
    unless File.exist?("#{path}/#{filename}#{extension}")
      raise "attempting installation on #{host}, #{path}/#{filename}#{extension} does not exist"
    end
    scp_to host, "#{path}/#{filename}#{extension}", "#{host['working_dir']}/#{filename}#{extension}"
    if extension =~ /gz/
      on host, "cd #{host['working_dir']}; gunzip #{filename}#{extension}"
      gpg_key_overwrite(host, 'tarball')
    end
    if extension =~ /tar/
      on host, "cd #{host['working_dir']}; tar -xvf #{filename}.tar"
      gpg_key_overwrite(host, 'tarball')
    end
  else
    if host['platform'] =~ /eos/
      extension = '.swix'
    else
      extension = link_exists?("#{path}/#{filename}.tar.gz") ? ".tar.gz" : ".tar"
    end
    if not link_exists?("#{path}/#{filename}#{extension}")
      raise "attempting installation on #{host}, #{path}/#{filename}#{extension} does not exist"
    end

    if host['platform'] =~ /eos/
      host.get_remote_file("#{path}/#{filename}#{extension}")
    else
      if opts[:fetch_local_then_push_to_host]
        fetch_and_push_pe(host, path, filename, extension)
        command_file_push = 'cat '
      else
        curlopts = opts[:use_proxy] ? "--proxy #{opts[:proxy_hostname]}:3128 " : ""
        command_file_push = "curl --fail --location --output #{filename}#{extension} #{curlopts}#{path}/"
      end

      retry_requirements = {
        desired_exit_codes: [0],
        max_retries: 3,
        verbose: 'true'
      }
      fetch_tarball_command = "cd #{host['working_dir']}; #{command_file_push}#{filename}#{extension}"
      retry_on(host, fetch_tarball_command, retry_requirements)
      on host, "cd #{host['working_dir']}; tar -xvf #{filename}#{extension}"

      gpg_key_overwrite(host, 'tarball')
    end
  end
end

#fetch_pe_on_windows(host, opts) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Determine the PE package to download/upload on a windows host, download/upload that package onto the host. Assumed file name format: puppet-enterprise-3.3.0-rc1-559-g97f0833.msi

Parameters:

  • host (Host) —

    The windows host to download/upload and unpack PE onto

  • opts (Hash{Symbol=>Symbol, String}) —

    The options

Options Hash (opts):

  • :pe_dir (String) —

    Default directory or URL to pull PE package from (Otherwise uses individual hosts pe_dir)

  • :pe_ver_win (String) —

    Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)

  • :fetch_local_then_push_to_host (Boolean) —

    determines whether you use Beaker as the middleman for this (true), or curl the file from the host (false; default behavior)



358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 358

def fetch_pe_on_windows(host, opts)
  path = host['pe_dir'] || opts[:pe_dir]
  local = File.directory?(path)
  filename = "#{host['dist']}"
  extension = ".msi"
  if local
    if not File.exists?("#{path}/#{filename}#{extension}")
      raise "attempting installation on #{host}, #{path}/#{filename}#{extension} does not exist"
    end
    scp_to host, "#{path}/#{filename}#{extension}", "#{host['working_dir']}/#{filename}#{extension}"
  else
    if not link_exists?("#{path}/#{filename}#{extension}")
      raise "attempting installation on #{host}, #{path}/#{filename}#{extension} does not exist"
    end
    if opts[:fetch_local_then_push_to_host]
      fetch_and_push_pe(host, path, filename, extension)
      on host, "cd #{host['working_dir']}; chmod 644 #{filename}#{extension}"
    elsif host.is_cygwin?
      curlopts = opts[:use_proxy] ? " --proxy #{opts[:proxy_hostname]}:3128" : ""
      on host, "cd #{host['working_dir']}; curl -L -O #{path}/#{filename}#{extension}#{curlopts}"
    else
      on host, powershell("$webclient = New-Object System.Net.WebClient;  $webclient.DownloadFile('#{path}/#{filename}#{extension}','#{host['working_dir']}\\#{filename}#{extension}')")
    end
  end
end

#frictionless_agent_installer_cmd(host, opts, pe_version) ⇒ String

Generate the command line string needed to from a frictionless puppet-agent install on this host in a PE environment.

Parameters:

  • host (Host) —

    The host to install puppet-agent onto

  • opts (Hash) —

    The full beaker options

  • pe_version (String) —

    The PE version string for capabilities testing

Options Hash (opts):

  • :use_puppet_ca_cert (Boolean) — default: false —

    if true the command will reference the local puppet ca cert to verify the master when obtaining the installation script

Returns:

  • (String) —

    of the commands to be executed for the install



170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 170

def frictionless_agent_installer_cmd(host, opts, pe_version)
  # PE 3.4 introduced the ability to pass in config options to the bash
  # script in the form of <section>:<key>=<value>
  frictionless_install_opts = []
  if host.has_key?('frictionless_options') and !  version_is_less(pe_version, '3.4.0')
    # since we have options to pass in, we need to tell the bash script
    host['frictionless_options'].each do |section, settings|
      settings.each do |key, value|
        frictionless_install_opts << "#{section}:#{key}=#{value}"
      end
    end
  end

  # PE 2018.1.0 introduced a pe_repo flag that will determine what happens during the frictionless install
  # Current support in beaker-pe is for:
  # --puppet-service-debug, when running puppet service enable, the debug flag is passed into puppt service
  if (host[:puppet_service_debug_flag] == true and ! version_is_less(pe_version, '2018.1.0'))
    frictionless_install_opts << '--puppet-service-debug'
  end

  # If this is an agent node configured to connect to the loadbalancer
  # using 'lb_connect' role, then use loadbalancer in the download url
  # instead of master
  downloadhost = master
  if host['roles'].include?('lb_connect')
    downloadhost = get_lb_downloadhost(host)
  end

  pe_debug = host[:pe_debug] || opts[:pe_debug] ? ' -x' : ''
  use_puppet_ca_cert = host[:use_puppet_ca_cert] || opts[:use_puppet_ca_cert]

  # PE 2019.8.3 had a bug in the frictionless install of osx-10.14 and osx-10.15
  # We need to do a bit of a hacky process to install the agent
  if host['platform'] =~ /osx-10\.1(4|5)/ && (pe_version.eql?('2019.8.3') || pe_version.eql?('2019.8.4'))
    return "curl -kO https://#{downloadhost}:8140/packages/current/#{host['platform']}.bash && bash #{host['platform']}.bash"
  end

  # PE 2019.8.5 has an issue with the GPG key that does not allow el-5 and sles-11 to install the puppet-agent
  if host['platform'] =~ /(el-5)|(sles-11)/ && pe_version.eql?('2019.8.5')
    on(host, "curl --remote-name --location http://yum.puppet.com/RPM-GPG-KEY-puppet-20250406")
    on(host, "rpm --import RPM-GPG-KEY-puppet-20250406")
  end

  if host['platform'] =~ /windows/ then
    if use_puppet_ca_cert
      frictionless_install_opts << '-UsePuppetCA'
      cert_validator = %Q{\\$callback = {param(\\$sender,[System.Security.Cryptography.X509Certificates.X509Certificate]\\$certificate,[System.Security.Cryptography.X509Certificates.X509Chain]\\$chain,[System.Net.Security.SslPolicyErrors]\\$sslPolicyErrors);\\$CertificateType=[System.Security.Cryptography.X509Certificates.X509Certificate2];\\$CACert=\\$CertificateType::CreateFromCertFile('#{host['puppetpath']}/ssl/certs/ca.pem') -as \\$CertificateType;\\$chain.ChainPolicy.ExtraStore.Add(\\$CACert);return \\$chain.Build(\\$certificate)};[Net.ServicePointManager]::ServerCertificateValidationCallback = \\$callback}
    else
      cert_validator = '[Net.ServicePointManager]::ServerCertificateValidationCallback = {\\$true}'
    end
    if version_is_less(pe_version, '2019.1.0') || require_tlsv1?(host) then
      protocol_to_use =''
    else
      protocol_to_use = '[System.Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12'
    end

    cmd = %Q{powershell -NoProfile -NonInteractive -NoLogo -ExecutionPolicy Bypass -c "cd #{host['working_dir']};#{protocol_to_use};#{cert_validator};\\$webClient = New-Object System.Net.WebClient;\\$webClient.DownloadFile('https://#{downloadhost}:8140/packages/current/install.ps1', '#{host['working_dir']}/install.ps1');#{host['working_dir']}/install.ps1 -verbose #{frictionless_install_opts.join(' ')}"}
  else
    curl_opts = %w{-O}
    if version_is_less(pe_version, '2019.1.0') || require_tlsv1?(host)
      curl_opts << '--tlsv1'
    end
    if use_puppet_ca_cert
      curl_opts << '--cacert /etc/puppetlabs/puppet/ssl/certs/ca.pem'
    else
      curl_opts << '-k'
    end

    cmd = "FRICTIONLESS_TRACE='true'; export FRICTIONLESS_TRACE; cd #{host['working_dir']} && curl #{curl_opts.join(' ')} https://#{downloadhost}:8140/packages/current/install.bash && bash#{pe_debug} install.bash #{frictionless_install_opts.join(' ')}".strip
  end

  return cmd
end

#generate_installer_conf_file_for(host, hosts, opts) ⇒ BeakerAnswers::Answers

Generates a Beaker Answers object for the passed host and creates the answer or pe.conf configuration file on the host needed for installation.

Expects the host to have been set, which is where the configuration will be written to, and will run MEEP or legacy depending on host

Parameters:

  • host (Beaker::Host) —

    The host to create a configuration file on

  • hosts (Array<Beaker::Host]) —

    All of the hosts to be configured

  • opts (Hash) —

    The Beaker options hash

Returns:

  • (BeakerAnswers::Answers) —

    the generated answers object



1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1594

def generate_installer_conf_file_for(host, hosts, opts)
  possible_mco_enabled_setting = get_mco_setting(hosts)
  opts ||= {}
  opts = possible_mco_enabled_setting.deep_merge(opts)
  beaker_answers_opts = setup_beaker_answers_opts(host, opts)
  answers = BeakerAnswers::Answers.create(
    opts[:pe_ver] || host['pe_ver'], hosts, beaker_answers_opts
  )
  configuration = answers.installer_configuration_string(host)

  step "Generate the #{host['pe_installer_conf_file']} on #{host}" do
    logger.debug(configuration)
    create_remote_file(host, host['pe_installer_conf_file'], configuration)
  end

  answers
end

#generic_install(hosts, opts = {}) ⇒ Object 



1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1120

def generic_install hosts, opts = {}
  step "Installing PE on a generic set of hosts"

  masterless = opts[:masterless]
  opts[:type] = opts[:type] || :install
  unless masterless
    pre30database = version_is_less(opts[:pe_ver] || database['pe_ver'], '3.0')
    pre30master = version_is_less(opts[:pe_ver] || master['pe_ver'], '3.0')
  end

  pe_versions = ( [] << opts['pe_ver'] << hosts.map{ |host| host['pe_ver'] } ).flatten.compact
  agent_only_check_needed = version_is_less('3.99', max_version(pe_versions, '3.8'))
  if agent_only_check_needed
    hosts_agent_only, hosts_not_agent_only = create_agent_specified_arrays(hosts)
  else
    hosts_agent_only, hosts_not_agent_only = [], hosts.dup
  end

  # On January 5th, 2017, the extended GPG key has expired. Rather then
  # every few months updating this gem to point to a new key for PE versions
  # less then PE 2016.4.0 we are going to just ignore the warning when installing
  ignore_gpg_key_warning_on_hosts(hosts, opts)

  # Set PE distribution for all the hosts, create working dir
  prepare_hosts(hosts_not_agent_only, opts)

  fetch_pe(hosts_not_agent_only, opts)

  install_hosts = hosts.dup
  unless masterless
    # If we're installing a database version less than 3.0, ignore the database host
    install_hosts.delete(database) if pre30database and database != master and database != dashboard
  end

  install_hosts.each do |host|
    solaris_key_chain_fix

    if agent_only_check_needed && hosts_agent_only.include?(host) || install_via_msi?(host)
      host['type'] = 'aio'
      install_params = {
        :puppet_agent_version => get_puppet_agent_version(host, opts),
        :puppet_agent_sha => host[:puppet_agent_sha] || opts[:puppet_agent_sha],
        :pe_ver => host[:pe_ver] || opts[:pe_ver],
        :puppet_collection => host[:puppet_collection] || opts[:puppet_collection],
        :pe_promoted_builds_url => host[:pe_promoted_builds_url] || opts[:pe_promoted_builds_url]
      }
      if host['platform'] =~ /sles-11/
        install_rpm_on_sles11_host(host, install_params[:puppet_agent_version], opts)
      else
        install_params.delete(:pe_promoted_builds_url) if install_params[:pe_promoted_builds_url].nil?
        install_puppet_agent_pe_promoted_repo_on(host, install_params)
      end
      # 1 since no certificate found and waitforcert disabled
      acceptable_exit_codes = [0, 1]
      acceptable_exit_codes << 2 if opts[:type] == :upgrade
      if masterless
        configure_type_defaults_on(host)
        on host, puppet_agent('-t'), :acceptable_exit_codes => acceptable_exit_codes
      else
        setup_defaults_and_config_helper_on(host, master, acceptable_exit_codes)
      end
    #Windows allows frictionless installs starting with PE Davis, if frictionless we need to skip this step
    elsif (host['platform'] =~ /windows/ && !(host['roles'].include?('frictionless')) || install_via_msi?(host))
      opts = { :debug => host[:pe_debug] || opts[:pe_debug] }
      msi_path = "#{host['working_dir']}\\#{host['dist']}.msi"
      install_msi_on(host, msi_path, {}, opts)

      # 1 since no certificate found and waitforcert disabled
      acceptable_exit_codes = 1
      if masterless
        configure_type_defaults_on(host)
        on host, puppet_agent('-t'), :acceptable_exit_codes => acceptable_exit_codes
      else
        setup_defaults_and_config_helper_on(host, master, acceptable_exit_codes)
      end
    else
      # We only need answers if we're using the classic installer
      version = host['pe_ver'] || opts[:pe_ver]
      if host['roles'].include?('frictionless') &&  (! version_is_less(version, '3.2.0'))
        # If We're *not* running the classic installer, we want
        # to make sure the master has packages for us.
        if host['packaging_platform'] != master['packaging_platform'] # only need to do this if platform differs
          deploy_frictionless_to_master(host)
        end
        install_ca_cert_on(host, opts)
        on host, installer_cmd(host, opts)
        configure_type_defaults_on(host)
      elsif host['platform'] =~ /osx|eos/
        # If we're not frictionless, we need to run the OSX special-case
        on host, installer_cmd(host, opts)
        acceptable_codes = host['platform'] =~ /osx/ ? [1] : [0, 1]
        setup_defaults_and_config_helper_on(host, master, acceptable_codes)
      else
        prepare_host_installer_options(host)
        register_feature_flags!(opts)
        setup_pe_conf(host, hosts, opts)

        on host, installer_cmd(host, opts)
        configure_type_defaults_on(host)
        download_pe_conf_if_master(host)
      end
    end
    # On each agent, we ensure the certificate is signed
    if !masterless
      if [master, database, dashboard].include?(host) && use_meep?(host['pe_ver'])
        # This step is not necessary for the core pe nodes when using meep
      else
        step "Sign certificate for #{host}" do
          sign_certificate_for(host)
        end
      end
    end
    # then shut down the agent
    step "Shutting down agent for #{host}" do
      stop_agent_on(host)
    end
  end

  unless masterless
    # Wait for PuppetDB to be totally up and running (post 3.0 version of pe only)
    sleep_until_puppetdb_started(database) unless pre30database

    step "First puppet agent run" do
      # Run the agent once to ensure everything is in the dashboard
      install_hosts.each do |host|
        # waitforlock in case stopping the agent run after stopping the agent service
        # takes a little longer than usual
        on host, puppet_agent("-t #{waitforlock_flag(host)}"), :acceptable_exit_codes => [0,2]

        # Workaround for PE-1105 when deploying 3.0.0
        # The installer did not respect our database host answers in 3.0.0,
        # and would cause puppetdb to be bounced by the agent run. By sleeping
        # again here, we ensure that if that bounce happens during an upgrade
        # test we won't fail early in the install process.
        if host == database && ! pre30database
          sleep_until_puppetdb_started(database)
          check_puppetdb_status_endpoint(database)
        end
        if host == dashboard
          check_console_status_endpoint(host)
        end
        #Workaround for windows frictionless install, see BKR-943 for the reason
        if (host['platform'] =~ /windows/) and (host['roles'].include? 'frictionless')
          remove_client_datadir(host)
        end
      end
    end

    # only appropriate for pre-3.9 builds
    if version_is_less(master[:pe_ver], '3.99')
      if pre30master
        task = 'nodegroup:add_all_nodes group=default'
      else
        task = 'defaultgroup:ensure_default_group'
      end
      on dashboard, "/opt/puppet/bin/rake -sf /opt/puppet/share/puppet-dashboard/Rakefile #{task} RAILS_ENV=production"
    end

    if manage_puppet_service?(master[:pe_ver], options)
      configure_puppet_agent_service(:ensure => 'stopped', :enabled => false)
    end

    step "Final puppet agent run" do
      # Now that all hosts are in the dashbaord, run puppet one more
      # time to configure mcollective
      install_hosts.each do |host|
        # waitforlock in case stopping the agent run after stopping the agent service
        # takes a little longer than usual
        on host, puppet_agent("-t #{waitforlock_flag(host)}"), :acceptable_exit_codes => [0,2]
        # To work around PE-14318 if we just ran puppet agent on the
        # database node we will need to wait until puppetdb is up and
        # running before continuing
        if host == database && ! pre30database
          sleep_until_puppetdb_started(database)
          check_puppetdb_status_endpoint(database)
        end
        if host == dashboard
          check_console_status_endpoint(host)
        end
      end
    end
  end
end

#get_console_dispatcher_for_beaker_pe(raise_exception = false) ⇒ Object

Being able to modify PE’s classifier requires the Scooter gem and helpers which are in beaker-pe-large-environments.



2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2094

def get_console_dispatcher_for_beaker_pe(raise_exception = false)
  # XXX RE-8616, once scooter is public, we can remove this and just
  # reference ConsoleDispatcher directly.
  if !respond_to?(:get_dispatcher)
    begin
      require 'scooter'
      Scooter::HttpDispatchers::ConsoleDispatcher.new(dashboard)
    rescue LoadError => e
      logger.notify('WARNING: gem scooter is required for frictionless installation post 3.8')
      raise e if raise_exception

      return nil
    end
  else
    get_dispatcher
  end
end

#get_console_dispatcher_for_beaker_pe! ⇒ Object

Will raise a LoadError if unable to require Scooter.



2113
2114
2115
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2113

def get_console_dispatcher_for_beaker_pe!
  get_console_dispatcher_for_beaker_pe(true)
end

#get_lb_downloadhost(host) ⇒ Object

Returns loadbalancer if host is an agent and loadbalancer has lb_connect role

Parameters:

  • agent (Host) —

    host with lb_connect role



124
125
126
127
128
129
130
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 124

def get_lb_downloadhost(host)
  downloadhost = master
  if !host['roles'].include?('loadbalancer') &&  lb_connect_loadbalancer_exists?
    downloadhost = loadbalancer
  end
  downloadhost
end

#get_mco_setting(hosts) ⇒ Object

PE 2018.1.0 has mco disabled by default. If we are running hosts with roles hub or spoke then we intend to test mco. In this case we need to change a setting in pe.conf to allow mco to be enabled.



1572
1573
1574
1575
1576
1577
1578
1579
1580
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1572

def get_mco_setting(hosts)
  pe_version = hosts[0]['pe_ver']
  if (!version_is_less(pe_version, '2018.1') && version_is_less(pe_version, '2018.1.999'))
        if (hosts.any? {|h| h['roles'].include?('hub') || h['roles'].include?('spoke')})
          return {:answers => { 'pe_install::disable_mco' => false }}
        end
  end
  return {}
end

#get_puppet_agent_version(host, local_options = {}) ⇒ String

Note:

This method does have a side-effect: if it reads the ‘aio_agent_version` property from master, it will store it in the local options hash so that it won’t have to do this more than once.

Gets the puppet-agent version, hopefully from the host or local options. Will fall back to reading the ‘aio_agent_version` property on the master if neither of those two options are passed

Parameters:

  • host (Beaker::Host) —

    Host to get puppet-agent for

  • local_options (Hash{Symbol=>String}) (defaults to: {}) —

    local method options hash

Returns:

  • (String) —

    puppet-agent version to install

Raises:

  • (ArgumentError) 


1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1366

def get_puppet_agent_version(host, local_options={})
  puppet_agent_version = host[:puppet_agent_version] || local_options[:puppet_agent_version]
  return puppet_agent_version if puppet_agent_version
  log_prefix = "No :puppet_agent_version in host #{host} or local options."
  fail_message = "#{log_prefix} Could not read facts from master to determine puppet_agent_version"
  # we can query the master because do_install is called passing
  # the {#sorted_hosts}, so we know the master will be installed
  # before the agents
  facts_result = on(master, 'puppet facts')
  raise ArgumentError, fail_message if facts_result.exit_code != 0
  facts_hash = JSON.parse(facts_result.stdout.chomp)
  # In Puppet 7, facts are at the top level of the hash. Before that
  # version, they were wrapped inside a `values` key. Since we're
  # trying to determine the agent version here, we can't just switch
  # our behavior by agent version, so we check both possible locations.
  puppet_agent_version = facts_hash['aio_agent_version'] || facts_hash.dig('values', 'aio_agent_version')
  raise ArgumentError, fail_message if puppet_agent_version.nil?
  logger.warn("#{log_prefix} Read puppet-agent version #{puppet_agent_version} from master")
  # saving so that we don't have to query the master more than once
  local_options[:puppet_agent_version] = puppet_agent_version
  puppet_agent_version
end

#get_unwrapped_pe_conf_value(key, pe_conf_path = PE_CONF_FILE) ⇒ Object

Returns a Ruby object of any root key in pe.conf.

Parameters:

  • key (String) —

    to lookup

  • pe_conf_path (String) (defaults to: PE_CONF_FILE) —

    defaults to /etc/puppetlabs/enterprise/conf.d/pe.conf

Returns:

  • a Ruby object of any root key in pe.conf.



2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2272

def get_unwrapped_pe_conf_value(key, pe_conf_path = PE_CONF_FILE)
  file_contents = on(master, "cat #{pe_conf_path}").stdout
  # Seem to need to use ConfigFactory instead of ConfigDocumentFactory
  # to get something that we can read values from?
  doc = Hocon::ConfigFactory.parse_string(file_contents)
  hocon_key = quoted_hocon_key(key)
  doc.has_path?(hocon_key) ?
    doc.get_value(hocon_key).unwrapped :
    nil
end

#gpg_key_overwrite(host, location) ⇒ Object

PE-32680, GPG expired on older PE versions, need to update with a new GPG key on the primary server only affects PE versions 2019.8.4 and earlier, and only needed for debian, ubuntu, and sles agent platforms If someone is using this gem and is not on the PE private network, they need to download the new private GPG key and host it somewhere, then set the URL as an enviromental variable GPG_URL.

Parameters:

  • host (Host) —

    to see if we need to update the gpg key

  • location (String) —

    of the GPG key we intend to overwrite



453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 453

def gpg_key_overwrite(host, location)
  gpg_url = ENV['GPG_URL'] || 'https://artifactory.delivery.puppetlabs.net/artifactory/generic__local/extended_gpg_key.asc'
  case location
  when 'tarball'
    path_to_gpg_key = "#{host['working_dir']}/#{host['dist']}/packages/GPG-KEY-puppet"
  when 'pe_repo'
    path_to_gpg_key = '/opt/puppetlabs/puppet/modules/pe_repo/files/GPG-KEY-puppet'
  when 'pe_repo_env'
    path_to_gpg_key = '/opt/puppetlabs/server/data/environments/enterprise/modules/pe_repo/files/GPG-KEY-puppet'
  else
    raise(StandardError, "gpg_key_overwrite requires a valid location: tarball, or pe_repo. #{location} was supplied")
  end

  if (host['roles'].include?('master') || host['roles'].include?('pe_postgres')) && version_is_less(host[:pe_ver], '2019.8.5') && hosts.any? {|agent| agent['platform'] =~ /(debian)|(ubuntu)|(sles)/}
    on(host, "rm -f #{path_to_gpg_key}")
    on(host, "curl #{gpg_url} --output #{path_to_gpg_key}")
    if location == 'pe_repo'
      gpg_key_overwrite(host, 'pe_repo_env')
    elsif location == 'pe_repo_env'
      on host, puppet('agent -t'), :acceptable_exit_codes => [0,2]
    end
  end
end

#has_all_roles?(host, roles) ⇒ Boolean

Returns:

  • (Boolean) 


677
678
679
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 677

def has_all_roles?(host, roles)
  roles.all? {|role| host['roles'].include?(role)}
end

#higgs_installer_cmd(host) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Create the Higgs install command string based upon the host and options settings. Installation command will be run as a background process. The output of the command will be stored in the provided host.

Parameters:

  • host (Host) —

    The host that Higgs is to be installed on The host object must have the ‘working_dir’, ‘dist’ and ‘pe_installer’ field set correctly.



1880
1881
1882
1883
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1880

def higgs_installer_cmd host
  higgs_answer = determine_higgs_answer(host['pe_ver'])
  "cd #{host['working_dir']}/#{host['dist']} ; nohup ./#{host['pe_installer']} <<<#{higgs_answer} > #{host['higgs_file']} 2>&1 &"
end

#ignore_gpg_key_warning_on_hosts(hosts, opts) ⇒ Object

For PE 3.8.5 to PE 2016.1.2 they have an expired gpg key. This method is for deb nodes to ignore the gpg-key expiration warning



1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1474

def ignore_gpg_key_warning_on_hosts(hosts, opts)
  hosts.each do |host|
    # RPM based platforms do not seem to be effected by an expired GPG key,
    # while deb based platforms are failing.
    if host['platform'] =~ /debian|ubuntu/
      host_ver = host['pe_ver'] || opts['pe_ver']

      if version_is_less(host_ver, '3.8.7') || (!version_is_less(host_ver, '2015.2.0') && version_is_less(host_ver, '2016.4.0'))
        on(host, "echo 'APT { Get { AllowUnauthenticated \"1\"; }; };' >> /etc/apt/apt.conf")
      end
    end
  end
end

#install_agents_only_on(agent_nodes, opts) ⇒ Object

Method to install just the agent nodes This method can be called only after installing PE on infrastructure nodes

Parameters:

  • agent (Array) —

    only nodes from Beaker hosts

  • opts (Hash) —

    The Beaker options hash



2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2349

def install_agents_only_on(agent_nodes, opts)
  solaris_key_chain_fix
  unless agent_nodes.empty?
    configure_type_defaults_on(agent_nodes)

     step "Setup frictionless installer on the master" do
       agent_nodes.each do |agent|
         # If We're *not* running the classic installer, we want
         # to make sure the master has packages for us.
         if agent['packaging_platform'] != master['packaging_platform'] # only need to do this if platform differs
           deploy_frictionless_to_master(agent)
         end
       end
     end

    # pe_postgres node needs new gpg key
    if hosts.any? {|host| host['roles'].include?('pe_postgres')}
      gpg_key_overwrite(pe_postgres, 'pe_repo')
    end

    # pe_repo needs updated with gpg key if sles/ubuntu/debian agent will be used
    if hosts.any? {|host| host['platform'] =~ /(sles)|(ubuntu)|(debian)/}
      gpg_key_overwrite(master, 'pe_repo')
    end

     step "Install agents" do
       block_on(agent_nodes, {:run_in_parallel => true}) do |host|
         install_ca_cert_on(host, opts)
         on(host, installer_cmd(host, opts))
       end
     end

     step "Sign agent certificates" do
       # This will sign all cert requests
       sign_certificate_for(agent_nodes)
     end

     step "Stop puppet agents to avoid interfering with tests" do
       stop_agent_on(agent_nodes, :run_in_parallel => true)
     end

     # waitforlock in case stopping the agent run after stopping the agent service
     # takes a little longer than usual
     step "Run puppet on all agent nodes" do
       block_on(agent_nodes, {:run_in_parallel => true}) do |host|
         on host, puppet_agent("-t #{waitforlock_flag(host)}"), :acceptable_exit_codes => [0,2]
       end
     end

     #Workaround for windows frictionless install, see BKR-943
     agent_nodes.select {|agent| agent['platform'] =~ /windows/}.each do |agent|
       remove_client_datadir(agent)
     end
  end
end

#install_ca_cert_on(host, opts) ⇒ Object

If host or opts has the :use_puppet_ca_cert flag set, then push the master’s ca cert onto the given host at /etc/puppetlabs/puppet/ssl/certs/ca.pem.

This in turn allows frictionless_agent_installer_cmd to generate an install which references the cert to verify the master when downloading resources.



94
95
96
97
98
99
100
101
102
103
104
105
106
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 94

def install_ca_cert_on(host, opts)
  if host[:use_puppet_ca_cert] || opts[:use_puppet_ca_cert]
    @cert_cache_dir ||= Dir.mktmpdir("master_ca_cert")
    local_cert_copy = "#{@cert_cache_dir}/ca.pem"
    step "Copying master ca.pem to agent for secure frictionless install" do
      agent_ca_pem_dir = "#{host['puppetpath']}/ssl/certs"
      master_ca_pem_path = "/etc/puppetlabs/puppet/ssl/certs/ca.pem"
      scp_from(master, master_ca_pem_path , @cert_cache_dir) unless File.exist?(local_cert_copy)
      on(host, "mkdir -p #{agent_ca_pem_dir}")
      scp_to(host, local_cert_copy, agent_ca_pem_dir)
    end
  end
end

#install_higgs(higgs_host = master) ⇒ Object

Note:

Either pe_ver and pe_dir should be set in the ENV or each host should have pe_ver and pe_dir set individually. Install file names are assumed to be of the format puppet-enterprise-VERSION-PLATFORM.(tar)|(tar.gz).

Install Higgs up till the point where you need to continue installation in a web browser, defaults to execution on the master node.

Examples:

install_higgs

Parameters:

  • higgs_host (Host) (defaults to: master) —

    The host to install Higgs on (supported on linux platform only)



1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1976

def install_higgs( higgs_host = master )
  #process the version files if necessary
  master['pe_dir'] ||= options[:pe_dir]
  master['pe_ver'] = master['pe_ver'] || options['pe_ver'] ||
    Beaker::Options::PEVersionScraper.load_pe_version(master[:pe_dir] || options[:pe_dir], options[:pe_version_file])
  if higgs_host['platform'] =~ /osx|windows/
    raise "Attempting higgs installation on host #{higgs_host.name} with unsupported platform #{higgs_host['platform']}"
  end
  #send in the global options hash
  do_higgs_install higgs_host, options
end

#install_pe ⇒ Object

Install PE based on global hosts with global options

See Also:



1717
1718
1719
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1717

def install_pe
  install_pe_on(hosts, options)
end

#install_pe_on(install_hosts, opts) ⇒ Object

Note:

Either pe_ver and pe_dir should be set in the ENV or each host should have pe_ver and pe_dir set individually. Install file names are assumed to be of the format puppet-enterprise-VERSION-PLATFORM.(tar)|(tar.gz) for Unix like systems and puppet-enterprise-VERSION.msi for Windows systems.

Note:

For further installation parameters (such as puppet-agent install) options, refer to #do_install documentation

Install PE based upon host configuration and options

Examples:

install_pe_on(hosts, {})

Parameters:

  • install_hosts (Host, Array<Host>) —

    One or more hosts to act upon

  • opts (Hash{Symbol=>String}) —

    Options to alter execution.

Options Hash (opts):

  • :silent (Boolean) — default: false —

    Do not produce log output

  • :acceptable_exit_codes (Array<Fixnum>) — default: [0] —

    An array (or range) of integer exit codes that should be considered acceptable. An error will be thrown if the exit code does not match one of the values in this list.

  • :accept_all_exit_codes (Boolean) — default: false —

    Consider all exit codes as passing.

  • :dry_run (Boolean) — default: false —

    Do not actually execute any commands on the SUT

  • :stdin (String) — default: nil —

    Input to be provided during command execution on the SUT.

  • :pty (Boolean) — default: false —

    Execute this command in a pseudoterminal.

  • :expect_connection_failure (Boolean) — default: false —

    Expect this command to result in a connection failure, reconnect and continue execution.

  • :environment (Hash{String=>String}) — default: {} —

    These will be treated as extra environment variables that should be set before running the command.

  • :masterless (Boolean) —

    Are we performing a masterless installation?

  • :puppet_agent_version (String) —

    Version of puppet-agent to install. Required for PE agent only hosts on 4.0+

  • :puppet_agent_sha (String) —

    The sha of puppet-agent to install, defaults to puppet_agent_version. Required for PE agent only hosts on 4.0+

  • :pe_ver (String) —

    The version of PE (will also use host), defaults to ‘4.0’

  • :puppet_collection (String) —

    The puppet collection for puppet-agent install.



1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1794

def install_pe_on(install_hosts, opts)
  confine_block(:to, {}, install_hosts) do
    sorted_hosts.each do |host|
      #process the version files if necessary
      host['pe_dir'] ||= opts[:pe_dir]
      if host['platform'] =~ /windows/
        # we don't need the pe_version if:
        # * master pe_ver > 4.0
        if not (!opts[:masterless] && master[:pe_ver] && !version_is_less(master[:pe_ver], '3.99'))
          host['pe_ver'] ||= Beaker::Options::PEVersionScraper.load_pe_version(host[:pe_dir] || opts[:pe_dir], opts[:pe_version_file_win])
        else
          # inherit the master's version
          host['pe_ver'] ||= master[:pe_ver]
        end
      else
        host['pe_ver'] ||= Beaker::Options::PEVersionScraper.load_pe_version(host[:pe_dir] || opts[:pe_dir], opts[:pe_version_file])
      end
    end
    do_install sorted_hosts, opts
  end
end

#install_puppet_agent_pe_promoted_repo_on(hosts, opts) ⇒ Object

Install shared repo of the puppet-agent on the given host(s). Downloaded from location of the form PE_PROMOTED_BUILDS_URL/PE_VER/puppet-agent/AGENT_VERSION/repo

Parameters:

  • hosts (Host, Array<Host>, String, Symbol) —

    One or more hosts to act upon, or a role (String or Symbol) that identifies one or more hosts.

  • opts (Hash{Symbol=>String}) —

    An options hash

Options Hash (opts):

  • :puppet_agent_version (String) —

    The version of puppet-agent to install, defaults to ‘latest’

  • :pe_ver (String) —

    The version of PE (will also use host), defaults to ‘4.0’

  • :copy_base_local (String) —

    Directory where puppet-agent artifact will be stored locally (default: ‘tmp/repo_configs’)

  • :copy_dir_external (String) —

    Directory where puppet-agent artifact will be pushed to on the external machine (default: ‘/root’)

  • :puppet_collection (String) —

    Defaults to ‘PC1’

  • :pe_promoted_builds_url (String) —

    Base URL to pull artifacts from



1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1072

def install_puppet_agent_pe_promoted_repo_on(hosts, opts)
  opts[:puppet_agent_version] ||= 'latest'

  block_on hosts do |host|
    pe_ver = host[:pe_ver] || opts[:pe_ver] || '4.0.0-rc1'
    opts = sanitize_opts(opts)
    opts[:download_url] =
      "#{opts[:pe_promoted_builds_url]}/puppet-agent/#{pe_ver}/#{opts[:puppet_agent_version]}/repos"
    opts[:copy_base_local]    ||= File.join('tmp', 'repo_configs')
    opts[:copy_dir_external]  ||= host.external_copy_base
    opts[:puppet_collection] ||= puppet_collection_for(:puppet_agent, opts[:puppet_agent_version])
    add_role(host, 'aio') # we are installing agent, so we want aio role
    release_path = opts[:download_url]
    variant, version, arch, codename = host['platform'].to_array
    copy_dir_local = File.join(opts[:copy_base_local], variant)
    onhost_copy_base = opts[:copy_dir_external]

    release_path_end, release_file, download_file =
    pe_puppet_agent_promoted_package_info_dispatch(
      host,
      opts[:puppet_collection], opts
    )
    release_path << release_path_end

    onhost_copied_download = File.join(onhost_copy_base, download_file)
    onhost_copied_file = File.join(onhost_copy_base, release_file)
    fetch_http_file(release_path, download_file, copy_dir_local)
    scp_to host, File.join(copy_dir_local, download_file), onhost_copy_base

    if variant == 'windows'
      result = on host, "echo #{onhost_copied_file}"
      onhost_copied_file = result.raw_output.chomp
      opts = { debug: host[:pe_debug] || opts[:pe_debug] }
      # couldn't pull this out, because it's relying on
      # {Beaker::DSL::InstallUtils::WindowsUtils} methods,
      # which I didn't want to attack right now. TODO
      install_msi_on(host, onhost_copied_file, {}, opts)
    else
      pe_puppet_agent_promoted_package_install_dispatch(
        host,
        onhost_copy_base, onhost_copied_download,
        onhost_copied_file, download_file, opts
      )
    end
    configure_type_defaults_on(host)
  end
end

#install_rpm_on_sles11_host(host, puppet_agent_ver, opts) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Determine the build package to download on a sles-11 (Intel) host, install that package onto the host. Assumed file name format: puppet-agent-7.29.1.26.gf344eeefa-1.sles11.x86_64.rpm. This method should be called after puppet is installed on the master since it relies on the master telling it the puppet agent version to form the download URL.

Parameters:

  • host (Host) —

    The sles-11 host to download and install the package on.

  • opts (Hash{Symbol=>Symbol, String}) —

    The options



335
336
337
338
339
340
341
342
343
344
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 335

def install_rpm_on_sles11_host(host, puppet_agent_ver, opts)
  # Since sles11 builds are not available in PE, download from agent-downloads.
  agent_downloads_url = "http://agent-downloads.delivery.puppetlabs.net/puppet-agent"
  master_aio_version = puppet_fact(master, 'aio_agent_build')
  stream = opts[:puppet_collection] || "puppet#{puppet_agent_ver[0]}"
  path = "#{agent_downloads_url}/#{puppet_agent_ver}/repos/sles/11/#{stream}/x86_64"
  filename = "puppet-agent-#{master_aio_version}-1.sles11.x86_64"
  extension = ".rpm"
  host.install_package_with_rpm("#{path}/#{filename}#{extension}")
end

#install_via_msi?(host) ⇒ Boolean

Deprecated.

the !version_is_less(host, ‘3.99’) can be removed once we no longer support pre 2015.2.0 PE versions

Check if windows host is able to frictionlessly install puppet

Parameters:

  • host (Beaker::Host) —

    that we are checking if it is possible to install frictionlessly to

Returns:

  • (Boolean) —

    true if frictionless is supported and not affected by known bugs



1411
1412
1413
1414
1415
1416
1417
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1411

def install_via_msi?(host)
  #windows agents from 4.0 -> 2016.1.2 were only installable via the aio method
  #powershell2 bug was fixed in PE 2016.4.3, and PE 2017.1.0, but not 2016.5.z.
  (host['platform'] =~ /windows/ && (version_is_less(host['pe_ver'], '2016.4.0') && !version_is_less(host['pe_ver'], '3.99'))) ||
    (host['platform'] =~ /windows-2008r2/ && (version_is_less(host['pe_ver'], '2016.4.3') && !version_is_less(host['pe_ver'], '3.99'))) ||
    (host['platform'] =~ /windows-2008r2/ && (!version_is_less(host['pe_ver'], '2016.4.99') && version_is_less(host['pe_ver'], '2016.5.99') && !version_is_less(host['pe_ver'], '3.99')))
end

#installer_cmd(host, opts) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Create the PE install command string based upon the host and options settings

Examples:

on host, "#{installer_cmd(host, opts)} -a #{host['working_dir']}/answers"

Parameters:

  • host (Host) —

    The host that PE is to be installed on For UNIX machines using the full PE installer, the host object must have the ‘pe_installer’ field set correctly.

  • opts (Hash{Symbol=>String}) —

    The options

Options Hash (opts):

  • :pe_ver (String) —

    Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)

  • :pe_debug (Boolean) — default: false —

    Should we run the installer in debug mode?

  • :interactive (Boolean) — default: false —

    Should we run the installer in interactive mode?



255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 255

def installer_cmd(host, opts)
  version = host['pe_ver'] || opts[:pe_ver]
  # Frictionless install didn't exist pre-3.2.0, so in that case we fall
  # through and do a regular install.
  if host['roles'].include? 'frictionless' and ! version_is_less(version, '3.2.0')
    frictionless_agent_installer_cmd(host, opts, version)
  elsif host['platform'] =~ /osx/
    version = host['pe_ver'] || opts[:pe_ver]
    pe_debug = host[:pe_debug] || opts[:pe_debug] ? ' -verboseR' : ''
    "cd #{host['working_dir']} && hdiutil attach #{host['dist']}.dmg && installer#{pe_debug} -pkg /Volumes/puppet-enterprise-#{version}/puppet-enterprise-installer-#{version}.pkg -target /"
  elsif host['platform'] =~ /eos/
    host.install_from_file("puppet-enterprise-#{version}-#{host['platform']}.swix")
  else
    pe_debug = host[:pe_debug] || opts[:pe_debug]  ? ' -D' : ''
    pe_cmd = "cd #{host['working_dir']}/#{host['dist']} && ./#{host['pe_installer']}#{pe_debug}"
    if ! version_is_less(host['pe_ver'], '2016.2.1') && ! opts[:interactive]
      # -y option sets "assume yes" mode where yes or whatever default will be assumed
      pe_cmd += " -y"
    end

    configure_flags = host['configure_flags'] || opts[:configure_flags]
    pe_cmd += " -- #{configure_flags}" if configure_flags

    # If we are doing an upgrade from 2016.2.0,
    # we can assume there will be a valid pe.conf in /etc that we can re-use.
    # We also expect that any custom_answers specified to beaker have been
    # added to the pe.conf in /etc.
    if opts[:type] == :upgrade && use_meep?(host[:previous_pe_ver])
      "#{pe_cmd}"
    else
      "#{pe_cmd} #{host['pe_installer_conf_setting']}"
    end
  end
end

#is_expected_pe_postgres_failure?(host) ⇒ Boolean

Check the lastest install log to confirm the expected failure is there

Returns:

  • (Boolean) 


2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2057

def is_expected_pe_postgres_failure?(host)
  installer_log_dir = '/var/log/puppetlabs/installer'
  latest_installer_log_file = on(host, "ls -1t #{installer_log_dir} | head -n1").stdout.chomp
  # As of PE Irving (PE 2018.1.x), these are the only two expected errors
  allowed_errors = ["The operation could not be completed because RBACs database has not been initialized",
    "Timeout waiting for the database pool to become ready",
    "Systemd restart for pe-console-services failed",
    "Execution of.*service pe-console-services.*: Reload timed out after 120 seconds"]

  allowed_errors.each do |error|
    if(on(host, "grep '#{error}' #{installer_log_dir}/#{latest_installer_log_file}", :acceptable_exit_codes => [0,1]).exit_code == 0)
      return true
    end
  end

  false
end

#lb_connect_loadbalancer_exists? ⇒ Boolean

Returns true if loadbalncer exists and is configured with ‘lb_connect’ role

Returns:

  • (Boolean) 


115
116
117
118
119
120
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 115

def lb_connect_loadbalancer_exists?
  if any_hosts_as?('loadbalancer')
    lb_node = select_hosts(roles: ['loadbalancer'])
    lb_node.first['roles'].include?('lb_connect')
  end
end

#loadbalancer_connecting_agents ⇒ Object

Return agent nodes with ‘lb_connect’ role that are not loadbalancers



109
110
111
112
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 109

def loadbalancer_connecting_agents
  lb_connect_nodes = select_hosts(roles: ['lb_connect'])
  lb_connect_agents = lb_connect_nodes.reject { |h| h['roles'].include?('loadbalancer')}
end

#manage_puppet_service?(version, opts) ⇒ Boolean

Whether or not PE should be managing the puppet service on agents. Puppet code to manage the puppet service was added to the next branches and is slated to be merged into 2018.1.x

Returns true if the version we are managing is greater than or equal to MANAGE_PUPPET_SERVICE_VERSION.

Temporarily, (until merged from ‘next’ branches into 2018.1.x), also checks the pe_modules_next flag to know whether or not the code for managing puppet service is present.

Returns:

  • (Boolean) 


1437
1438
1439
1440
1441
1442
1443
1444
1445
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1437

def manage_puppet_service?(version, opts)
  # PE-23651 remove vv
  register_feature_flags!(opts)

  temporary_flag = !!feature_flag?('pe_modules_next', opts)
  # ^^

  !version_is_less(version, MANAGE_PUPPET_SERVICE_VERSION) && temporary_flag
end

#original_pe_ver(host) ⇒ Object

Return the original pe_ver setting for the passed host. Beaker resets pe_ver to the value of pe_upgrade_ver during its upgrade process. If the hosts’s original configuration did not have a pe_ver, return the value of pe_ver set directly in options. It’s the Host that gets overwritten by Beaker on upgrade. So if the original host config did not have a pe_ver set, there should be a pe_ver set in options and we can use that.



2258
2259
2260
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2258

def original_pe_ver(host)
  options[:HOSTS][host.name][:pe_ver] || options[:pe_ver]
end

#pe_puppet_agent_promoted_package_info_dispatch(host, puppet_collection = nil, opts = {}) ⇒ Object 



1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1044

def pe_puppet_agent_promoted_package_info_dispatch(host, puppet_collection = nil, opts = {})
  case host
  when Windows::Host
    return pe_puppet_agent_promoted_package_info_windows(host, puppet_collection, opts)
  when Mac::Host
    return pe_puppet_agent_promoted_package_info_mac(host, puppet_collection, opts)
  when Unix::Host
    return pe_puppet_agent_promoted_package_info_unix(host, puppet_collection, opts)
  end
  raise RuntimeError("#{host.class} not one of Beaker::(Windows|Mac|Unix)::Host")
end

#pe_puppet_agent_promoted_package_info_mac(host, puppet_collection = nil, opts = {}) ⇒ String

Gets host-specific information for PE promoted puppet-agent packages (Mac version)

Parameters:

  • host (Host)
  • puppet_collection (String) (defaults to: nil) —

    Name of the puppet collection to use

  • opts (Hash{Symbol=>String}) (defaults to: {}) —

    Options hash to provide extra values

Returns:

  • (String, String, String) —

    Host-specific information for packages

    1. release_path_end Suffix for the release_path. Used on Windows. Check

    Windows::Pkg#pe_puppet_agent_promoted_package_info to see usage.

    1. release_file Path to the file on release build servers

    2. download_file Filename for the package itself

Raises:

  • (ArgumentError) 


905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 905

def pe_puppet_agent_promoted_package_info_mac(host, puppet_collection = nil, opts = {} )
  error_message = "Must provide %s argument to get puppet agent dev package information"
  raise ArgumentError, error_message % "puppet_collection" unless puppet_collection

  variant, version, arch, _codename = host['platform'].to_array
  release_file = "/repos/apple/#{version}/#{puppet_collection}/#{arch}/puppet-agent-*"

  # macOS puppet-agent tarballs haven't always included arch
  agent_version = opts[:puppet_agent_version]
  agent_version_f = agent_version&.to_f

  download_file = if agent_version_f.nil? || (agent_version_f < 6.28 || (agent_version_f >= 7.0 && agent_version_f < 7.18))
                    "puppet-agent-#{variant}-#{version}.tar.gz"
                  else
                    "puppet-agent-#{variant}-#{version}-#{arch}.tar.gz"
                  end

  return '', release_file, download_file
end

#pe_puppet_agent_promoted_package_info_unix(host, puppet_collection = nil, _opts = {}) ⇒ String

Gets host-specific information for PE promoted puppet-agent packages (Unix version)

Parameters:

  • host (Host)
  • puppet_collection (String) (defaults to: nil) —

    Name of the puppet collection to use

  • opts (Hash{Symbol=>String}) —

    Options hash to provide extra values

Returns:

  • (String, String, String) —

    Host-specific information for packages

    1. release_path_end Suffix for the release_path. Used on Windows. Check

    Windows::Pkg#pe_puppet_agent_promoted_package_info to see usage.

    1. release_file Path to the file on release build servers

    2. download_file Filename for the package itself

Raises:

  • (ArgumentError) 


959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 959

def pe_puppet_agent_promoted_package_info_unix(host, puppet_collection = nil, _opts = {} )
  error_message = "Must provide %s argument to get puppet agent dev package information"
  raise ArgumentError, error_message % "puppet_collection" unless puppet_collection

  variant, version, arch, codename = host['platform'].to_array
  case variant
  when /^(fedora|el|centos|redhat|opensuse|sles)$/
    variant = ((['centos', 'redhat'].include?(variant)) ? 'el' : variant)
    release_file = "/repos/#{variant}/#{version}/#{puppet_collection}/#{arch}/puppet-agent-*.rpm"
    download_file = "puppet-agent-#{variant}-#{version}-#{arch}.tar.gz"
  when /^(debian|ubuntu|cumulus)$/
    if arch == 'x86_64'
      arch = 'amd64'
    end
    version = version[0,2] + '.' + version[2,2] if (variant.include?('ubuntu') && !version.include?("."))
    release_file = "/repos/apt/#{codename}/pool/#{puppet_collection}/p/puppet-agent/puppet-agent*#{arch}.deb"
    download_file = "puppet-agent-#{variant}-#{version}-#{arch}.tar.gz"
  when /^solaris$/
    if arch == 'x86_64'
      arch = 'i386'
    end
    release_file = "/repos/solaris/#{version}/#{puppet_collection}/"
    download_file = "puppet-agent-#{variant}-#{version}-#{arch}.tar.gz"
  else
    raise "No pe-promoted installation step for #{variant} yet..."
  end
  return '', release_file, download_file
end

#pe_puppet_agent_promoted_package_info_windows(host, _puppet_collection = nil, _opts = {}) ⇒ String

Gets host-specific information for PE promoted puppet-agent packages (Windows version)

Parameters:

  • host (Host)
  • puppet_collection (String) —

    Name of the puppet collection to use

  • opts (Hash{Symbol=>String}) —

    Options hash to provide extra values

Returns:

  • (String, String, String) —

    Host-specific information for packages

    1. release_path_end Suffix for the release_path

    2. release_file Path to the file on release build servers

    3. download_file Filename for the package itself



935
936
937
938
939
940
941
942
943
944
945
946
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 935

def pe_puppet_agent_promoted_package_info_windows(host, _puppet_collection = nil, _opts = {} )
  is_config_32 = host['ruby_arch'] == 'x86' || host['install_32'] || host['install_32']
  should_install_64bit = host.is_x86_64? && !is_config_32
  # only install 64bit builds if
  # - we do not have install_32 set on host
  # - we do not have install_32 set globally
  arch_suffix = should_install_64bit ? '64' : '86'
  release_path_end = "/windows"
  release_file = "/puppet-agent-x#{arch_suffix}.msi"
  download_file = "puppet-agent-x#{arch_suffix}.msi"
  return release_path_end, release_file, download_file
end

#pe_puppet_agent_promoted_package_install_dispatch(host, onhost_copy_base, onhost_copied_download, onhost_copied_file, download_file, _opts) ⇒ Object 



988
989
990
991
992
993
994
995
996
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 988

def pe_puppet_agent_promoted_package_install_dispatch(host, onhost_copy_base, onhost_copied_download, onhost_copied_file, download_file, _opts)
  case host
  when Mac::Host
    return pe_puppet_agent_promoted_package_install_mac(host, onhost_copy_base, onhost_copied_download, onhost_copied_file, download_file, _opts)
  when Unix::Host
    return pe_puppet_agent_promoted_package_install_unix(host, onhost_copy_base, onhost_copied_download, onhost_copied_file, download_file, _opts)
  end
  raise RuntimeError("#{host.class} not one of Beaker::(Mac|Unix)::Host")
end

#pe_puppet_agent_promoted_package_install_mac(host, onhost_copy_base, onhost_copied_download, onhost_copied_file, _download_file, _opts) ⇒ Object

Installs a given PE promoted package on a Mac host

Parameters:

  • host (Host) —

    Host to install package on

  • onhost_copy_base (String) —

    Base copy directory on the host

  • onhost_copied_download (String) —

    Downloaded file path on the host

  • onhost_copied_file (String) —

    Copied file path once un-compressed

  • download_file (String) —

    File name of the downloaded file

  • opts (Hash{Symbol=>String}) —

    additional options

Returns:

  • nil



1008
1009
1010
1011
1012
1013
1014
1015
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1008

def pe_puppet_agent_promoted_package_install_mac(
      host, onhost_copy_base, onhost_copied_download, onhost_copied_file, _download_file, _opts
    )
  host.execute("tar -zxvf #{onhost_copied_download} -C #{onhost_copy_base}")
  # move to better location
  host.execute("mv #{onhost_copied_file}.dmg .")
  host.install_package("puppet-agent-*")
end

#pe_puppet_agent_promoted_package_install_unix(host, onhost_copy_base, onhost_copied_download, onhost_copied_file, download_file, _opts) ⇒ Object

Installs a given PE promoted package on a Unix host

Parameters:

  • host (Host) —

    Host to install package on

  • onhost_copy_base (String) —

    Base copy directory on the host

  • onhost_copied_download (String) —

    Downloaded file path on the host

  • onhost_copied_file (String) —

    Copied file path once un-compressed

  • download_file (String) —

    File name of the downloaded file

  • opts (Hash{Symbol=>String}) —

    additional options

Returns:

  • nil



1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1028

def pe_puppet_agent_promoted_package_install_unix(
      host, onhost_copy_base, onhost_copied_download, onhost_copied_file, download_file, _opts
    )
  host.uncompress_local_tarball(onhost_copied_download, onhost_copy_base, download_file )
  if /^solaris/.match?(host['platform'])
    # above uncompresses the install from .tar.gz -> .p5p into the
    # onhost_copied_file directory w/a weird name. We have to read that file
    # name from the filesystem, so that we can provide it to install_local...
    pkg_filename = host.execute( "ls #{onhost_copied_file}" )
    onhost_copied_file = "#{onhost_copied_file}#{pkg_filename}"
  end

  host.install_local_package(onhost_copied_file, onhost_copy_base )
  nil
end

#prep_host_for_upgrade(host, opts = {}, path = '') ⇒ Object

Prep a host object for upgrade; used inside upgrade_pe_on !macro common_opts

Examples:

prep_host_for_upgrade(master, {}, "http://neptune.puppetlabs.lan/3.0/ci-ready/")

Parameters:

  • host (Host) —

    A single host object to prepare for upgrade

  • path (String) (defaults to: '') —

    A path (either local directory or a URL to a listing of PE builds). Will contain a LATEST file indicating the latest build to install. This is ignored if a pe_upgrade_ver and pe_upgrade_dir are specified in the host configuration file.



1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1860

def prep_host_for_upgrade(host, opts={}, path='')
  host['pe_dir'] = host['pe_upgrade_dir'] || path
  host['previous_pe_ver'] = host['pe_ver']
  if host['platform'] =~ /windows/
    host['pe_ver'] = host['pe_upgrade_ver'] || opts['pe_upgrade_ver'] ||
      Options::PEVersionScraper.load_pe_version(host['pe_dir'], opts[:pe_version_file_win])
  else
    host['pe_ver'] = host['pe_upgrade_ver'] || opts['pe_upgrade_ver'] ||
      Options::PEVersionScraper.load_pe_version(host['pe_dir'], opts[:pe_version_file])
  end
  if version_is_less(host['pe_ver'], '3.0')
    host['pe_installer'] ||= 'puppet-enterprise-upgrader'
  end
end

#prepare_host_installer_options(host) ⇒ Beaker::Host

Set installer options on the passed host according to current version.

Sets:

* 'pe_installer_conf_file'
* 'pe_installer_conf_setting'

Parameters:

  • host (Beaker::Host) —

    The host object to configure

Returns:

  • (Beaker::Host) —

    The same host object passed in



1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1497

def prepare_host_installer_options(host)
  if use_meep?(host['pe_ver'])
    conf_file = "#{host['working_dir']}/pe.conf"
    host['pe_installer_conf_file'] = conf_file
    host['pe_installer_conf_setting'] = "-c #{conf_file}"
  else
    conf_file = "#{host['working_dir']}/answers"
    host['pe_installer_conf_file'] = conf_file
    host['pe_installer_conf_setting'] = "-a #{conf_file}"
  end
  host
end

#prepare_hosts(hosts, local_options = {}) ⇒ Object

Note:

that these steps aren’t necessary for all hosts. Specifically, ‘agent_only’ hosts do not require these steps to be executed.

Prepares hosts for rest of #do_install operations. This includes doing these tasks:

  • setting ‘pe_installer’ property on hosts

  • setting ‘dist’ property on hosts

  • creating and setting ‘working_dir’ property on hosts

Parameters:

  • hosts (Array<Host>) —

    Hosts to prepare

  • local_options (Hash{Symbol=>String}) (defaults to: {}) —

    Local options, used to pass misc configuration required for the prep steps

Returns:

  • nil



1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1318

def prepare_hosts(hosts, local_options={})
  use_all_tar = ENV['PE_USE_ALL_TAR'] == 'true'
  hosts.each do |host|
    host['pe_installer'] ||= 'puppet-enterprise-installer'
    if host['platform'] !~ /windows|osx/
      platform = use_all_tar ? 'all' : host['platform']
      version = host['pe_ver'] || local_options[:pe_ver]
      host['dist'] = "puppet-enterprise-#{version}-#{platform}"
    elsif host['platform'] =~ /osx/
      version = host['pe_ver'] || local_options[:pe_ver]
      host['dist'] = "puppet-enterprise-#{version}-#{host['platform']}"
    elsif host['platform'] =~ /windows/
      version = host[:pe_ver] || local_options['pe_ver_win']
      is_config_32 = true == (host['ruby_arch'] == 'x86') || host['install_32'] || local_options['install_32']
      should_install_64bit = !(version_is_less(version, '3.4')) && host.is_x86_64? && !is_config_32
      #only install 64bit builds if
      # - we are on pe version 3.4+
      # - we do not have install_32 set on host
      # - we do not have install_32 set globally
      if !(version_is_less(version, '3.99'))
        if should_install_64bit
          host['dist'] = "puppet-agent-#{version}-x64"
        else
          host['dist'] = "puppet-agent-#{version}-x86"
        end
      elsif should_install_64bit
        host['dist'] = "puppet-enterprise-#{version}-x64"
      else
        host['dist'] = "puppet-enterprise-#{version}"
      end
    end
    host['dist'] = "puppet-enterprise-#{version}-#{host['packaging_platform']}" if host['packaging_platform'] =~ /redhatfips/
    host['working_dir'] = host.tmpdir(Time.new.strftime("%Y-%m-%d_%H.%M.%S"))
  end
end

#quoted_hocon_key(key) ⇒ Object

If the key is unquoted and does not contain pathing (‘.’), quote to ensure that puppet namespaces are protected

Examples:

quoted_hocon_key("puppet_enterprise::database_host")
# => '"puppet_enterprise::database_host"'


2243
2244
2245
2246
2247
2248
2249
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2243

def quoted_hocon_key(key)
  case key
  when /^[^"][^.]+/
    then %Q{"#{key}"}
  else key
  end
end

#register_feature_flags!(opts) ⇒ Object

The pe-modules-next package is being used for isolating large scale feature development of PE module code. The feature flag is a pe.conf setting ‘feature_flags::pe_modules_next’, which if set true will cause the installer shim to install the pe-modules-next package instead of pe-modules.

This answer can be explicitly added to Beaker’s cfg file by adding it to the :answers section.

But it can also be picked up transparently from CI via the PE_MODULES_NEXT environment variable. If this is set ‘true’, then the opts will be set with feature_flags::pe_modules_next.

Answers set in Beaker’s config file will take precedence over the environment variable.

NOTE: This has implications for upgrades, because upgrade testing will need the flag, but upgrades from different pe.conf schema (or no pe.conf) will need to generate a pe.conf, and that workflow is likely to happen in the installer shim. If we simply supply a good pe.conf via beaker-answers, then we have bypassed the pe.conf generation aspect of the upgrade workflow. (See PE-19438)



1565
1566
1567
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1565

def register_feature_flags!(opts)
  Beaker::DSL::InstallUtils::FeatureFlags.new(opts).register_flags!
end

#remove_client_datadir(host) ⇒ Object

Remove client_datadir on the host

Parameters:

  • the (Host) —

    host



134
135
136
137
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 134

def remove_client_datadir(host)
  client_datadir = host.puppet['client_datadir']
  on(host, "rm -rf #{client_datadir}")
end

#require_tlsv1?(host) ⇒ Boolean

Return true if tlsv1 protocol needs to be enforced param [Host] the host

Returns:

  • (Boolean) 


141
142
143
144
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 141

def require_tlsv1?(host)
  tlsv1_platforms = [/el-5/, /solaris-1[0,1]-[i,x]/, /sles-11/,/windows-2008/]
  return tlsv1_platforms.any? {|platform_regex| host['platform'] =~ platform_regex}
end

#run_puppet_on_non_infrastructure_nodes(all_hosts) ⇒ Object

Runs puppet on all nodes, unless they have the roles: master,database,console/dashboard

Parameters:

  • hosts (Array<Host>) —

    The sorted hosts to install or upgrade PE on



1421
1422
1423
1424
1425
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1421

def run_puppet_on_non_infrastructure_nodes(all_hosts)
  pe_infrastructure = select_hosts({:roles => ['master', 'compile_master', 'pe_compiler', 'dashboard', 'database']}, all_hosts)
  non_infrastructure = all_hosts.reject{|host| pe_infrastructure.include? host}
  on non_infrastructure, puppet_agent('-t'), :acceptable_exit_codes => [0,2], :run_in_parallel => true
end

#setup_beaker_answers_opts(host, opts) ⇒ Hash

Adds in settings needed by BeakerAnswers:

  • :format => :bash or :hiera depending on which legacy or meep format we need

  • :include_legacy_database_defaults => true or false. True indicates that we are upgrading from a legacy version and BeakerAnswers should include the database defaults for user which were set for the legacy install.

Parameters:

  • host (Beaker::Host) —

    that we are generating answers for

  • opts (Hash) —

    The Beaker options hash

Returns:

  • (Hash) —

    a dup of the opts hash with additional settings for BeakerAnswers



1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1521

def setup_beaker_answers_opts(host, opts)
  beaker_answers_opts = use_meep?(host['pe_ver']) ?
    { :format => :hiera } :
    { :format => :bash }

  beaker_answers_opts[:include_legacy_database_defaults] =
    opts[:type] == :upgrade && !use_meep?(host['previous_pe_ver'])

  modified_opts = opts.merge(beaker_answers_opts)

  answers_hash = modified_opts[:answers] ||= {}
  if !answers_hash.include?(:meep_schema_version)
    if feature_flag?(:meep_classification, opts)
      answers_hash[:meep_schema_version] = '2.0'
    elsif use_meep?(host['pe_ver'])
      answers_hash[:meep_schema_version] = '1.0'
    end
  end

  modified_opts
end

#setup_defaults_and_config_helper_on(host, master, acceptable_exit_codes = nil) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Helper for setting up pe_defaults & setting up the cert on the host

Parameters:

  • host (Host) —

    host to setup

  • master (Host) —

    the master host, for setting up the relationship

  • acceptable_exit_codes (Array<Fixnum>) (defaults to: nil) —

    The exit codes that we want to ignore

Returns:

  • nil



1706
1707
1708
1709
1710
1711
1712
1713
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1706

def setup_defaults_and_config_helper_on(host, master, acceptable_exit_codes=nil)
  configure_type_defaults_on(host)
  #set the certname and master
  on host, puppet("config set server #{master}")
  on host, puppet("config set certname #{host}")
  #run once to request cert
  on host, puppet_agent('-t'), :acceptable_exit_codes => acceptable_exit_codes
end

#setup_pe_conf(host, hosts, opts = {}) ⇒ Object 



2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2309

def setup_pe_conf(host, hosts, opts={})
  if opts[:type] == :upgrade && use_meep?(host['previous_pe_ver'])
    # In this scenario, Beaker runs the installer such that we make
    # use of recovery code in the configure face of the installer.
    if host['roles'].include?('master')
      step "Updating #{MEEP_DATA_DIR}/conf.d with answers/custom_answers" do
        # merge answers into pe.conf
        if opts[:answers] && !opts[:answers].empty?
          update_pe_conf(opts[:answers])
        end

        if opts[:custom_answers] && !opts[:custom_answers].empty?
          update_pe_conf(opts[:custom_answers])
        end
      end
    else
      step "Uploading #{BEAKER_MEEP_TMP}/conf.d that was generated on the master" do
        # scp conf.d to host
        scp_to(host, "#{BEAKER_MEEP_TMP}/conf.d", MEEP_DATA_DIR)
      end
    end
  else
    # Beaker creates a fresh pe.conf using beaker-answers, as if we were doing an install
    generate_installer_conf_file_for(host, hosts, opts)
  end
end

#simple_monolithic_install(master, agents, opts = {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Install PE on a monolithic master and some number of frictionless agents.

Examples:

simple_monolithic_install(master, agents, {:type => :install, :pe_ver => '2017.2.0'})

Parameters:

  • master (Host) —

    The node to install the master on

  • agents (Array<Host>) —

    The nodes to install agents on

  • opts (Hash{Symbol=>Symbol, String}) (defaults to: {}) —

    The options for how to install or upgrade PE

Returns:

  • nil



739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 739

def simple_monolithic_install(master, agents, opts={})
  step "Performing a standard monolithic install with frictionless agents"
  all_hosts = [master, *agents]
  configure_type_defaults_on([master])

  # Set PE distribution on the agents, creates working directories
  prepare_hosts(all_hosts, opts)
  fetch_pe([master], opts)
  prepare_host_installer_options(master)
  register_feature_flags!(opts)
  generate_installer_conf_file_for(master, all_hosts, opts)
  step "Install PE on master" do
    on master, installer_cmd(master, opts)
  end

  step "Stop agent on master" do
    stop_agent_on(master)
  end

  if manage_puppet_service?(master[:pe_ver], options)
    configure_puppet_agent_service(:ensure => 'stopped', :enabled => false)
  end

  # waitforlock in case stopping the agent run after stopping the agent service
  # takes a little longer than usual
  step "Run puppet to setup mcollective and pxp-agent" do
    on(master, puppet_agent("-t #{waitforlock_flag(master)}"), :acceptable_exit_codes => [0,2])
  end

  install_agents_only_on(agents, opts)

  step "Run puppet a second time on the primary to populate services.conf (PE-19054)" do
    on(master, puppet_agent('-t'), :acceptable_exit_codes => [0,2])
  end
end

#simple_monolithic_install_with_preload(master, agents, opts = {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Configure PE on a monolithic master and some number of frictionless agents if a node is using a PE preloaded image for the master node.

Examples:

simple_monolithic_install_with_preload(master, agents, {:type => :install, :pe_ver => '2017.2.0'})

Parameters:

  • master (Host) —

    The node to configure the master on

  • agents (Array<Host>) —

    The nodes to install agents on

  • opts (Hash{Symbol=>Symbol, String}) (defaults to: {}) —

    The options for how to install or upgrade PE

Returns:

  • nil



786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 786

def simple_monolithic_install_with_preload(master, agents, opts={})
  step "Performing a standard monolithic install with frictionless agents on a preloaded image"
  all_hosts = [master, *agents]
  configure_type_defaults_on([master])

  # Set PE distribution on the agents, creates working directories
  prepare_hosts(all_hosts, opts)
  register_feature_flags!(opts)
  generate_installer_conf_file_for(master, all_hosts, opts)

  step "Stop agent on master" do
    stop_agent_on(master)
  end

  if manage_puppet_service?(master[:pe_ver], options)
    configure_puppet_agent_service(:ensure => 'stopped', :enabled => false)
  end

  install_agents_only_on(agents, opts)
end

#solaris_key_chain_fix ⇒ Object

If installing older versions of solaris, make sure the DigiCert cert is added to the keychain



1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1613

def solaris_key_chain_fix
digicert = <<-EOM
-----BEGIN CERTIFICATE-----
MIIFkDCCA3igAwIBAgIQBZsbV56OITLiOQe9p3d1XDANBgkqhkiG9w0BAQwFADBi
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSEwHwYDVQQDExhEaWdpQ2VydCBUcnVzdGVkIFJvb3Qg
RzQwHhcNMTMwODAxMTIwMDAwWhcNMzgwMTE1MTIwMDAwWjBiMQswCQYDVQQGEwJV
UzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3d3cuZGlnaWNlcnQu
Y29tMSEwHwYDVQQDExhEaWdpQ2VydCBUcnVzdGVkIFJvb3QgRzQwggIiMA0GCSqG
SIb3DQEBAQUAA4ICDwAwggIKAoICAQC/5pBzaN675F1KPDAiMGkz7MKnJS7JIT3y
ithZwuEppz1Yq3aaza57G4QNxDAf8xukOBbrVsaXbR2rsnnyyhHS5F/WBTxSD1If
xp4VpX6+n6lXFllVcq9ok3DCsrp1mWpzMpTREEQQLt+C8weE5nQ7bXHiLQwb7iDV
ySAdYyktzuxeTsiT+CFhmzTrBcZe7FsavOvJz82sNEBfsXpm7nfISKhmV1efVFiO
DCu3T6cw2Vbuyntd463JT17lNecxy9qTXtyOj4DatpGYQJB5w3jHtrHEtWoYOAMQ
jdjUN6QuBX2I9YI+EJFwq1WCQTLX2wRzKm6RAXwhTNS8rhsDdV14Ztk6MUSaM0C/
CNdaSaTC5qmgZ92kJ7yhTzm1EVgX9yRcRo9k98FpiHaYdj1ZXUJ2h4mXaXpI8OCi
EhtmmnTK3kse5w5jrubU75KSOp493ADkRSWJtppEGSt+wJS00mFt6zPZxd9LBADM
fRyVw4/3IbKyEbe7f/LVjHAsQWCqsWMYRJUadmJ+9oCw++hkpjPRiQfhvbfmQ6QY
uKZ3AeEPlAwhHbJUKSWJbOUOUlFHdL4mrLZBdd56rF+NP8m800ERElvlEFDrMcXK
chYiCd98THU/Y+whX8QgUWtvsauGi0/C1kVfnSD8oR7FwI+isX4KJpn15GkvmB0t
9dmpsh3lGwIDAQABo0IwQDAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIB
hjAdBgNVHQ4EFgQU7NfjgtJxXWRM3y5nP+e6mK4cD08wDQYJKoZIhvcNAQEMBQAD
ggIBALth2X2pbL4XxJEbw6GiAI3jZGgPVs93rnD5/ZpKmbnJeFwMDF/k5hQpVgs2
SV1EY+CtnJYYZhsjDT156W1r1lT40jzBQ0CuHVD1UvyQO7uYmWlrx8GnqGikJ9yd
+SeuMIW59mdNOj6PWTkiU0TryF0Dyu1Qen1iIQqAyHNm0aAFYF/opbSnr6j3bTWc
fFqK1qI4mfN4i/RN0iAL3gTujJtHgXINwBQy7zBZLq7gcfJW5GqXb5JQbZaNaHqa
sjYUegbyJLkJEVDXCLG4iXqEI2FCKeWjzaIgQdfRnGTZ6iahixTXTBmyUEFxPT9N
cCOGDErcgdLMMpSEDQgJlxxPwO5rIHQw0uA5NBCFIRUBCOhVMt5xSdkoF1BN5r5N
0XWs0Mr7QbhDparTwwVETyw2m+L64kW4I1NsBm9nVX9GtUw/bihaeSbSpKhil9Ie
4u1Ki7wb/UdKDd9nZn6yW0HQO+T0O/QEY+nvwlQAUaCKKsnOeMzV6ocEGLPOr0mI
r/OSmbaz5mEP0oUA51Aa5BuVnRmhuZyxm7EAHu/QD09CbMkKvO5D+jpxpchNJqU1
/YldvIViHTLSoCtU7ZpXwdv6EM8Zt4tKG48BtieVU+i2iW1bvGjUI+iLUaJW+fCm
gKDWHrO8Dw9TdSmq6hN35N6MgSGtBxBHEa2HPQfRdbzP82Z+
-----END CERTIFICATE-----
EOM
  hosts.each do |host|
    if host.platform=~ /solaris-11(\.2)?-(i386|sparc)/
      create_remote_file(host, "DigiCert_Trusted_Root_G4.pem", digicert)
      on(host, 'chmod a+r /root/DigiCert_Trusted_Root_G4.pem')
      on(host, 'cp -p /root/DigiCert_Trusted_Root_G4.pem /etc/certs/CA/')
      on(host, 'rm /root/DigiCert_Trusted_Root_G4.pem')
      on(host, '/usr/sbin/svcadm restart /system/ca-certificates')
      timeout = 60
      counter = 0
      while on(host, 'svcs -x ca-certificates').output !~ /State: online/ do
        raise 'ca-certificates services failed start up' if counter > timeout
        sleep 5
        counter = counter + 5
      end
    end
  end
end

#sync_pe_conf(host, pe_conf_file = PE_CONF_FILE) ⇒ Object

Sync pe.conf from the master to another infrastructure node. Useful when updating pe.conf to reconfigure infrastructure, where you first update_pe_conf then sync_pe_conf to infrastructure hosts.

Parameters:

  • host (Host) —

    The host to sync to

  • pe_conf_file (String) (defaults to: PE_CONF_FILE) —

    The file to sync (/etc/puppetlabs/enterprise/conf.d/pe.conf by default)



2229
2230
2231
2232
2233
2234
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2229

def sync_pe_conf(host, pe_conf_file = PE_CONF_FILE)
  Dir.mktmpdir('sync_pe_conf') do |tmpdir|
    scp_from(master, pe_conf_file, tmpdir)
    scp_to(host, File.join(tmpdir, File.basename(pe_conf_file)), pe_conf_file)
  end
end

#update_pe_conf(parameters, pe_conf_file = PE_CONF_FILE) ⇒ Object

Given a hash of parameters, updates the primary master’s pe.conf, adding or replacing, or removing the given parameters.

To remove a parameter, pass a nil as its value

Handles stringifying and quoting namespaced keys, and also preparing non string values using Hocon::ConfigValueFactory.

Logs the state of pe.conf before and after.

Examples:

# Assuming pe.conf looks like:
# {
# "bar": "baz"
# "old": "item"
# }

update_pe_conf(
  {
    "foo" => "a",
    "bar" => "b",
    "old" => nil,
  }
)

# Will produce a pe.conf like:
# {
# "bar": "b"
# "foo": "a"
# }

Parameters:

  • parameters (Hash) —

    Hash of parameters to be included in pe.conf.

  • pe_conf_file (String) (defaults to: PE_CONF_FILE) —

    The file to update (/etc/puppetlabs/enterprise/conf.d/pe.conf by default)



2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2188

def update_pe_conf(parameters, pe_conf_file = PE_CONF_FILE)
  step "Update #{pe_conf_file} with #{parameters}" do
    hocon_file_edit_in_place_on(master, pe_conf_file) do |host,doc|
      updated_doc = parameters.reduce(doc) do |pe_conf,param|
        key, value = param

        hocon_key = quoted_hocon_key(key)

        hocon_value = case value
        when String
          # ensure unquoted string values are quoted for uniformity
          then value.match(/^[^"]/) ? %Q{"#{value}"} : value
        else Hocon::ConfigValueFactory.from_any_ref(value, nil)
        end

        updated = case value
        when String
          pe_conf.set_value(hocon_key, hocon_value)
        when nil
          pe_conf.remove_value(hocon_key)
        else
          pe_conf.set_config_value(hocon_key, hocon_value)
        end

        updated
      end

      # return the modified document
      updated_doc
    end
    on(master, "cat #{pe_conf_file}")
  end
end

#upgrade_pe(path = nil) ⇒ Object

Upgrade PE based upon global host configuration and global options

See Also:



1818
1819
1820
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1818

def upgrade_pe path=nil
  upgrade_pe_on(hosts, options, path)
end

#upgrade_pe_on(upgrade_hosts, opts, path = nil) ⇒ Object

Note:

Install file names are assumed to be of the format puppet-enterprise-VERSION-PLATFORM.(tar)|(tar.gz) for Unix like systems and puppet-enterprise-VERSION.msi for Windows systems.

Upgrade PE based upon host configuration and options

Examples:

upgrade_pe_on(agents, {}, "http://neptune.puppetlabs.lan/3.0/ci-ready/")

Parameters:

  • upgrade_hosts (Host, Array<Host>) —

    One or more hosts to act upon

  • opts (Hash{Symbol=>String}) —

    Options to alter execution.

  • path (String) (defaults to: nil) —

    A path (either local directory or a URL to a listing of PE builds). Will contain a LATEST file indicating the latest build to install. This is ignored if a pe_upgrade_ver and pe_upgrade_dir are specified in the host configuration file.

Options Hash (opts):

  • :silent (Boolean) — default: false —

    Do not produce log output

  • :acceptable_exit_codes (Array<Fixnum>) — default: [0] —

    An array (or range) of integer exit codes that should be considered acceptable. An error will be thrown if the exit code does not match one of the values in this list.

  • :accept_all_exit_codes (Boolean) — default: false —

    Consider all exit codes as passing.

  • :dry_run (Boolean) — default: false —

    Do not actually execute any commands on the SUT

  • :stdin (String) — default: nil —

    Input to be provided during command execution on the SUT.

  • :pty (Boolean) — default: false —

    Execute this command in a pseudoterminal.

  • :expect_connection_failure (Boolean) — default: false —

    Expect this command to result in a connection failure, reconnect and continue execution.

  • :environment (Hash{String=>String}) — default: {} —

    These will be treated as extra environment variables that should be set before running the command.



1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1834

def upgrade_pe_on upgrade_hosts, opts, path=nil
  confine_block(:to, {}, upgrade_hosts) do
    set_console_password = false
    # if we are upgrading from something lower than 3.4 then we need to set the pe console password
    if (dashboard[:pe_ver] ? version_is_less(dashboard[:pe_ver], "3.4.0") : true)
      set_console_password = true
    end
    # get new version information
    hosts.each do |host|
      prep_host_for_upgrade(host, opts, path)
    end

    do_install(sorted_hosts, opts.merge({:type => :upgrade, :set_console_password => set_console_password}))
    opts['upgrade'] = true
  end
end

#upgrading_to_pe_ver(host) ⇒ Object

Returns the version of PE that the host will be upgraded to If no upgrade is planned then just the version of PE to install is returned



2264
2265
2266
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 2264

def upgrading_to_pe_ver(host)
  options[:HOSTS][host.name][:pe_upgrade_ver] || options[:pe_ver]
end

#use_meep?(version) ⇒ Boolean

True if version is greater than or equal to MEEP_CUTOVER_VERSION (2016.2.0)

Returns:

  • (Boolean) 


1390
1391
1392
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1390

def use_meep?(version)
  !version_is_less(version, MEEP_CUTOVER_VERSION)
end

#use_meep_for_classification?(version, opts) ⇒ Boolean

True if version is greater than or equal to MEEP_CLASSIFICATION_VERSION (PE-18718) AND the temporary feature flag is true.

The temporary feature flag is meep_classification and can be set in the :answers hash given in beaker’s host.cfg, inside a feature_flags hash. It will also be picked up from the environment as MEEP_CLASSIFICATION. (See register_feature_flags!())

The :answers hash value will take precedence over the env variable.

Parameters:

  • version —

    String the current PE version

  • opts —

    Hash options hash to inspect for :answers

Returns:

  • (Boolean) —

    Boolean true if version and flag allows for meep classification feature.



1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 1461

def use_meep_for_classification?(version, opts)
  # PE-19470 remove vv
  register_feature_flags!(opts)

  temporary_flag = feature_flag?('meep_classification', opts)
  temporary_flag = DEFAULT_MEEP_CLASSIFICATION if temporary_flag.nil?
  # ^^

  !version_is_less(version, MEEP_CLASSIFICATION_VERSION) && temporary_flag
end

#verify_network_resources(hosts, network_resources) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Check for availability of required network resources

Examples:

verify_network_resources(hosts, network_resources)

Parameters:

  • hosts (Array<Host>)
  • network_resources (Array<String>)

Returns:

  • nil



582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 582

def verify_network_resources(hosts, network_resources)
  logger.notify("Checking the availability of network resources.")
  hosts.each do |host|
    # if options[:net_diag_hosts] isn't set, skip this check
    if network_resources != nil
      network_resources.each do |resource|
        # curl the network resource silently (-s), only connect (-I), and don't print the output
        on host, "curl -I -s #{resource} > /dev/null", :accept_all_exit_codes => true
        if host.connection.logger.last_result.exit_code != 0
          logger.warn("Connection error: #{host.host_hash[:vmhostname]} was unable to connect to #{resource}. Please ensure that your test does not require this resource.")
        end
      end
    end
    hosts.each do |target_host|
      ping_opts = host['platform'] =~ /windows/ ? "-n 1" : "-c1"
      on host, "ping #{ping_opts} #{target_host.host_hash[:vmhostname]} > /dev/null", :accept_all_exit_codes => true
      if host.connection.logger.last_result.exit_code != 0
        logger.warn("Connection error: #{host.host_hash[:vmhostname]} was unable to connect to #{target_host.host_hash[:vmhostname]} in your testing infrastructure.")
      end
    end
  end
end

#verify_vm_resources(hosts) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Check system resources, so that we might be able to find correlations between absurd load levels and transients.

Examples:

verify_vm_resources(hosts)

Parameters:

  • hosts (Array<Host>)

Returns:

  • nil



615
616
617
618
619
620
621
622
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 615

def verify_vm_resources(hosts)
  logger.notify("Checking the status of system (CPU/Mem) resources on PE Infrastructure nodes.")
  pe_infrastructure = select_hosts({:roles => ['master', 'compile_master', 'pe_compiler', 'dashboard', 'database']}, hosts)
  pe_infrastructure.each do |host|
    on host, "top -bn1", :accept_all_exit_codes => true
    on host, "vmstat 1 1", :accept_all_exit_codes => true
  end
end

#waitforlock_flag(host) ⇒ Object

Return ‘–waitforlock 1’ if the agent is >= 6.16 which was used in 2019.8.0. In the test running with legacy agents, the pe_ver value on that host corresponds to the PE version the old agent comes from. Also, puppet_agent_version is present on these host objects.



150
151
152
153
154
155
156
157
158
 
# File 'lib/beaker-pe/install/pe_utils.rb', line 150

def waitforlock_flag(host)
  agent_version = host['puppet_agent_version']
  pe_ver = host['pe_ver']
  if (agent_version && version_is_less(agent_version, '6.16.0')) || version_is_less(host['pe_ver'], '2019.8.0')
    ''
  else
    '--waitforlock 1'
  end
end