4.3. Client Management

This chapter provides guidance to install and configure SIMP clients with the resources supplied by the SIMP installation.

This guide also assumes that your SIMP server is a yum package repository and that you are configuring the production SIMP Omni-Environment.

4.3.1. System Requirements

Client systems should meet the following minimum requirements:

  • Hardware/Virtual Machine (VM): Capable of running RHEL 7 or 8 x86_64

  • RAM: 2048 MB

  • HDD: 22 GB

4.3.2. Configuring the Puppet Server

Perform the following actions as root on the Puppet Server prior to attempting to install a client. Configure DNS

In SIMP, numerous and/or large configuration files are distributed via rsync by Puppet to minimize management cost. These managed files presently include DNS configuration files and can be found at /var/simp/environments/production/rsync/OSTYPE/MAJORRELEASE/bind_dns/default.

This section is not a complete manual for named. For more complete documentation on how to set up named, see named(8) and named.conf(5).

The following configuration steps are for a SIMP-managed setup. However, you can use an existing DNS infrastructure.

  1. Navigate to /var/simp/environments/production/rsync/OSTYPE/MAJORRELEASE/bind_dns/default

  2. Modify the named files to correctly reflect the environment.

    • The relevant files under bind_dns/default/ are as follows:

      • named/etc/named.conf

      • named/etc/zones/your.domain

      • named/var/named/forward/your.domain.db

      • named/var/named/reverse/0.0.10.db

    • Review named/etc/named.conf and update the following:

      • Update the IP for allow-query and allow-recursion

      • Delete any unnecessary zone stanzas (i.e. forwarding) if not necessary

      • Substitute in the FQDN of your domain for all occurrences of your.domain

    • Add clients to named/var/named/forward/your.domain.db and named/var/named/reverse/0.0.10.db and then rename these files to appropriately match your environment.

  3. Run puppet agent -t --tags named on the Puppet Server to apply the changes.

  4. Validate DNS and ensure the /etc/resolv.conf is updated appropriately.

  5. If an rndc.key error appears when starting named, see the BIND Documentation. Once you have resolved the issue, re-run puppet agent -t on the Puppet Server to apply.


You can adjust the list of clients in your named/var/named/forward/<your.domain>.db and named/var/named/reverse/<your reverse domain>.db files at any time; just remember to run puppet agent -t --tags named on the Puppet server to propagate the updates. Configure DHCP


The dhcpd.conf file was updated in SIMP 6.2 to include logic in the pxeclients class that determines the appropriate boot loader file on the TFTP server, based on whether the client is booting in UEFI or BIOS mode. If you have configured DHCP using an earlier version of SIMP and need to add UEFI support, make sure you update your dhcpd.conf in the rsync directory, appropriately.

MAC addresses in the following section need to be lower case letters.

Perform the following actions as root on the Puppet Server system prior to attempting to install a client.

Open /var/simp/environments/production/rsync/<OSTYPE>/Global/dhcpd/dhcpd.conf and edit it to suit the necessary environment. Make sure the following is done:

  • The next-server setting in the pxeclients class block points to the IP Address of the TFTP server.

  • Create a subnet block and edit the following:

    • Make sure the routers, subnet-mask, and netmask are correct for your environment.

    • Enter the hardware ethernet and fixed-address for each client that will be kickstarted. For increased security, it is suggested that SIMP environments not allow clients to pick random IP Address in a subnet. The MAC address must be associated with and IP Address here. (You can add additional ones as needed.)

    • Enter the domain name for option domain-name

    • Enter the IP Address of the DNS server for option domain-name-servers

    • If this DHCP server is used for PXE booting, make sure each filename parameter corresponds to the correct boot loader file on the TFTP server. If you are using SIMP’s simp::server::kickstart class to manage the TFTP server, the default filename values listed in the pxeclients class of the sample dhpcd.conf will be correct.

Save and close the file.

Run puppet agent -t on the Puppet Server to apply the changes.

4.4. Configure PXE Boot

Sample kickstart templates have been provided in the /var/www/ks directory on the SIMP server and on the SIMP DVD under /ks. Pre-boot images are located in the DVD under /images/pxeboot. If you have an existing Preboot Execution Environment (PXE) setup you can use these to PXE a SIMP client. Follow your own sites procedures for this.

In this section we describe how to configure the Kickstart and TFTP servers to PXE boot a SIMP client. (The DNS and DHCP server setup, also required for PXE booting, are discussed in an earlier chapter.)


This example sets up a PXE boot for a system that is the same OS as the SIMP Server. If you are setting up a PXE boot for a different OS then you must make sure that the OS packages are available for all systems you are trying to PXE boot through YUM. There are notes throughout the instructions to help in setting multiple OS but they are not comprehensive. You should understand DHCP, KS, YUM and TFTP relationships for PXE booting before attempting this.

4.4.1. Setting up Kickstart

This section describes how to configure the kickstart server.

  1. Add the Kickstart Server Profile

    • In the Puppet server-specific Hiera file (by default located at /etc/puppetlabs/code/environments/production/data/hosts/puppet.your.domain.yaml), add the simp::server::kickstart class.

        - simp::server::kickstart
    • This profile class adds management of DHCP, DNS, the Kickstart service, as well as the example provisioning script.

    • After adding the above class, run puppet: puppet agent -t.

  2. Locate the following files in the /var/www/ks directory

    • pupclient_x86_64.cfg: Example client kickstart configuration script.

    • diskdetect.sh: Example script to determine disks available on a system and then apply disk configuration. This script is used by pupclient_x86_64.cfg.

  3. Open the pupclient_x86_64.cfg file and follow the instructions provided within it to replace the variables listed and to customize for BIOS/UEFI boot and/or FIPS/non-FIPS mode. If you have servers that require different boot mode or FIPS options, you will need to make customized copies of this file to provide those distinct configurations. You will also have to configure TFTP to point to the appropriate files.

    • Instructions are provided both at the top of the file and throughout the body of the file.

    • You need to know the IP Addresses of the YUM, Kickstart, and TFTP servers. (They default to the SIMP server in simp config).

    • Use the commands described in the comments at the top of the file to generate the root and GRUB passwords hashes. Be sure to replace password with your root password.

    • Follow the instructions throughout the file to customize for BIOS/UEFI boot.

    • Follow the instructions throughout the file to customize for FIPS/non-FIPS mode.

  4. Open the diskdetect.sh script and customize the disk device names and/or partitions as appropriate for your site. The sample diskdetect.sh script will work, as is, for most systems, as long as your disk device names are in the list. In addition, the sample script provides STIG-compliant partitioning.

  5. Type chown root.apache /var/www/ks/* to ensure that all files are owned by root and in the apache group.

  6. Type chmod 640 /var/www/ks/* to change the permissions so the owner can read and write the file and the apache group can only read.


Two major changes were made to pupclient_x86_64.cfg in SIMP 6.2:

  • UEFI PXE support was added.

  • To address timeout issues that caused Puppet bootstrap failures, the use of the runpuppet script to bootstrap Puppet on the client was replaced with the use of two scripts, both provided by the simp::server::kickstart class:

    • A systemd unit file

    • A common bootstrap script (bootstrap_simp_client) used by both.


The URLs and locations in the file are set up for a default SIMP install. That means the same OS and version as the SIMP server, all servers in one location (on the SIMP server) and in specific directories. If you have installed these servers in a different location than the defaults, you may need to edit URLs or directories.


If you want to PXE boot more than this operating system, make a copy of these files, name them appropriately and update URLS and links inside and anything else you may need. (You must know what you are doing before attempting this.) If you are booting more than one OS you must also make sure your YUM server has the OS packages for the other OSs. By default, the YUM server on SIMP has the packages only for the version of OS installed on the SIMP server.

4.4.2. Setting up TFTP

This section describes the process of setting up static files and manifests for TFTP. The example in this section assumes we are configuring the production SIMP Omni-Environment.


The tftp root directory was changed in SIMP 6.2 to conform to DISA STIG standards. In previous versions it was /tftpboot, and in 6.2 and later it is /var/lib/tftpboot. If you are upgrading to 6.2 from a prior release and wish the files to remain in the /tftpboot directory, set tftpboot::tftpboot_root_dir to /tftpboot in Hiera. Static Files

Verify the static files are in the correct location:

  1. cd into /var/simp/environments/production/rsync/OSTYPE/Global/tftpboot/

    ({OSTYPE} under rsync is the OS type of the SIMP server.)

  2. Verify there is a linux-install directory and cd to this directory.

  3. Under the linux-install directory you should find a directory named OSTYPE-MAJORRELEASE.MINORRELEASE-ARCH/ and a relative-link to this directory named OSTYPE-MAJORRELEASE-ARCH/.

  4. Under OSTYPE-MAJORRELEASE.MINORRELEASE-ARCH/, you should find the files:

    • initrd.img

    • vmlinuz

    If these files are not where they should be, then create the directories as needed and copy the files from /var/www/yum/<OSTYPE>/<MAJORRELEASE>/<ARCH>/images/pxeboot/ or from the images directory on the SIMP DVD. The link name is what is used in the resources in the tftpboot.pp manifest examples.


    The images in tftpboot/ need to match the distribution. For example, if you upgrade your repo from CentOS 7.3 to 7.4 and will be using this repo to kickstart machines, you must also upgrade the images in tftpboot/. If they do not match you may encounter errors, such as unknown file system type 'xfs'.

  5. Next, you need to set up the boot files for either BIOS boot mode, UEFI mode, or both.

For more information, see the RedHat 7 Installation Source or RedHat 8 Installation Source Installation Guides.


UEFI support has been automated since SIMP 6.2. If you are using an older version of SIMP please refer to that documentation for setting up UEFI manually. Dynamic Linux Model Files

Create a site profile module for the TFTP server on the Puppet Server to set up the various files to model different systems.

  1. Create the file tftpboot.pp in your site profile. This file will contain Linux models for different types of systems and a mapping of MAC addresses to each model.

    Use the source code example below. Linux model examples are given for CentOS 7 and 8 using both UEFI and BIOS boot mode.

    • Replace KSSERVER with the IP address of kickstart server (or the code to look up the IP Address using Hiera).

    • Replace OSTYPE, MAJORRELEASE and ARCH with the correct values for the systems you will be PXE booting.

    • MODEL NAME is usually of the form OSTYPE-MAJORRELEASE-ARCH for consistency.

    • You will need to know what kickstart file you are using. UEFI and BIOS mode require separate kickstart files. Other things that might require a different kickstart file to be configured are disk drive configurations and FIPS configuration. Create a different Linux model file for each different kickstart file needed.


      If using the default cfg files, know that they do not have the _el[8,7] tags at the end of their names.


      The simp_disk_crypt option shown below switches on transparent disk encryption as described in the Disk Encryption documentation and is recommended if you have a requirement for disk encryption and cannot enter a password at system boot time.

      Simply omit the option if you do not wish to use the capability.

      class site::tftpboot {
        include '::tftpboot'
        # for CentOS/RedHat 7 Legacy/BIOS boot
        tftpboot::linux_model { 'el7_x86_64':
          kernel => 'OSTYPE-MAJORRELEASE-ARCH/vmlinuz',
          initrd => 'OSTYPE-MAJORRELEASE-ARCH/initrd.img',
          ks     => "https://KSSERVER/ks/pupclient_x86_64_el7.cfg",
          extra  => "inst.noverifyssl ksdevice=bootif simp_disk_crypt\nipappend 2"
        # For CentOS/RedHat 8 Legacy/BIOS boot
        tftpboot::linux_model { 'el8_x86_64':
          kernel => 'OSTYPE-MAJORRELEASE-ARCH/vmlinuz',
          initrd => 'OSTYPE-MAJORRELEASE-ARCH/initrd.img',
          ks     => "https://KSSERVER/ks/pupclient_x86_64_el8.cfg",
          extra  => "inst.noverifyssl ksdevice=bootif simp_disk_crypt\nipappend 2"
        # NOTE UEFI boot uses the linux_model_efi module and has different
        # `extra` arguments. You also would use a different kickstart file
        # because the bootloader command within the kickstart file is
        # different. Read the instructions in the default pupclient_x86_64.cfg
        # file and make sure you have the correct bootloader line.
        # For CentOS/RedHat 7 UEFI boot
        tftpboot::linux_model_efi { 'el7_x86_64_efi':
          kernel => 'OSTYPE-MAJORRELEASE-ARCH/vmlinuz',
          initrd => 'OSTYPE-MAJORRELEASE-ARCH/initrd.img',
          ks     => "https://KSSERVER/ks/pupclient_x86_64_efi_el7.cfg",
          extra  => "inst.noverifyssl simp_disk_crypt"
        # For CentOS/RedHat 8 UEFI boot
        tftpboot::linux_model_efi { 'el8_x86_64_efi':
          kernel      => 'OSTYPE-MAJORRELEASE-ARCH/vmlinuz',
          initrd      => 'OSTYPE-MAJORRELEASE-ARCH/initrd.img',
          ks          => "https://KSSERVER/ks/pupclient_x86_64_el8.cfg",
          extra       => "inst.noverifyssl simp_disk_crypt"
        # If desired, create defaults boot configuration for BIOS and UEFI.
        # Note that the name of the default UEFI configuration file needs
        # to be 'grub.cfg'.
        tftpboot::assign_host { 'default': model => 'el7_x86_64' }
        tftpboot::assign_host_efi { 'grub.cfg': model => 'el7_x86_64_efi' }
        # For each system define what module you want to use by pointing
        # its MAC address to the appropriate model. Note that the MAC
        # address is preceded by ``01-``.
        tftpboot::assign_host { '01-aa-ab-ac-1d-05-11': model => 'el7_x86_64' }
        tftpboot::assign_host_efi { '01-aa-bb-cc-dd-00-11': model => 'el7_x86_64_efi' }
  2. Add the site::tftpboot class on your puppet server node via Hiera. Create the file (or edit if it exists): /etc/puppetlabs/code/environments/production/data/hosts/<tftp.server.fqdn>.yaml. (By default, the TFTP server is the same as your puppet server so it should exist.) Add the following example code to that yaml file.

      - 'site::tftpboot'
  3. After updating the above file, type puppet agent -t --tags tftpboot on the Puppet Server.


    To provide PXE boot configuration for more OSs, create, in the tftpboot.pp file, a tftpboot::linux_model or tftpboot::linux_model_efi block for each OS type. Then, assign individual hosts to each model by adding tftpboot::assign_host or tftpboot::assign_host_efi resources.

  4. Finally, make sure DHCP is set up correctly. In SIMP 6.2 the example dhcpd.conf was updated to determine the appropriate boot loader file to use, depending upon the boot mode of the PXE client. These changes are needed if you are booting UEFI systems.

For more information see the RedHat 8 PXE or RedHat 7 PXE Installation Guides.

4.5. Apply Certificates

All clients in a SIMP system should have Public Key Infrastructure (PKI) keypairs generated for the server. These are the referred to as the infrastructure or server keys. These certificates are used to encrypt communication and identify clients and are used by common applications such as LDAP and Apache.


These keypairs are not the keys that the Puppet server uses for its operation. Do not get the two confused.

See Certificate Management for more information.

SIMP uses the simp/pki module to help distribute infrastructure keypairs. The global variable, simp_options::pki determines what parts of the module are included. It can be overridden in hiera data at several levels if different hosts or applications need to handle certificates differently.

simp_options::pki can have one of three settings:

  1. simp - Keypairs are distributed from a central location on the Puppet server to the /etc/pki/simp/x509 directory on the client. Any applications using them will then make a copy in /etc/pki/simp_apps/<app name>/x509 with the correct permissions for an application to use.

  2. true - Applications on the clients will copy the keypairs from a local directory on the client to /etc/pki/simp_apps/<app name>/x509. The default local directory to copy from is /etc/pki/simp/x509 but this can be overridden by setting the simp_options::pki::source variable.

  3. false - The user will have to manage keypairs themselves. You will need to look at each module that uses PKI on a client to determine what variables need to be set.


    A setting of false does not disable the use of PKI in a module.

The following sections describe how to populate the central key distribution directory that :pupmod:’simp/pki` uses, when simp_options::pki is set to simp.

4.5.1. Installing Official Certificates

This section describes how to install infrastructure certificates from an official certificate authority on the SIMP Server for distribution to client servers. You need to have simp_options::pki set to simp on the client for this to work.

The key distribution directory on the SIMP Server is the site_files/pki_files/files/keydist sub-directory located under the SIMP Secondary Environment, /var/simp/environments/environment. Within the keydist/ directory, the SIMP system expects there to be:

  • A directory named cacerts/ that contains the CA public certificates.

  • Client-specific directories, each of which contains the public and private certificates for an individual client. The name of each client directory must be the certname of that client, which by default is the client’s FQDN.

Here is an example key distribution directory for a production SIMP Omni-Environment:


To install official certificates on the SIMP Server, do the following:

  1. Copy the certificates received from a proper CA to the SIMP server.

  2. Add the certificates for the node to the key distribution directory in site_files/.

    1. Make the directory under the key distribution directory for the client’s certificates using the client’s certname.

    2. Copy the official public and private certificates to that directory.

    For example, to install certificates for a system named mycomputer.my.domain into the production environment:

    mkdir -p /var/simp/environments/production/site_files/pki_files/files/keydist/mycomputer.my.domain
    mv /dir/where/the/certs/were/myprivatecert.pem \
    mv /dir/where/the/certs/were/mypubliccert.pub \
  3. Create and populate the CA certificates directory.

    1. Make the CA directory, cacerts/.

    2. Copy the root CA public certificates into cacerts/ in PEM format, one per file.

    cd /var/simp/environments/production/site_files/pki_files/files/keydist
    mkdir cacerts
    cd cacerts
    for file in *.pem; do ln -s $file `openssl x509 -in $file -hash -noout`.0; done
  4. Make sure the permissions are correct.

    chown -R root.puppet /var/simp/environments/production/site_files/pki_files/files/keydist
    chmod -R u=rwX,g=rX,o-rwx /var/simp/environments/production/site_files/pki_files/files/keydist


The site_files/ sub-directory of the SIMP Secondary Environment is configured as another module path in each Puppet Environment’s environment.conf. For example, for the production environment, /etc/puppetlabs/code/environments/production/environment.conf would contain:

modulepath = modules:/var/simp/environments/production/site_files:$basemodulepath

4.5.2. Generating Infrastructure Certificates from the Fake CA

The “Fake” (self-signing) Certificate Authority (Fake CA) is provided by SIMP as a way to manage server certificates if official certificates could not be obtained at the time of client installation or the servers are operating in testing environments.


This option should not be used for any operational system that can use proper enterprise PKI certificates.

Below are the steps to generate the certificates using the SIMP-provided, Fake CA. These steps assume the production environment.

  1. cd to /var/simp/environments/production/FakeCA/

  2. Run vi togen

    1. Remove old entries from the file and add the Fully Qualified Domain Name (FQDN) of the systems (one per line) for which certificates will be created.


      To use alternate DNS names for the same system, separate the names with commas and omit any spaces.

      For example, .name,alt.name1,alt.name2.

  3. Run wc cacertkey

    1. Verify that the cacertkey file is not empty.

    2. If it is empty: enter text into the file, then save and close the file.

  4. Run ./gencerts_nopass.sh


If the clean.sh command is run after the certificates have been generated, you will not be able to generate new host certificates under the old CA. To troubleshoot certificate problems, see the Troubleshooting Certificate Issues section.

If issues arise while generating keys, navigate to the /var/simp/environments/production/FakeCA/ directory, then type ./clean.sh to start over.

After running the clean.sh script, type ./gencerts_nopass.sh to run the script again using the previous procedure table.

The certificates generated by the FakeCA in SIMP are set to expire annually. To change this, edit the following files with the number of days for the desired lifespan of the certificates:

  • /var/simp/environments/production/FakeCA/CA

  • /var/simp/environments/production/FakeCA/ca.cnf

  • /var/simp/environments/production/FakeCA/default_altnames.cnf

  • /var/simp/environments/production/FakeCA/default.cnf

  • /var/simp/environments/production/FakeCA/user.cnf

In addition, any certificates that have already been created and signed will have a config file containing all of its details in /var/simp/environments/production/FakeCA/output/conf/.


Editing any entries in the above mentioned config files will not affect existing certificates. Existing certificates must be regenerated if you need to make changes.

The following is an example of how to change the expiration time from one year (the default) to five years for any newly created certificate:

for file in $(grep -rl 365 /var/simp/environments/production/FakeCA/)
   sed -i 's/365/1825/' $file

4.6. Setting up the Client

4.6.1. Existing Clients

Cloud environments, such as AWS, Azure, OpenStack, and GCE do not need to follow the PXE model shown below. Likewise, pre-existing physical clients can be integrated into the SIMP environment using the method outlined in this section.

The SIMP system contains a bootstrap script that is able to be downloaded from the server. You should examine the actual client application to determine if it meets your needs as written but, in general, it should be well suited most applications.

The following invocation waits for the server to provide a signed PKI certificate prior to proceeding. This is the safest method but will hang if the client certificate is not signed.

curl -k -O https://<puppet.server.fqdn>/ks/bootstrap_simp_client

# Use the puppet provided ruby for a guaranteed compatible version
/opt/puppetlabs/puppet/bin/ruby ./bootstrap_simp_client \
  --puppet-server <puppet.server.fqdn> \
  --puppet-ca <puppet.server.fqdn> \
  --puppet-wait-for-cert 0 \

4.6.2. PXE Booting

The following lists the steps to PXE boot the system and set up the client.

  1. Set up your client’s boot settings to boot off of the network.

  2. Make sure the MAC address of the client is set up in DHCP (see Configure DHCP for more info.)

  3. Restart the system.

  4. Once the client installs, reboots, and begins to bootstrap, it will check in for the first time.

  5. By default, Puppet will not autosign Puppet certificates, so the agent initially runs with --waitforcert enabled. This means the client will check in every 30 seconds for a signed certificate. Log on to the Puppet master and run puppetserver ca sign --certname <puppet.agent.fqdn>.

Upon successful deployment of a new client, it is highly recommended that LDAP administrative accounts be created.

4.6.3. Troubleshooting Puppet Issues

If the client has been kickstarted, but is not communicating with the Puppet master, try the following options:

  • Check the forward and reverse DNS entries on the client and server; both must be correct. The nslookup command will help here.

  • Check the time on the systems. More than an hour’s difference will cause serious issues with certificates.

  • Remove /etc/puppetlabs/puppet/ssl on the client system; run puppetserver ca clean --certname <CLIENT.FQDN> on the Puppet Server and try again.

If you are getting permission errors, make sure the SELinux context is correct on all files, as well as the owner and group permissions.

4.6.4. Troubleshooting Certificate Issues

If host certificates do not appear to be working, ensure that all certificates verify against the installed CA certificates.

The table below lists the steps to determine which certificates are working and which are not.

  1. Navigate to /var/simp/environments/production/site_files/pki_files/files/keydist/

  2. Run find . -name "*<YOUR.DOMAIN>.pub” -exec openssl verify -CApath cacerts {} ;

    The screen displays ./<Host Name>.<Your.Domain>/<Hostname>.<Your.Domain>.pub: OK If anything other than OK appears for each host, analyze the error and ensure that the CA certificates are correct.

    If the TXT_DB error number 2 appears, revoke the certificate that is being regenerated. The table below lists the steps to revoke the certificate.

  3. Navigate to the directory containing the CA certificates. For the FakeCA, it is /var/simp/environments/production/FakeCA/. The directory should contain the file default.cnf.

  4. Run

    OPENSSL_CONF=default.cnf openssl ca -revoke /var/simp/environments/production\
    /site_files/pki_files/files/keydist/*<Host to Revoke>*/*<Host to Revoke>*.pub