5.3.2.6.1. Pre-Release Checklist¶
The bulk of the work to release both EL 6 and EL 7 versions of a SIMP ISO is to verify that each ISO is ready for release. Below is the list of verifications that must be executed for each ISO, before proceeding with the release of that ISO. If any of these steps fail, the problem identified must be fixed before you can proceed with the tag and release steps.
- Update Policy Evaluation Response Reports
- Verify RPMs are available in PackageCloud
- Verify a valid Puppetfile exists
- Verify the Changelog.rst
- Verify the dependencies.yaml
- Verify the simp-core RPMs can be created
- Verify simp-core tests pass
- Verify ISOs can be created
- Verify SIMP ISO boot options work
- Verify component interoperability
- Verify otherwise untested capabilities
- Verify SIMP server RPM install
- Verify SIMP server RPM upgrade
- Verify SIMP server R10K install
5.3.2.6.1.1. Update Policy Evaluation Response Reports¶
Since one of the main goals of SIMP is to assist with compliance of various standards, we should add a response to the latest security scans that we use.
Given that most scanners are only one view on the world and often are not flexible enough to meet all possible solutions to a given policy, it is expected that there will be explanations of both false positives as well as helpful material on why the SIMP framework is compliant for the benefit of our users.
These scans should be added, as applicable, to the Evaluation Artifacts section of the documentation.
5.3.2.6.1.2. Verify RPMs are available in PackageCloud¶
This check is to verify that all artifacts used to create the ISO exist as signed RPMs in PackageCloud. This will include:
- SIMP-owned Puppet modules
- Other Puppet modules
- SIMP utility RPMs (
rubygem-simp-cli
,simp-adapter
,simp-utils
, etc.) simp-doc
- SIMP application RPMs
- External vendor application RPMs
- OS RPMs
For nearly all the projects listed in Puppetfile.tracking
, you can verify that
the RPMs for those projects exist by executing the pkg:check_published
Rake command:
Checkout the
simp-core
project.git clone https://github.com/simp/simp-core.git cd simp-core
Verify the
Puppetfile.tracking
file contains the component tags for the release.Execute the
pkg:check_published
Rake commandbundle exec rake pkg:check_published > check_published.out
Examine the
check_published.out
content to verify that, except for thesimp-doc
project, no projects listsRPM Publish Required:
orGit Release Tag Required:
. What you should see are lines such as:... Found Existing Remote RPM: pupmod-simp-stunnel-6.1.0-0.noarch.rpm Found Existing Remote RPM: pupmod-simp-sudo-5.0.3-0.noarch.rpm ...
Important
If you see a message like
Warning: Unable to generate build-specific YUM cache
, your results are invalid, as connection to PackageCloud failed.Manually verify the appropriate
simp-doc
RPM exists at PackageCloud.
5.3.2.6.1.2.1. For the external vendor RPMs¶
- Upload all vendor RPMs to the
VERSION_Dependencies
repository in PackageCloud. Any existing RPMs will not be overwritten.package_cloud push simp-project/VERSION_Dependencies/el/OS_MAJOR_VERSION /path/to/packages
Warning
DO NOT push any Core Operating System RPMs up to PackageCloud, those should be retrieved from official vendor sources.
5.3.2.6.1.3. Verify a valid Puppetfile exists¶
This check is to verify that that Puppetfile.tracking
file for the
simp-core
project is complete and accurate:
- It includes all the SIMP-owned Puppet modules, other Puppet modules that are dependencies of SIMP-owned Puppet modules, and utilities to configure the SIMP system when installed from ISO.
- The URL for each artifact corresponds to the tag for its signed, published RPM.
5.3.2.6.1.4. Verify the Changelog.rst¶
This check is to verify that the simp-core
Changelog.rst has
been updated:
- Manually inspect
5.3.2.6.1.5. Verify the dependencies.yaml¶
This check is to verify that simp-core/build/rpm/dependencies.yaml
contains the correct adjustments to the RPM dependencies, obsoletes,
requires, and/or release fields for any of the components listed
in the Puppetfile.tracking
file.
Manually inspect the file to verify there are entries for
- All non-SIMP Puppet modules that have more dependencies listed in
their
metadata.json
files than are actually required on a SIMP system. Each entry must list all the relevant dependencies in a:requires
element. - Any component that has changed name (e.g.
pupmod-saz-timezone
changing topupmod-simp-timezone
). Each entry must list the package and version obsoleted in an:obsoletes
element. - Any component for which for which the RPM release field must be
specified (e.g. a component with a RPM-packaging-only change).
Each entry must list a
:requires
element.
5.3.2.6.1.6. Verify the simp-core RPMs can be created¶
This check verifies that an RPM can be generated for simp-core
:
git clone https://github.com/simp/simp-core.git
cd simp-core/src/assets/simp
bundle update
bundle exec rake pkg:rpm
Note
This command will build the RPM for the OS of the server on which it was executed.
5.3.2.6.1.7. Verify simp-core tests pass¶
This check verifies that the simp-core
unit and acceptance test
have succeeded.
To verify that the simp-core
unit tests have succeeded, examine
the test results in TravisCI.
Navigate to the project’s TravisCI results page and verify the tests for the development branch to be tagged and released have passed. For our project, this page is https://travis-ci.org/simp/simp-core/branches
Important
If the tests in TravisCI fail, you must fix them before proceeding. The automated release procedures will only succeed, if the unit tests succeed in TravisCI.
To verify that the simp-core
acceptance tests have succeeded
Checkout the
simp-core
project for the last SIMP release.git clone https://github.com/simp/simp-core.git cd simp-core
Run the default
simp-core
acceptance testsbundle update bundle exec rake beaker:suites
Note
If the GitLab instance for simp-core
is current (it is sync’d
every 3 hours), you can look at the latest acceptance test results
run by GitLab, instead. The results will be at
https://gitlab.com/simp/simp-core/pipelines.
5.3.2.6.1.8. Verify ISOs can be created¶
This check verifies that SIMP ISOs for CentOS 6 and CentOS 7 can be
built from the local simp-core
clone and RPMs pushed to PackageCloud.
For CentOS 6 and CentOS 7:
Login to a machine that has Docker installed and the
docker
service running.Important
In our development environment, the version of Docker that is available with CentOS works best.
Checkout the
simp-core
project for the last SIMP release.git clone https://github.com/simp/simp-core.git cd simp-core
Populate
simp-core/ISO
directory with CentOS 6/7 distribution ISOsmkdir ISO cp /net/ISO/Distribution_ISOs/CentOS-6.9-x86_64-bin-DVD*.iso ISO/ cp /net/ISO/Distribution_ISOs/CentOS-7-x86_64-1708.iso ISO/
Build each ISO for CentOS 6 and CentOS 7. For example,
bundle update SIMP_BUILD_docs=no \ SIMP_BUILD_verbose=yes \ SIMP_PKG_verbose=yes \ bundle exec rake beaker:suites[rpm_docker]
Important
- By default, the
default.yml
for therpm_docker
suite builds an ISO for CentOS 7. You must manually edit thedefault.yml
file to disable theel7-build-server
instead of theel6-build-server
, in order to create a CentOS 6 ISO. - The most reliable way to build each ISO is from a clean checkout
of
simp-core
.
- By default, the
Verify none of the RPMs in the ISO that SIMP would have generated are signed by the SIMP development GPG key. For example, for a CentOS 7 build:
cd build/distributions/CentOS/7/x86_64/SIMP/RPMS/noarch # The 7da6f216 key ID may change as the SIMP signing keys get updated over time # The output of this command should be *EMPTY* rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} %{SIGPGP:pgpsig} %{SIGGPG:pgpsig}\n' -p * | grep -v 7da6f216
5.3.2.6.1.9. Verify SIMP ISO boot options work¶
This hefty check verifies that a server booted from the SIMP ISO can be bootstrapped for the ‘simp’ scenario and following boot options:
- Using default boot option
- Using disk encryption boot option
- Using FIPS disabled boot option
- Using disk encryption and FIPS disabled boot options
- Using simp-prompt option
- Using simp-prompt and disk encryption boot options
- Using simp-prompt and FIPS disabled boot options
- Using simp-prompt, disk encryption, and FIPS disabled boot options
- Using linux-min boot option
- Using linux-min and disk encryption boot options
- Using linux-min and FIPS disabled boot options
- Using linux-min, disk encryption, and FIPS disabled boot options
For the default boot options with/without encryption and the FIPS disabled boot option with/without encryption test cases, the simp-packer project is the easiest way to verify a SIMP VM can be booted from the ISO and bootstrapped. Otherwise, the check has to be done manually:
Boot a VM with the SIMP ISO
Select the appropriate boot options
Once the server boots, login to the server as root
Bootstrap the system
simp config simp bootstrap reboot
Login to the server as root and run
puppet agent -t
until the results are stableVerify the server is/is not in FIPS mode by inspecting /proc/sys/crypto/fips_enabled
Verify the appropriate disk is/is not encrypted by executing
blkid
Verify the appropriate disk partitioning
lsblk
Important
For the linux-min
test cases, the only verification required is
verification that the server boots up.
5.3.2.6.1.10. Verify component interoperability¶
This check verifies, with simp-core
and pupmod-simp-simp
acceptance tests, that this aggregation of module versions interoperate.
(These tests provide extensive, cross-component, integration tests.)
Note
If simp-core
and pupmod-simp-simp
acceptance tests have
effectively already passed on one of our continuous integration
platforms (e.g., in GitLab), you can skip this painful step.
However, you must be sure that the tests were run with the correct
component versions.
Checkout the
simp-core
project.git clone https://github.com/simp/simp-core.git cd simp-core
Verify the
Puppetfile.tracking
file contains the component tags for the release.Run the default
simp-core
acceptance testsbundle update bundle exec rake beaker:suites
Checkout the version of
pupmod-simp-simp
corresponding to thissimp-core
versionbundle exec rake deps:checkout cd src/puppet/modules/pupmod-simp-simp
Create a
.fixtures.yml
file that sets the version of each dependency to the version contained in thePuppetfile.tracking
file for this ISO release.Run all the functioning acceptance tests with and without FIPS mode enabled
bundle update BEAKER_fips=yes bundle exec rake beaker:suites bundle exec rake beaker:suites BEAKER_fips=yes bundle exec rake beaker:suites[base_apps] bundle exec rake beaker:suites[base_apps] BEAKER_fips=yes bundle exec rake beaker:suites[no_simp_server] bundle exec rake beaker:suites[no_simp_server] BEAKER_fips=yes bundle exec rake beaker:suites[scenario_one_shot] bundle exec rake beaker:suites[scenario_one_shot] BEAKER_fips=yes bundle exec rake beaker:suites[scenario_poss] bundle exec rake beaker:suites[scenario_poss] BEAKER_fips=yes bundle exec rake beaker:suites[scenario_remote_access] bundle exec rake beaker:suites[scenario_remote_access]
5.3.2.6.1.11. Verify otherwise untested capabilities¶
This check verifies that all other major capabilities (not otherwise tested in acceptance/simp-packer tests) do function as advertised:
Note
In order to speed time to market, the goal is to automate as many of these manual tests as possible!
A SIMP client can be PXE booted using the kickstart files from the SIMP ISO
A SIMP client can use the SIMP server for DNS
A ‘simp_lite’ client operates with a SIMP server
- login operations (PAM, LDAP, local user)
- NFS operations (home directory)
- logging operations (rsyslog)
- auditing operations
A ‘simp_poss’ client operates with a SIMP server
The SIMP server can be converted from FIPS enabled to FIPS disabled mode.
The SIMP server can be converted from Selinux enforcing to Selinux permissive.
The SIMP server can be converted from Selinux permissive to Selinux enforcing.
A local user with sudo privileges can be created and login to both the SIMP server and client on CentOS 6 and CentOS 7.
An LDAP user user in the
administrators
group can login to both the SIMP server and client on CentOS 6 and CentOS 7.Local and LDAP users can change their passwords on both the SIMP server and client on CentOS 6 and CentOS 7.
The Rsyslog rules from
simp_rsyslog
,syslog
and SIMP application modules (aide
,tlog
, etc.) result in application log messages being written to the correct local and remote log files.Note
Although the
simp_rsyslog
andsyslog
modules have excellent acceptance tests, neither has a full-system test to verify integration with actual log producers. The tests for these modules uselogger
as a mock message sender.The compliance map reports for a full SIMP system are accurate.
- No reports list non-compliant configuration that is really a parameter mismatches. (Parameter tested differs from parameter that should have been tested; value tested differs from actual values allowed, etc.)
- SIMP server and SIMP client reports are generated.
simp-utils
executables that are not tested otherwise work as advertisedunpack_dvd
gen_ldap_update
updaterepos
The SIMP HOWTO Guides are still correct.
5.3.2.6.1.12. Verify SIMP server RPM install¶
This check verifies that CentOS 6 and CentOS 7 SIMP servers can be
installed using the set of RPMs contained in the SIMP ISOs
The verification steps largely follow the details in
Installing SIMP From A Repository. All RPMs except
the simp-core
RPM should be able to be pulled from PackageCloud.
5.3.2.6.1.13. Verify SIMP server RPM upgrade¶
This check verifies that the set of RPMs in the SIMP ISO can upgrade the last full SIMP release.
Bring up a CentOS server that was booted from the appropriate SIMP ISO and for which
simp config
andsimp bootstrap
has been run.Note
If the VirtualBox for the last SIMP ISO was created by the simp-packer project, you can simply setup the appropriate VirtualBox network for that box and then bring up that bootstrapped image with
vagrant up
.Copy the SIMP and system RPMs packaged in the SIMP ISO to the server and install with yum.
FIXME Should put RPMs into appropriate updates repos, run something like the following
cd <updates dir> createrepo . chown -R root.apache ./* find . -type f -exec chmod 640 {} \; find . -type d -exec chmod 750 {} \; yum clean all; yum make cache yum update
Verify
puppet agent -t
runs cleanlyVerify no custom content is removed by the upgrade (e.g.,
environments/production/modules/site/manifests
, content inenvironments/production/data
)
5.3.2.6.1.14. Verify SIMP server R10K install¶
This check verifies that CentOS 6 and CentOS 7 SIMP servers can be
installed via r10k. Since this capability is already automatically
tested in a simp-core
acceptance test, all verification is handled by
Verify simp-core tests pass.