Creation of Foreman RPM packages with Docker containers
The ATIX development team spent a lot of time improving orcharhino’s build environment during the last scrum.
So far, we’ve been using Jenkins build jobs, which have done the job quite well. However, this approach is not always as flexible as we need it to be to maintain quality product delivery for our customers. The Jenkins build steps were mainly shell scripts configured directly in the Jenkins user interface.
Docker containers were used for the build process itself. Since the whole structure has grown historically, we had a lot of duplicates for almost identical tasks, which created almost identical Dockerfiles. This was generally not a big problem unless we needed to make changes. Depending on the changes needed, this could affect a whole bunch of files at once, resulting in a whole bunch of necessary changes and a waste of time and effort. We had to change that and improve it!
After playing around with different sandbox environments based on Jenkins pipelines, we came to the conclusion that the Jenkins pipeline route, i.e. a “code-based” build environment, would be the most elegant solution. Since we knew that the upstream Foreman team was successfully using tito / mock to build Foreman, we initially considered this tool. However, we came to the conclusion that building the RPM packages within Docker is at least as good as the tito / mock toolchain.
In addition, it has the advantage that the build environment only needs to be created from scratch once and reused instead of recreating it for each package that is in the build process. Overall, this means that we can reduce RPM build time with a Docker-based approach.
The challenges in creating Foreman packages are as follows
- there are many different Foreman packages
- multiple Git repositories: the RPM specification file and some resources are stored in foreman-packaging, but the main source code is in the Foreman Git repository
- the Foreman build requires a large number of dependencies
The goals we wanted to achieve with a Jenkins pipeline approach using Docker were:
- Acceleration of the entire construction process for a specific orcharhino release
- Everything as code – even the build instructions
- Reuse the good code from the old Jenkins build steps and rewrite some bad scripts
- DRY – Do not repeat yourself. No more duplicate codes, docker files etc.
- KISS – Keep it simple stupid, so that everyone can understand and improve the entire process
I would now like to show you how our new process ultimately works:
We use CentOS 7 as the host system. If you are using a different distribution, you may need to adapt some commands.
If you do not have Docker installed, please add the following repo:
[docker] name=Docker Repository baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/ enabled=1 gpgcheck=1 gpgkey=https://yum.dockerproject.org/gpg
Then install Docker with the following command:
yum install docker-engine
Start and activate the Docker service:
systemctl start docker systemctl enable docker
The next step is to pull the Docker container from the Docker Hub, as we will need it later as the basis for the RPM build container:
docker pull centos
To use the script foreman_docker_build, you must also install rpm-build and ruby:
yum install rpm-build ruby git
Now it’s time to check out the foreman_docker_build github repository and take a look at the README:
git clone https://github.com/ATIX-AG/foreman_docker_build.git
Switch to foreman_docker_build and start building your Foreman package
cd foreman_docker_build ./build_foreman_packages.sh -n foreman -b 1.15-stable
That’s it already. After the build is complete, the RPMs are located in the “foreman/RPM” directory. How does the whole thing work internally?
After cloning the foreman-packaging and the Foreman repository, the script collects all required build files and moves them to foreman/_pkg_build.
Then comes the interesting part.
The script uses erb (embedded Ruby) to dynamically create a Dockerfile and the script build_rpm.sh. These two files are stored in foreman/_pkg_build. The build_rpm.sh script is later used within the Docker container to create the RPM. The Dockerfile contains the build instructions for the Docker container. Since the script is also able to build other Foreman packages from the Foreman packaging distribution, the Dockerfile is generated dynamically based on variables located in foreman/_pkg_build/docker_vars.rb. The latter file is created by the build_foreman_packages.sh script.
The next step is to create the Docker container using the Dockerfile and some files in foreman/_pkg_build.
Finally, the Docker container is started to build the RPM. This can take some time, as the build dependencies from the foreman.spec file are now evaluated and installed in the Docker container. In the last step,
the RPM packages are created and saved in “foreman/RPM”.
Pretty simple, isn’t it?
You might also like
Der ATIX-Crew besteht aus Leuten, die in unterschiedlichen Bereichen tätig sind: Consulting, Development/Engineering, Support, Vertrieb und Marketing.
