Building foreman RPM packages using docker container
The ATIX development team spent a lot of time in the last Scrum improving the build environment of orcharhino. So far we’ve been using Jenkins build jobs, which did the job already pretty well. However this approach is not always as flexible as we need it for maintaining a valuable product delivery for our customers. The Jenkins build steps were mainly shell scripts which were directly configured in the Jenkins UI.
The build process itself utilised Docker-containers. Since the whole structure grew historically, we had a lot of duplicates for nearly identical tasks, generating nearly identical Dockerfiles. This was not a big issue in general, unless we had to make amendements. Depending on the necessary changes this could affect a bunch of files at once, causing a whole series of necessary changes and waste of time and effort. We needed to change and improve this!
After playing around with various sandbox environments based on Jenkins pipelines, we came to the conclusion that moving towards the Jenkins pipeline way, i.e. a “code-based” build environment would be the most elegant solution. As we knew that the upstream Foreman team is successfully using tito / mock to build Foreman, we first took this tool into consideration. However, our conclusion was that building the RPM packages within Docker is at least as good as the tito / mock tool chain.
On top it has the advantage of creating the building environment only once from scratch and reusing it, rather than recreating it for every package which is in the build process. Overall this means we can decrease the RPM building time, using a Docker based approach.
The challenging parts of creating Foreman packages are as follows:
- there are many different Foreman packages
- multiple git repositories: The RPM spec file and some resources are stored in foreman-packaging, however the main source code is in the foreman git repository
- the Foreman build requires a lot of dependencies
The objectives we wanted to achieve with a Jenkins pipeline approach, utilising Docker were:
- Speed up the overall building process for one specific orcharhino release
- Everything as code – even the build instructions
- Reuse the good code from the old Jenkins build steps and rewrite some bad scripts
- DRY – Don’t repeat yourself. No more duplicate codes, Dockerfiles, etc.
- KISS – Keep it simple stupid, so everyone can understand and improve the whole process
We are using CentOS 7 as the host system. In case of using another distribution, you may have to adapt some commands.
In case you don’t have Docker installed, please add the following repo:
[docker] name=Docker Repository baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/ enabled=1 gpgcheck=1 gpgkey=https://yum.dockerproject.org/gpg
Afterwards, install Docker with the following command:
yum install docker-engine
Start and enable the Docker service:
systemctl start docker systemctl enable docker
The next step is to pull the Docker container from the Docker Hub, as we need it later as a basis for the RPM build container:
docker pull centos
To use the foreman_docker_build script you will also need to install rpm-build and ruby:
yum install rpm-build ruby git
Now it’s time to checkout the foreman_docker_build github repository and take a look at the README:
git clone https://github.com/ATIX-AG/foreman_docker_build.git
Change to the foreman_docker_build and start building your Foreman package
cd foreman_docker_build ./build_foreman_packages.sh -n foreman -b 1.15-stable
That’s all. After the build is done, the RPMs are located in the directory „foreman/RPM“. How does it work internally?
After cloning the foreman-packaging and the Foreman repository the script collects all necessary build files and moves them to foreman/_pkg_build.
Then the interesting part happens.
The script uses erb (embedded ruby) to dynamically create a Dockerfile and the build_rpm.sh script. Those two files are stored in foreman/_pkg_build. The build_rpm.sh is later used inside the Docker container to build the RPM. The Dockerfile contains the build instructions for the Docker container. Since the script is able to create other Foreman packages from the Foreman-packaging distribution as well, the Dockerfile is dynamically generated, based on variables which are located in foreman/_pkg_build/docker_vars.rb. This latter file is written by the build_foreman_packages.sh script.
The next step is to build the Docker container using the Dockerfile and some files in foreman/_pkg_build.
Finally, the Docker container is started to build the RPM. This might take some time, since now the build dependencies from the foreman.spec file are evaluated and installed within the Docker container. In the last step,
the RPM packages are build and stored in „foreman/RPM“.
Pretty easy, isn’t it?
ATIX-Crew
Neueste Artikel von ATIX-Crew (alle ansehen)
- Kafka mit Ansible automatisieren - 10. August 2023
- Konfigurationsmanagement über verschiedene Netze mit AWX - 11. Juli 2023
- Vergangenheit, Gegenwart und Zukunft von pulp_deb - 22. Februar 2023
