{"id":794,"date":"2018-09-02T18:34:52","date_gmt":"2018-09-02T16:34:52","guid":{"rendered":"https:\/\/blog.unetresgrossebite.com\/?p=794"},"modified":"2018-10-26T21:46:49","modified_gmt":"2018-10-26T19:46:49","slug":"packages-build-pipeline-with-openshift","status":"publish","type":"post","link":"https:\/\/blog.unetresgrossebite.com\/?p=794","title":{"rendered":"Packages Build Pipeline with OpenShift"},"content":{"rendered":"

As an other follow-up to my previous OpenShift posts, today we would look into Jenkins and Nexus integration with OpenShift, while building a dummy package shipping SSH Keys, both as a debien archive and RPM package.<\/p>\n

If you’re not concerned with automating Nexus configuration, then you may use sonatype\/nexus3<\/i> from the Docker hub setting up Nexus Repository Manager on OpenShift.
\nAs I wanted to automate a few configuration tasks, I eventually started working on my own image, forking from a repository offered by Accenture<\/a>. My copy isn’t yet released publicly, so I’ld just point out it creates a couple users uploading and downloading Artifacts.<\/p>\n

Another subject to address would be to prepare a couple images building our Debian and RPM packages. Regarding RPMs, we could divert from Jenkins base slave image<\/a>:<\/p>\n

FROM openshift\/jenkins-slave-base-centos7<\/p>\n

RUN yum -y install epel-release \\
\n    && yum -y install @development-tools centos-packager rpmdevtools \\
\n    && yum -y install make wget git curl<\/p>\n

USER 1001<\/p><\/blockquote>\n

While for Debian we would want to build some Stretch-based equivalent:<\/p>\n

FROM debian:stretch<\/p>\n

ENV HOME=\/home\/jenkins \\
\n    DEBIAN_FRONTEND=noninteractive<\/p>\n

USER root<\/p>\n

ADD config\/* \/usr\/local\/bin\/<\/p>\n

RUN apt-get -y update \\
\n    && apt-get -y install bc gettext git subversion openjdk-8-jre-headless gnupg curl wget \\
\n                lsof rsync tar unzip debianutils zip bzip2 make gcc g++ devscripts debhelper \\
\n    && apt-get clean \\
\n    && rm -rf \/var\/lib\/apt\/lists\/* \\
\n    && mkdir -p \/home\/jenkins \\
\n    && chown -R 1001:0 \/home\/jenkins \\
\n    && chmod -R g+w \/home\/jenkins \\
\n    && chmod 664 \/etc\/passwd \\
\n    && chmod -R 775 \/etc\/alternatives \/usr\/lib\/jvm \\
\n    && chmod 775 \/usr\/bin \/usr\/share\/man\/man1<\/p>\n

USER 1001<\/p>\n

ENTRYPOINT [“\/usr\/local\/bin\/run-jnlp-client”]<\/p><\/blockquote>\n

From there, the last item we’ll need, building our packages, is their sources.<\/p>\n

Building RPMs, we would write a Spec file such as the following:<\/p>\n

\nSummary: My Package
\nName: my-package
\nVersion: 0.0.1
\nRelease: 1%{?dist}
\nLicense: MIT
\nSource: https:\/\/repo\/sources\/el-%{name}-%{version}.tar.gz
\nURL: https:\/\/my.example.com<\/p>\n

Autoreq: no
\nBuildRequires: git
\nBuildRequires: make<\/p>\n

%description
\nDoes something awesome<\/p>\n

%global __os_install_post %{nil}
\n%define debug_package %{nil}
\n%prep
\n%autosetup
\n%build
\n%install
\nmake install PREFIX=%{buildroot}<\/p>\n

%pre
\n%prerun
\n%post
\n%files
\n%defattr(-,root,root)
\n%dir %{_datadir}\/mydir
\n%{_datadir}\/mydir\/myfile<\/p>\n

%changelog
\n * Thu Aug 30 2018 It’s Me <mario@example.com> 0.0.1-1
\n – Initial release – In an other castle?\n<\/p><\/blockquote>\n

Now regarding Debian packages, we would need to create a couple subdirectories, configuration files and scripts:<\/p>\n

$ mkdir -p debian\/source
\n$ echo “3.0 (quit)” >debian\/source\/format
\n$ echo 9 >debian\/compat
\n$ for i in postinst preinst prerm pstrm; do
\ncat <<EOF >debian\/$i
\n#!\/bin\/sh
\n# $i script for my-package<\/p>\n

set -e<\/p>\n

case “$1” in
\n  purge|remove|abort-install|disappear) ;;<\/p>\n

  upgrade|failed-upgrade|abort-upgrade) ;;<\/p>\n

  *)
\n    echo “postrm called with unknown argument \\`$1′” >&2
\n    exit 1
\n    ;;
\nesac<\/p>\n

#DEBHELPER#<\/p>\n

exit 0
\nEOF
\nchmod +x debian\/$i
\ndone
\n$ for i in docs copyright missing-sources README.Debian; do
\ntouch $i
\ndone
\n$ cat <<EOF >debian\/rules
\n#!\/usr\/bin\/make -f
\n#DH_VERBOSE = 1<\/p>\n

DPKG_EXPORT_BUILDFLAGS = 1
\ninclude \/usr\/share\/dpkg\/default.mk<\/p>\n

# see FEATURE AREAS in dpkg-buildflags(1)
\nexport DEB_BUILD_MAINT_OPTIONS = hardening=+all<\/p>\n

# main packaging script based on dh7 syntax
\n%:
\n        dh $@<\/p>\n

override_dh_auto_install:
\n        $(MAKE) install PREFIX=$(CURDIR)\/debian\/my-package<\/p>\n

override_dh_auto_build:
\n        echo nothing to do<\/p>\n

override_dh_auto_test:
\n        echo nothing to do
\nEOF
\n$ chmod +x debian\/rules
\n$ cat <<EOF >debian\/changelog
\nmy-package (0.0.1-1) unstable; urgency=low<\/p>\n

  * Initial release – In an other castle?<\/p>\n

— It’s Me <mario@example.com> Thu, 30 Aug 2018 11:30:42 +0200
\nEOF\n<\/p><\/blockquote>\n

From there, we ensure our sources ships with a Makefile, providing with the following rules:<\/p>\n

\nSHARE_DIR = $(PREFIX)\/usr\/share<\/p>\n

createdebsource:
\n    LANG=C debuild -S -sa<\/p>\n

createdebbin:
\n    LANG=C dpkg-buildpackage -us -uc<\/p>\n

createrpm:
\n    versionNumber=`awk ‘\/^Version:\/{print $$2;exit;}’ el\/my-package.spec`; \\
\n    wdir=”`pwd`\/..”; \\
\n    buildroot=”$$wdir\/rpmbuild”; \\
\n    for d in SOURCES SPECS BUILD RPMS SRPMS; \\
\n    do \\
\n      mkdir -p “$$buildroot\/$$d”; \\
\n    done; \\
\n    cp -p “$$wdir\/el-my-package-$$versionNumber.tar.gz” “$$buildroot\/SOURCES\/”; \\
\n    cp -p “$$wdir\/my-package\/el\/my-package.spec” “$$buildroot\/SPECS\/”; \\
\n    if ! whoami >\/dev\/null 2>&1; then \\
\n      chown -R root:root “$$buildroot\/SOURCES” “$$buildroot\/SPECS”; \\
\n    elif whoami 2>\/dev\/null | grep default >\/dev\/null; then \\
\n      chown -R :root “$$buildroot\/SOURCES” “$$buildroot\/SPECS”; \\
\n    fi; \\
\n    ( \\
\n      cd “$$buildroot”; \\
\n      LANG=C rpmbuild –define “_topdir $$buildroot” -ba SPECS\/my-package.spec && \\
\n      find *RPMS -type f | while read output; \\
\n        do \\
\n          mv “$$output” $$wdir\/; \\
\n        done; \\
\n    )<\/p>\n

createinitialarchive:
\n    rm -fr .git .gitignore README.md
\n    versionNumber=`cat debian\/changelog | awk ‘\/my-package\/{print $$2;exit}’ | sed -e ‘s|[()]||g’ -e ‘s|\\\\(.*\\\\)-[0-9]*\\$$|\\\\1|’`; \\
\n    ( \\
\n      cd ..; \\
\n      tar -czf my-package_$$versionNumber.orig.tar.gz my-package; \\
\n      mv my-package my-package-$$versionNumber; \\
\n      tar -czf el-my-package-$$versionNumber.tar.gz my-package-$$versionNumber; \\
\n      mv my-package-$$versionNumber my-package; \\
\n    )<\/p>\n

install:
\n    mkdir -p $(SHARE_DIR)\/mydir
\n    install -c -m 0644 myfile $(SHARE_DIR)\/mydir\/myfile\n<\/p><\/blockquote>\n

At which point, we may use the following OpenShift Template, creating a few secrets and a pair of Jenkins Pipelines, building Debian and RPM packages based on our previous images, then uploading their Artifacts to Nexus :<\/p>\n

\napiVersion: v1
\nkind: Template
\nmetadata:
\n  name: my-package-template
\nobjects:
\n– apiVersion: v1
\n  kind: Secret
\n  metadata:
\n    annotations:
\n      jenkins.io\/credentials-description : ${APPLICATION_NAME} Git Token credential from Kubernetes
\n    labels:
\n      jenkins.io\/credentials-type: secretText
\n    name: git-${APPLICATION_NAME}
\n  stringData:
\n    text: ${GIT_DEPLOYMENT_TOKEN}
\n– apiVersion: v1
\n  kind: Secret
\n  metadata:
\n    annotations:
\n      jenkins.io\/credentials-description : ${APPLICATION_NAME} Nexus Credentials from Kubernetes
\n    labels:
\n      jenkins.io\/credentials-type: usernamePassword
\n    name: nexus-${APPLICATION_NAME}
\n  stringData:
\n  password: ${NEXUS_ARTIFACTS_PASSWORD}
\n  username: ${NEXUS_ARTIFACTS_USERNAME}
\n– apiVersion: v1
\n  kind: BuildConfig
\n  metadata:
\n    annotations:
\n      description: Builds ${APPLICATION_NAME} rpm archive
\n    name: ${APPLICATION_NAME}-rpm
\n  spec:
\n    strategy:
\n      jenkinsPipelineStrategy:
\n        jenkinsfile: |-
\n          try {
\n            def pkgname = “${APPLICATION_NAME}”
\n            def label = “${pkgname}-${UUID.randomUUID().toString()}”
\n            podTemplate(label: label, name: label, cloud: ‘openshift’,
\n                containers: [ containerTemplate(name: ‘jnlp’, image: ‘${DOCKER_REGISTRY}\/${CENTOS_IMAGE}’) ],
\n                inheritFrom: ‘nodejs’, serviceAccount: ‘jenkins’) {
\n              timeout(time: 40, unit: ‘MINUTES’) {
\n                node (label) {
\n                  stage(“Fetch”) {
\n                    sh “git config –global http.sslVerify false”
\n                    sh “mkdir ${pkgname}”
\n                    withCredentials([string(credentialsId: “git-${pkgname}”, variable: ‘GIT_TOKEN’)]) {
\n                      sh “echo ‘${SOURCE_REPOSITORY_URL}’ | sed ‘s|^\\\\(http[s]*:\/\/\\\\)\\\\(.*\\\\)|\\\\1${GIT_TOKEN}@\\\\2|’ >cloneFrom 2>\/dev\/null”
\n                    }
\n                    def cloneAddress = readFile(‘cloneFrom’).trim()
\n                    dir (“${pkgname}”) {
\n                      git([ branch: “master”, changelog: false, poll: false, url: cloneAddress ])
\n                    }
\n                  }
\n                  stage(“Build”) {
\n                    sh “””
\n                    ( cd ${pkgname} ; git rev-parse –short HEAD ) >gitHash
\n                    ( cd ${pkgname} ; make createinitialarchive ; make createrpm )
\n                    awk ‘\/^Release:\/{print \\$2;exit;}’ ${pkgname}\/el\/${pkgname}.spec | cut -d% -f1 >patchNumber
\n                    awk ‘\/^Version:\/{print \\$2;exit;}’ ${pkgname}\/el\/${pkgname}.spec >versionNumber
\n                    “””
\n                  }
\n                  stage(“Upload”) {
\n                    def gitHash = readFile(‘gitHash’).trim()
\n                    def patch = readFile(‘patchNumber’).trim()
\n                    def version = readFile(‘versionNumber’).trim()
\n                    sh “echo Uploading artifacts for ${version}-${patch}-${gitHash}”
\n                    nexusArtifactUploader(
\n                      nexusVersion: ‘${NEXUS_VERSION}’,
\n                      protocol: “${NEXUS_PROTO}”,
\n                      nexusUrl: “${NEXUS_REMOTE}”,
\n                      groupId: “${NEXUS_GROUP_ID}”,
\n                      version: “${version}-${patch}-${gitHash}”,
\n                      repository: “${NEXUS_RPM_REPOSITORY}”,
\n                      credentialsId: “nexus-${pkgname}”,
\n                      artifacts: [
\n                      [ artifactId: “${pkgname}-rpm”,
\n                      classifier: ”, type: ‘rpm’,
\n                      file: “${pkgname}-${version}-${patch}.el7.src.rpm” ],
\n                      [ artifactId: “${pkgname}-rpm”,
\n                      classifier: ”, type: ‘rpm’,
\n                      file: “${pkgname}-${version}-${patch}.el7.x86_64.rpm” ],
\n                      [ artifactId: “${pkgname}-rpm”,
\n                      classifier: ”, type: ‘tar.gz’,
\n                      file: “el-${pkgname}-${version}.tar.gz” ]
\n                      ]
\n                    )
\n                  }
\n                }
\n              }
\n            }
\n          } catch (err) {
\n            echo “in catch block”
\n            echo “Caught: ${err}”
\n            currentBuild.result = ‘FAILURE’
\n            throw err
\n          }
\n      type: JenkinsPipeline
\n– apiVersion: v1
\n  kind: BuildConfig
\n  metadata:
\n    annotations:
\n      description: Builds ${APPLICATION_NAME} deb archive
\n    name: ${APPLICATION_NAME}-deb
\n  spec:
\n    strategy:
\n      jenkinsPipelineStrategy:
\n        jenkinsfile: |-
\n          try {
\n            def pkgname = “${APPLICATION_NAME}”
\n            def label = “${pkgname}-${UUID.randomUUID().toString()}”
\n            podTemplate(label: label, name: label, cloud: ‘openshift’,
\n                containers: [ containerTemplate(name: ‘jnlp’, image: ‘${DOCKER_REGISTRY}\/${DEBIAN_IMAGE}’) ],
\n                inheritFrom: ‘nodejs’, serviceAccount: ‘jenkins’) {
\n              timeout(time: 40, unit: ‘MINUTES’) {
\n                node (label) {
\n                  stage(“Fetch”) {
\n                    sh “git config –global http.sslVerify false”
\n                    sh “mkdir ${pkgname}”
\n                    withCredentials([string(credentialsId: “git-${pkgname}”, variable: ‘GIT_TOKEN’)]) {
\n                      sh “echo ‘${SOURCE_REPOSITORY_URL}’ | sed ‘s|^\\\\(http[s]*:\/\/\\\\)\\\\(.*\\\\)|\\\\1${GIT_TOKEN}@\\\\2|’ >cloneFrom 2>\/dev\/null”
\n                    }
\n                    def cloneAddress = readFile(‘cloneFrom’).trim()
\n                    dir (“${pkgname}”) {
\n                      git([ branch: “master”, changelog: false, poll: false, url: cloneAddress ])
\n                    }
\n                  }
\n                  stage(“Build”) {
\n                    sh “””
\n                    ( cd ${pkgname} ; git rev-parse –short HEAD ) >gitHash
\n                    ( cd ${pkgname} ; make createinitialarchive ; make createdebbin )
\n                    cat ${pkgname}\/debian\/changelog | awk ‘\/${pkgname}\/{print \\$2;exit}’ | sed -e ‘s|[()]||g’ -e ‘s|.*-\\\\([0-9]*\\\\)\\$|\\\\1|’ >patchNumber
\n                    cat ${pkgname}\/debian\/changelog | awk ‘\/${pkgname}\/{print \\$2;exit}’ | sed -e ‘s|[()]||g’ -e ‘s|\\\\(.*\\\\)-[0-9]*\\$|\\\\1|’ >versionNumber
\n                    “””
\n                  }
\n                  stage(“Upload”) {
\n                    def gitHash = readFile(‘gitHash’).trim()
\n                    def patch = readFile(‘patchNumber’).trim()
\n                    def version = readFile(‘versionNumber’).trim()
\n                    sh “echo Uploading artifacts for ${version}-${patch}-${gitHash}”
\n                    nexusArtifactUploader(
\n                      nexusVersion: ‘${NEXUS_VERSION}’,
\n                      protocol: “${NEXUS_PROTO}”,
\n                      nexusUrl: “${NEXUS_REMOTE}”,
\n                      groupId: “${NEXUS_GROUP_ID}”,
\n                      version: “${version}-${patch}-${gitHash}”,
\n                      repository: “${NEXUS_DEB_REPOSITORY}”,
\n                      credentialsId: “nexus-${pkgname}”,
\n                      artifacts: [
\n                      [ artifactId: “${pkgname}-deb”,
\n                      classifier: ”, type: ‘deb’,
\n                      file: “${pkgname}_${version}-${patch}_all.deb” ],
\n                      [ artifactId: “${pkgname}-deb”,
\n                      classifier: ”, type: ‘txt’,
\n                      file: “${pkgname}_${version}-${patch}_amd64.buildinfo” ],
\n                      [ artifactId: “${pkgname}-deb”,
\n                      classifier: ”, type: ‘txt’,
\n                      file: “${pkgname}_${version}-${patch}_amd64.changes” ],
\n                      [ artifactId: “${pkgname}-deb”,
\n                      classifier: ”, type: ‘tar.xz’,
\n                      file: “${pkgname}_${version}-${patch}.debian.tar.xz” ],
\n                      [ artifactId: “${pkgname}-dev”,
\n                      classifier: ”, type: ‘tar.gz’,
\n                      file: “${pkgname}_${version}.orig.tar.gz” ],
\n                      [ artifactId: “${pkgname}-deb”,
\n                      classifier: ”, type: ‘txt’,
\n                      file: “${pkgname}_${version}-${patch}.dsc” ]
\n                      ]
\n                    )
\n                  }
\n                }
\n              }
\n            }
\n          } catch (err) {
\n            echo “in catch block”
\n            echo “Caught: ${err}”
\n            currentBuild.result = ‘FAILURE’
\n            throw err
\n          }
\n      type: JenkinsPipeline
\nparameters:
\n– name: APPLICATION_NAME
\n  description: Package Name – should match that expected by package we’ll build
\n  displayName: Package Name
\n  value: my-package
\n– name: DEBIAN_IMAGE
\n  description: Jenkins Debian Agent Image – relative to DOCKER_REGISTRY
\n  displayName: Jenkins Debian Agent Image
\n  required: true
\n  value: “cicd\/jenkins-agent-debian:latest”
\n– name: DOCKER_REGISTRY
\n  description: Docker Registry
\n  displayName: Docker Registry
\n  required: true
\n  value: docker-registry.default.svc:5000
\n– name: CENTOS_IMAGE
\n  description: Jenkins Centos Agent Image – relative to DOCKER_REGISTRY
\n  displayName: Jenkins Centos Agent Image
\n  required: true
\n  value: “cicd\/jenkins-agent-centos:latest”
\n– name: GIT_DEPLOYMENT_TOKEN
\n  description: Git deployment token
\n  displayName: Git Deployment Token
\n  required: true
\n– name: NEXUS_ARTIFACTS_PASSWORD
\n  description: Nexus Artifacts Upload Password
\n  displayName: Nexus Artifacts Upload Password
\n  required: true
\n  value: admin123
\n– name: NEXUS_ARTIFACTS_USERNAME
\n  description: Nexus Artifacts Upload Username
\n  displayName: Nexus Artifacts Upload Username
\n  required: true
\n  value: admin
\n– name: NEXUS_GROUP_ID
\n  description: Nexus Group ID
\n  displayName: Nexus Group ID
\n  required: true
\n  value: com.example
\n– name: NEXUS_DEB_REPOSITORY
\n  description: Nexus Artifact Debian Repository – remote repository name
\n  displayName: Nexus Artifact Debian Repository
\n  required: true
\n  value: debian
\n– name: NEXUS_PROTO
\n  description: Nexus Proto – http or https
\n  displayName: Nexus Proto
\n  required: true
\n  value: http
\n– name: NEXUS_REMOTE
\n  description: Nexus Remote URL – proto-less URI connecting to Nexus
\n  displayName: Nexus Remote URL
\n  value: “nexus:8081”
\n  required: true
\n– name: NEXUS_RPM_REPOSITORY
\n  description: Nexus Artifact EL Repository – remote repository name
\n  displayName: Nexus Artifact EL Repository
\n  required: true
\n  value: centos
\n– name: NEXUS_VERSION
\n  description: Nexus Repository Version
\n  displayName: Nexus Repository Version
\n  required: true
\n  value: nexus3
\n– name: SOURCE_REPOSITORY_URL
\n  description: The URL of the repository with your application source code
\n  displayName: Git Repository URL
\n  required: true
\n  value: https:\/\/git.example.com\/project\/my-package\n<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"

As an other follow-up to my previous OpenShift posts, today we would look into Jenkins and Nexus integration with OpenShift, while building a dummy package shipping SSH Keys, both as a debien archive and RPM package. If you’re not concerned with automating Nexus configuration, then you may use sonatype\/nexus3 from the Docker hub setting up […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[12,10,13],"tags":[],"_links":{"self":[{"href":"https:\/\/blog.unetresgrossebite.com\/index.php?rest_route=\/wp\/v2\/posts\/794"}],"collection":[{"href":"https:\/\/blog.unetresgrossebite.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.unetresgrossebite.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.unetresgrossebite.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.unetresgrossebite.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=794"}],"version-history":[{"count":9,"href":"https:\/\/blog.unetresgrossebite.com\/index.php?rest_route=\/wp\/v2\/posts\/794\/revisions"}],"predecessor-version":[{"id":819,"href":"https:\/\/blog.unetresgrossebite.com\/index.php?rest_route=\/wp\/v2\/posts\/794\/revisions\/819"}],"wp:attachment":[{"href":"https:\/\/blog.unetresgrossebite.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=794"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.unetresgrossebite.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=794"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.unetresgrossebite.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=794"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}