CI Workflows as Kubernetes First Class Citizens

  • Press

Oreon Lothamer
November 22, 2021 147 views

Laptop screen displays Tekton Pipelines

Back in July I wrote a blog post about our journey with ArgoCD. By leveraging the gitops methodology it made it easy for us to enable developers to deploy applications onto our K8s clusters. Each cluster had a repo for the app of apps where we could add a new application manifest and it would be deployed through ArgoCD. To make it even easier for our developers, who had varying degrees of experience with K8s, we setup a helm chart that had all the components we needed for a basic application. This way almost all the application manifests would look the same with just a few parameters changed like the container image location and tag.

 

The challenge with this approach was getting the applications into a CI workflow. We already had ArgoCD handling the CD for the applications, but how should they be built/tested/scanned/uploaded? It would erase all the ease of deployment if the developers had to engage another team to setup pipelines for their application or add the application to another system to deploy the CI workflows. Since everything else was already in K8s and using gitops we looked for a solution that was K8s native. Enter Tekton Pipelines.

 

Tekton Pipelines allowed us to define our CI process directly in K8s through CRDs (Tasks, Pipelines, TaskRuns, PipelineRuns, etc). They conveniently  have a catalog of predefined Tasks that allowed us to get many of the pieces for our Pipelines (like cloning from git, scanning with Sonarqube, building and uploading the docker image with kaniko, and syncing with ArgoCD). We created some additional Tasks for additional compliance checks, like running Chef Inspec and uploading to Heimdall and scanning with Dependency Track. We then pieced everything together with Pipelines triggered through Tekton Triggers (which provides additional CRDs for event handling). These core Tasks and Pipelines could be defined once and used by all applications.

 

The only pieces we needed to include with each application were the Triggers and TriggerBindings to kick off the PipelineRuns. And since they were defined through K8s manifests we could include those as part of our helm chart. We also included a TaskRun when the helm chart was deployed that would run a Task to add the github webhook for the repo to send to the Tekton EventListener defined in the Trigger. So, everything we needed for a developer to add an application to the cluster and have it fully automated through CI/CD was just to create a single application manifest in the app of apps repo.

 

There were some rough patches with the implementation of course. With Tekton Pipelines being relatively new and still in Beta, a lot of the examples and articles online are outdated. For instance, many of them use PipelineResources, which are deprecated in the Beta release. We deployed the Tekton Dashboard for easy visibility into the CI process, but it doesn’t support any auth mechanisms which poses a major security concern. We also ran into some quirks due to the use of ArgoCD with Tekton. Because a PipelineRun inherits the labels of the Pipeline it is running, it has the label that ArgoCD uses to track which application resources belong to and will erroneously be marked out of sync as part of that application. We were able to get around this by adding an exclusion to ArgoCD for PipelineRuns, but it is not ideal.

 

By having all applications defined in a uniform way with built-in CI/CD it makes it super easy to enable developers to iterate. We look forward to exploring additional ways we can leverage Tekton Pipelines, ArgoCD, and other cloud native applications to further empower developers and remove roadblocks in the deployment of applications.



More Stories

  • Another Contract Win

    Zara Ikram
    April 9, 2021

    The Centers for Medicare & Medicaid Services (CMS) awarded to Softrams, LLC a Task Order contract for the development of a modern Medicaid Drug Programs (MDP) suite of products. The Softrams team includes three subcontracting partners.

  • Mobile Security Often Overlooked

    Yusuf Richardson
    October 30, 2021

    The extensive usage of mobile devices in today’s work environment has led to a significant rise in the risk for data theft. Securing mobile devices requires a multi-layered approach and investment in enterprise solutions.

  • No Phishing Allowed!

    Harshita
    October 19, 2021

    Phishing is one of the most common forms of cyber attacks and is still a danger to users on the internet. This blog goes over what phishing is and some of the best ways to protect oneself from such attacks.