I was recently listening to a podcast from devtools.fm and monolithic repos were brought up, and one of arguments for them, was that it is hard to do CI/CD for lots of repos. In this post we are going to talk about a couple different ways to easily manage CI/CD jobs for hundreds of repos, relatively easy, with either Github Actions or Jenkins.

Let’s talk about Jenkins. First, if you are not using shared libraries, you should be, and there will be another blog post about those later. Now for the plugin, we use Github Branch Source Plugin which can scan all repos for an organization or user, and based on a file in the repo, run a certain pipeline from a completely different repo. This is where some thought needs to used to come up with a naming schema for the files in the repos.

We use something like this:

• Jenkinsfile.config – File in the repo that holds environment variables loaded in the pipeline, and the file that is looked for, for the Pull Request jobs in the Github Org plugin.
  • Files linked to Jenkinsfile.config
    • Jenkinsfile.S3.config – Tells the Github Org plugin in Jenkins to run the S3 jobs.
    • Jenkinsfile.NODE.config – Tells the Github Org plugin in Jenkins to run the NODE jobs.
• Example content

  env.NODEJS = 'NodeJS-16.x'
  env.NODEJS_BUILD = 'true'
  env.NODEJS_BUILD_CMD = 'yarn --frozen-lockfile --audit'
  env.YARN_INSTALL = 'true'
  env.NODEJS_TEST_CMD = 'npm run test'
  env.SONAR_RUN_SCANNER_CLI = 'true'
  env.SONAR_SRC_DIR = '.'
  env.NODE_ENV='UNIT_TEST'
  // Define main branch
  env.MAIN_BRANCH = 'dev'
  // Define hotfix branch
  env.HOTFIX_BRANCH = 'hotfix'
  // CD Environment for auto deploy
  env.CD_ENVIRONMENT = 'dev'

Now all you need to do is create your Jenkinsfile pipelines and make sure you read in the Jenkinsfile.config to set your environment variables.

• Example of pipeline

  #!/usr/bin/env groovy
  
  node() {
  
    stage('Get Jenkinsfile.config') {
      env.JENKINS_AGENT = 'linux'
      env.NODEJS = 'NodeJS-LTS'
      env.SONAR_CREATE_PROJECT = 'true'
      env.SONARSCANNER = 'sonar-scanner-current'
      env.SONARSCANNERMSBUILD = 'sonar-scanner-msbuild-4.5.0.1761'
      env.SONAR_GO_COVERAGE_REPORT_PATH = 'coverage.out'
      env.SONAR_GO_TEST_REPORT_PATH = 'test-report.out'
      env.SONAR_RUN_MSBUILD = 'false'
      env.SONAR_RUN_QG = 'false'
      env.SONAR_RUN_SCANNER_CLI = 'false'
      env.SONAR_VERBOSE = 'false'
      env.SONAR_SRC_DIR = '.'
      env.SONAR_LCOV_PATHS = 'coverage/lcov.info'
      env.YARN_INSTALL = 'false'
      env.TESTS_MUST_PASS = 'true'
      checkout scm
      load "Jenkinsfile.config"
    }
  }
  
  pipeline {
    options {
      disableConcurrentBuilds()
      ansiColor('xterm')
    }
  agent { label env.JENKINS_AGENT }
  .......
  
  

Here is what the Github Org Plugin in Jenkins would look like for Pull Requests.

Here is what the Github Org Plugin in Jenkins would look like for builds that go to S3

With the above examples setup, it allows the developers of the repo’s to have some control of the CI/CD jobs as they can easily make changes to the Jenkinsfile.config file. We use this setup for an organization with over 200 repositories for all our pull requests, builds, and deploy jobs and over a dozen different pipelines for S3, Node, Ansible, .net, ASP, and Lambda. With this setup, it is very easy to add a new pipeline or support a new language, maybe 15 minutes more of work vs adding a custom pipeline to a single repo.