Pushing Docker images to DockerHub with CircleCI

In the previous post I described how to version Docker images using Git and Gradle. Now I’m going to show how to push the image to DockerHub repository using Circle CI 2.0
We will be using:

• Circle CI 2.0
• DockerHub
• GitHub
• Git
• Gradle 4.1
• Git-Version Gradle Plugin 0.7.3
• Gradle-Docker plugin

If you prefer reading code than text you can skip reading and go straight to GitHub:
https://github.com/rgrebski/gradle-docker-push-to-private-repo

Pushing Docker images to DockerHub with CircleCI – Introduction

At Stepwise we love to automate things. Especially when it comes to tests (smoke, unit, integration, e2e, etc.) and automated deployment. In the previous post I described how to version Docker images. In this post I’m going to show you how we automate pushing Docker images.
During almost 10 years of software development we worked with different CI/CD servers like Jenkins, Bamboo (R.I.P.), Team City, GitLab CI, recently Bitbucket Pipeline and CircleCI. BitBucket Pipeline was missing multiple features we were interested in (Docker Daemon support, Test Reports, Dependencies caching etc.) that’s why we switched to CircleCI 2 which seems to be mature, feature rich CI/CD server in a cloud.

Project configuration

I am using following accounts for project setup:

• GitHub account — git repo
• DockerHub account — Docker images repository
• CircleCI account — Continuous Integration server

Gradle configuration

I am using a code from the previous post related to versioning Docker images as a base, it already has versioning implemented so whats left is to slightly update build.gradle and configure Circle CI to build and push Docker image to DockerHub.
In the previous post I had buildDocker gradle task which I have modified slightly (changed image name):

task buildDocker(type: Docker, dependsOn: build) {
    push = project.findProperty('push')?.asBoolean() ?: false    baseImage "java:8"
    maintainer 'Radek Grebski '
    applicationName = jar.baseName
    tagVersion = gitVersion()
    tag = "rgrebski/gradle-docker-push-to-private-repo"
    addFile(jar.archivePath, 'app.jar')
    runCommand("sh -c 'touch /app.jar'")
    exposePort(8080)
    entryPoint([ "sh", "-c", 'java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar' ])    doFirst {
        copy {
            from jar
            into stageDir
        }
    }
}

Having a task that builds Docker image with proper version (gitVersion() is calculating current version basing on git) it’s time to push the image to our Docker repository. The easiest way for me is to write another Gradle task that is going to push the image to DockerHub:

task pushDockerWithGitVersion(type: Exec, dependsOn: 'buildDocker') {
    def newImageTag = "rgrebski/gradle-docker-push-to-private-repo:${gitVersion()}"
    logger.warn("Pushing new image: $newImageTag")    commandLine 'bash', '-e', '-c', """
        docker login -u \$DOCKER_USERNAME -p \$DOCKER_PASSWORD
        docker push $newImageTag
        """
    doLast {
        logger.warn("Pushed new image: $newImageTag")
    }
}

The above task is pretty simple, it’s of type Exec and depends on buildDocker task, so we are sure that Docker image has been build and is present in local Docker repository.
Line 2 defines the image tag with proper version.
Lines 5–8 do the job — its pushing image to our DockerHub repository. Modern CI servers use Docker images for performing builds and this is the same for Circle CI. This is why in line 6 we have to perform docker login command to create auth file inside a container (DOCKER_USERNAME and DOCKER_PASSWORD should be set as environment variables on CI side). Then on line 7 we call docker push command which simply pushes the image to Docker repository.

Circle CI configuration

Having Gradle properly configured it’s time to configure Circle CI YAML file. To do so we need to create .circleci/config.yml file within our project. I wanted to create following workflow:

• checkout code
• build
• test
• push the image if branch is master

Here is my config.yml I ended up with (the one below contains a looooot of comments, see cleaner version of config.yml on GitHub).

# lets define default values for jobs and give it a name 'workdirAndImage'
defaults: &workdirAndImage
  working_directory: ~/workspace
  docker:
    - image: circleci/openjdk:8-jdk# versions is 2 for CircleCI 2.0
version: 2# 1) jobs defined here are going to be used in workflows (pipeline). You can treat jobs as steps in workflow/pipeline
# 2) jobs in workflow are run in SEPARATE docker containers, so techniques like caching or storing to workspace is a common thing
# 3) jobs don't have to be run within a workflow, they can be run separately, see CircleCI documentation for that
jobs:  checkout_code:
    #apply defaults defined at the top of the config (working dir + docker image)
    <<: *workdirAndImage    #here we are defining steps for given job/step
    steps:
      # checkout is a built-in step which simply pulls git repository. path parameter is optional
      - checkout:
          path: ~/workspace/repo      # we checked out the code in the previous stepw, lets store it to the workspace
      - persist_to_workspace:
          root: ~/workspace
          paths:
            - repo/  build:
    # restore defaults named 'workdirAndImage'
    <<: *workdirAndImage
    # override working directory (its defined as ~/workspace in 'workdirAndImage'), we want work on checked out code
    working_directory: ~/workspace/repo    steps:
      # restore workspace - in checkout_code step we persisted checked out code under ~/workspace/repo
      - attach_workspace:
          at: ~/workspace
      # restore cache (saving it is at the end of this job), it contains downloaded dependencies + build artifacts.
      - restore_cache:
          keys:
          # this key relates to build.gradle. If this file has not been changed since the last build, cache will be used
          # {{ checksum "build.gradle" }} simply tells Circle CI to calculate checksum from build.gradle
          - v2-dependencies-{{ checksum "build.gradle" }}
          # fallback to using the latest cache if no exact match is found
          - v2-dependencies-      # build but skip tests
      - run: gradle build -x test      # after performing build lets store dependencies and build artifacts
      - save_cache:
          paths:
            - ~/.gradle
            - ~/.m2
          key: v2-dependencies-{{ checksum "build.gradle" }}
  test:
    <<: *workdirAndImage
    working_directory: ~/workspace/repo    steps:
      - attach_workspace:
          at: ~/workspace
      # before running tests, lets restore cache with dependencies and artifacts
      - restore_cache:
          keys:
          - v2-dependencies-{{ checksum "build.gradle" }}
          # fallback to using the latest cache if no exact match is found
          - v2-dependencies-      # run tests
      - run: gradle test      # save test reports in ~/junit dir
      - run:
          name: Save test results
          command: |
            mkdir -p ~/junit/
            find . -type f -regex ".*/build/test-results/.*xml" -exec cp {} ~/junit/ \;
          when: always      # let CircleCI know where test results are, so there are available on UI after a build
      - store_test_results:
          path: ~/junit      # in case you want to have access to reports later
      - store_artifacts:
          path: ~/junit
  push_docker_image:
    <<: *workdirAndImage
    working_directory: ~/workspace/repo
    steps:
      - attach_workspace:
          at: ~/workspace      # this step setups docker deamon, so we can use it later
      - setup_remote_docker
      - restore_cache:
          keys:
          - v2-dependencies-{{ checksum "build.gradle" }}
          # fallback to using the latest cache if no exact match is found
          - v2-dependencies-      # lets build and push docker with version calculated using git tags
      # this gradle task performs 'docker login ...' and 'docker push ...' commands
      - run: gradle pushDockerWithGitVersion# lets define workflow
workflows:
  version: 2  # workflow name
  build_test_and_deploy:    #define jobs within the workflow
    jobs:
      - checkout_code      # because we need to run build after checking out the code, we need to add 'requires' attribute
      - build:
          requires:
            - checkout_code
      # run tests after build
      - test:
          requires:
            - build      # after we are sure the tests are passing, lets push Docker image.
      # we want to do it only for 'master' branch
      - push_docker_image:
          requires:
            - test
          filters:
            branches:
              only: master

Results

Here is how it looks like on Circle CI:

As you can see all the workflow steps have passed and Docker image was pushed to DockerHub.