Building an End-to-End CI/CD Pipeline for Java Applications - Part 4: Using Docker and Webhooks

Introduction

In this project, we will create an end-to-end Continuous Integration and Continuous Deployment (CI/CD) pipeline for a Java application. This is the fourth and final part of our series, focusing on Docker for containerization and GitHub webhooks for automated pipeline triggering.

Code Repository

The sample code for this project is available at Spring Boot Web Application GitHub Repository.

Prerequisites

Before starting this project, make sure you have the following prerequisites:

1. Docker: Install Docker on your server.
2. Java: Ensure your Java application is built and ready for deployment.
3. Maven: Install Maven or use the Maven version bundled with Jenkins.
4. SonarQube: Install SonarQube on your server.
5. Jenkins: Install Jenkins on your server.
6. Trivy: Install Trivy on your server.
7. EC2 Instance: Create an EC2 instance of type t2.medium.

Setting Up Jenkins

For this we need to use an EC2 instance of type t2.large so that we have enough resources to run the Jenkins server and other services like Docker, SonarQube, and Trivy.

Steps to install Jenkins on an EC2 instance:

1. SSH into the EC2 instance and first install Java:

sudo apt update -y
sudo apt install openjdk-17-jdk -y
java -version

2. Download and install Jenkins:

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins

Don’t forget to open port 8080 in the security group to access Jenkins.

3. Open the Jenkins URL in the browser and get the initial admin password by following the instructions in the browser.
4. Install suggested plugins.
5. Set up the admin user.
6. Once Jenkins is set up, you can access the Jenkins dashboard.

Installing Required Jenkins Plugins

You need to install the following plugins:

1. JDK –> Eclipse Temurin
2. Loading plugin extensions
3. OWASP Dependency Check
4. Authentication Tokens API
5. SonarQube Scanner
6. Trivy
7. Docker
8. Docker Pipeline
9. Docker API
10. Docker-build-step
11. Docker commons
12. Javadoc
13. Maven Integration
14. JSch dependency

To install these plugins:

1. Go to Manage Jenkins → Manage Plugins → Available
2. Search for each plugin and install it
3. Restart Jenkins after installation

Configuring Jenkins Global Tools

Configure the global tools by going to Manage Jenkins → Global Tool Configuration:

1. JDK: Add JDK → Name it as jdk11 → Install automatically → Install from Adoptium → Click on 11 version
2. Maven: Add Maven → Name it as maven3 → Install automatically → Use version 3.8.6
3. SonarQube Scanner: Add SonarQube Scanner → Name it as sonarqube-scanner → Install automatically
4. Trivy: Add Trivy → Name it as trivy → Install automatically
5. Dependency Check: Add Dependency Check → Name it as DP → Install automatically
6. Docker: Add Docker → Name it as docker → Install automatically

Setting Up Docker

Install Docker on the Jenkins server/EC2 instance:

sudo apt-get update
sudo apt-get install docker.io -y
sudo systemctl start docker
sudo docker run hello-world
sudo systemctl enable docker
docker --version
sudo usermod -a -G docker $(whoami)
sudo usermod -a -G docker jenkins
newgrp docker
sudo systemctl restart jenkins

Setting Up SonarQube

Set up SonarQube on the Jenkins server using Docker:

docker run -d --name sonarqube -p 9000:9000 sonarqube:lts-community

Access SonarQube by navigating to http://<jenkins-server-ip>:9000 in your browser.

The default username and password are both: admin

To create a token for connecting Jenkins with SonarQube:

1. Go to Administration tab
2. Then go to security tab
3. Click on users
4. Click on update token
5. Create a token named jenkins and save it

Now add this token in Jenkins credentials:

1. Go to Manage Jenkins → Manage Credentials → Global
2. Click on Add Credentials
3. Select Secret text as the type
4. Add the token you created in SonarQube
5. Add ID as sonar and description as sonar

Setting Up Trivy

Install Trivy on the Jenkins server:

sudo apt-get install wget apt-transport-https gnupg lsb-release
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy

Creating a Jenkins Pipeline

1. Go to Dashboard → New Item → Enter the name of the project → Select Pipeline → OK
2. First choose the discard old builds → And then choose the number of builds to keep. For this example, we’ll use 2.
3. Next choose the SCM as Git
4. Add the repository URL: https://github.com/jaiswaladi246/SpringBoot-WebApplication.git
5. Choose the branch as main
6. In the pipeline section, we’ll add our pipeline script

Building the CI/CD Pipeline

Let’s create a comprehensive pipeline that includes all the necessary stages:

Stage 1: Git Checkout

pipeline {
    agent any
    tools {
        maven 'maven3'
        jdk 'jdk11'
    }
    stages {
        stage('git checkout code') {
            steps {
               git branch: 'main', changelog: false, poll: false, url: 'https://github.com/jaiswaladi246/SpringBoot-WebApplication.git'
            }
        }
    }
}

Stage 2: Compile Code

stages {
    stage('compile code') {
        steps {
            sh 'mvn compile'
        }
    }
}

Stage 3: Test Code

stages {
    stage('test code') {
        steps {
            sh 'mvn test'
        }
    }
}

Stage 4: SonarQube Analysis

For this step, we’ll use the pipeline syntax helper to generate the proper syntax:

stages {
    stage('SonarQube Analysis') {
        environment {
            scannerHome = tool 'sonarqube-scanner'
        }
        steps {
            withSonarQubeEnv('sonarqube') {
                sh '''
                ${scannerHome}/bin/sonar-scanner \
                -Dsonar.projectKey=spring-petclinic \
                -Dsonar.projectName=spring-petclinic \
                -Dsonar.projectVersion=1.0 \
                -Dsonar.sources=src/ \
                -Dsonar.java.binaries=target/classes/ \
                -Dsonar.exclusions=src/test/java/****/*.java \
                -Dsonar.qualitygate.wait=true
                '''
            }
        }
    }
}

Stage 5: OWASP Dependency Check

stages {
    stage('OWASP Dependency Check') {
        steps {
            dependencyCheck additionalArguments: '--disableYarnAudit --scan ./', odcInstallation: 'DP'
            dependencyCheckPublisher pattern: '**/dependency-check-report.xml', healthy: '0', unhealthy: '1', unstableTotalHigh: '0'
        }
    }
}

Stage 6: Build Maven Project

stages {
    stage('build maven project') {
        steps {
            sh 'mvn clean install'
        }
    }
}

Stage 7: Build Docker Image

For this step, we need to add Docker Hub credentials in Jenkins:

1. Go to Manage Jenkins → Manage Credentials → Global
2. Click on Add Credentials
3. Choose Username with password
4. Enter your Docker Hub username and password
5. Add ID as docker-hub and description as docker-hub

Now we can add the Docker build stage:

stages {
    stage('build docker image') {
        steps {
            script {
                withCredentials([string(credentialsId: 'docker-hub', variable: 'dockerhubpwd')]) {
                    sh 'docker build -t yourdockerhubusername/spring-boot-app:latest .'
                }
            }
        }
    }
}

Stage 8: Trivy Scan

stages {
    stage('Trivy Scan') {
        steps {
            sh 'trivy image yourdockerhubusername/spring-boot-app:latest'
            sh 'trivy image yourdockerhubusername/spring-boot-app:latest > trivy-report.txt'
        }
    }
}

Stage 9: Push Docker Image

stages {
    stage('push docker image') {
        steps {
            script {
                withCredentials([string(credentialsId: 'docker-hub', variable: 'dockerhubpwd')]) {
                    sh 'docker login -u yourdockerhubusername -p ${dockerhubpwd}'
                    sh 'docker push yourdockerhubusername/spring-boot-app:latest'
                }
            }
        }
    }
}

Stage 10: Run Docker Container

stages {
    stage('Run Docker Container') {
        steps {
            script {
                sh 'docker stop spring-boot-app || true'
                sh 'docker rm spring-boot-app || true'
                sh 'docker run -d -p 8080:8080 --name spring-boot-app yourdockerhubusername/spring-boot-app:latest'
            }
        }
    }
}

Complete Pipeline Script

Here’s the complete pipeline script:

pipeline {
    agent any

    tools {
        maven 'maven3'
        jdk 'jdk11'
    }

    stages {
        stage('Git Checkout') {
            steps {
                git branch: 'main',
                    url: 'https://github.com/jaiswaladi246/SpringBoot-WebApplication.git'
            }
        }

        stage('Compile Code') {
            steps {
                sh 'mvn compile'
            }
        }

        stage('Test Code') {
            steps {
                sh 'mvn test'
            }
        }

        stage('SonarQube Analysis') {
            environment {
                scannerHome = tool 'sonarqube-scanner'
            }
            steps {
                withSonarQubeEnv('sonarqube') {
                    sh '''
                    ${scannerHome}/bin/sonar-scanner \
                    -Dsonar.projectKey=spring-petclinic \
                    -Dsonar.projectName=spring-petclinic \
                    -Dsonar.projectVersion=1.0 \
                    -Dsonar.sources=src/ \
                    -Dsonar.java.binaries=target/classes/ \
                    -Dsonar.exclusions=src/test/java/****/*.java \
                    -Dsonar.qualitygate.wait=true
                    '''
                }
            }
        }

        stage('OWASP Dependency Check') {
            steps {
                dependencyCheck additionalArguments: '--disableYarnAudit --scan ./', odcInstallation: 'DP'
                dependencyCheckPublisher pattern: '**/dependency-check-report.xml', healthy: '0', unhealthy: '1', unstableTotalHigh: '0'
            }
        }

        stage('Build Maven Project') {
            steps {
                sh 'mvn clean install'
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    withCredentials([string(credentialsId: 'docker-hub', variable: 'dockerhubpwd')]) {
                        sh 'docker build -t yourdockerhubusername/spring-boot-app:latest .'
                    }
                }
            }
        }

        stage('Trivy Scan') {
            steps {
                sh 'trivy image yourdockerhubusername/spring-boot-app:latest'
                sh 'trivy image yourdockerhubusername/spring-boot-app:latest > trivy-report.txt'
            }
        }

        stage('Push Docker Image') {
            steps {
                script {
                    withCredentials([string(credentialsId: 'docker-hub', variable: 'dockerhubpwd')]) {
                        sh 'docker login -u yourdockerhubusername -p ${dockerhubpwd}'
                        sh 'docker push yourdockerhubusername/spring-boot-app:latest'
                    }
                }
            }
        }

        stage('Run Docker Container') {
            steps {
                script {
                    sh 'docker stop spring-boot-app || true'
                    sh 'docker rm spring-boot-app || true'
                    sh 'docker run -d -p 8080:8080 --name spring-boot-app yourdockerhubusername/spring-boot-app:latest'
                }
            }
        }
    }

    post {
        success {
            echo 'Pipeline completed successfully!'
        }
        failure {
            echo 'Pipeline failed!'
        }
    }
}

Creating a Dockerfile

To build a Docker image, you need a Dockerfile in your project root. Here’s a simple Dockerfile for a Spring Boot application:

FROM openjdk:11-jre-slim

WORKDIR /app

COPY target/*.jar app.jar

EXPOSE 8080

CMD ["java", "-jar", "app.jar"]

Setting Up GitHub Webhooks

To automatically trigger the Jenkins pipeline when code is pushed to GitHub, we need to set up a webhook:

1. Go to your GitHub repository
2. Click on Settings
3. Click on Webhooks
4. Click on Add webhook
5. Add the payload URL: http://<jenkins-ip-address>:8080/github-webhook/
6. Set Content type to application/json
7. For “Which events would you like to trigger this webhook?”, select “Just the push event”
8. Click on Add webhook

Now in Jenkins:

1. Go to your pipeline job
2. Click on Configure
3. In the “Build Triggers” section, check “GitHub hook trigger for GITScm polling”
4. Save the configuration

This setup will trigger the pipeline automatically whenever code is pushed to the main branch of your GitHub repository.

Understanding the Pipeline Execution

Let’s examine what happens in each stage of our pipeline:

1. Git Checkout: Clones the code from the GitHub repository.
2. Compile Code: Compiles the Java code to check for syntax errors.
3. Test Code: Runs unit tests to ensure functionality.
4. SonarQube Analysis: Analyzes code quality and security issues.
5. OWASP Dependency Check: Scans dependencies for vulnerabilities.
6. Build Maven Project: Creates a JAR file from the code.
7. Build Docker Image: Creates a Docker image with the application.
8. Trivy Scan: Scans the Docker image for vulnerabilities.
9. Push Docker Image: Pushes the image to Docker Hub for storage and sharing.
10. Run Docker Container: Deploys the application by running a Docker container.

Pipeline Visualization and Monitoring

Jenkins provides a visualization of your pipeline execution. You can:

1. View the pipeline stages in the Stage View
2. See detailed logs for each stage
3. Monitor build times and success/failure rates
4. Download artifacts created during the build

Debugging Common Issues

Docker Permission Issues

If you encounter permission errors when running Docker commands, make sure:

• The Jenkins user is added to the Docker group
• Jenkins has been restarted
• The container name isn’t already in use

SonarQube Analysis Failures

If SonarQube analysis fails:

• Check that the SonarQube server is running
• Verify that the token is correct
• Ensure the project key is unique

Docker Hub Authentication Issues

If pushing to Docker Hub fails:

• Verify your Docker Hub credentials in Jenkins
• Check that the username in the pipeline matches your Docker Hub account
• Ensure you have permissions to push to the repository

Security Considerations

Our pipeline includes several security checks:

1. SonarQube Analysis: Identifies code vulnerabilities and quality issues
2. OWASP Dependency Check: Finds vulnerabilities in dependencies
3. Trivy Scan: Detects vulnerabilities in the Docker image

These checks help ensure your application is secure before deployment.

Scaling the Pipeline

For larger projects, consider these enhancements:

1. Parallel Execution: Run tests and security scans in parallel
2. Deployment Environments: Add stages for deploying to different environments (dev, staging, production)
3. Approval Gates: Require manual approval before deploying to production
4. Notifications: Send email or Slack notifications on build results

Fast Deployment Alternative

For a quicker deployment of a Java application using Docker, follow these simplified steps:

1. Clone the repository:

   git clone https://github.com/jaiswaladi246/Ekart
   

2. Build the application:

   cd Ekart
   mvn clean install --DskipTests
   

3. Build and run Docker container:

   cd docker
   docker build -t ekart .
   docker run -d -p 8080:8080 ekart
   

This approach allows you to deploy a Java application within minutes, skipping the comprehensive CI/CD pipeline when quick deployment is needed.

Conclusion

In this tutorial, we’ve created a comprehensive CI/CD pipeline for Java applications that:

1. Automatically triggers on code changes
2. Compiles and tests the code
3. Analyzes code quality and security
4. Builds a Docker image
5. Scans the image for vulnerabilities
6. Deploys the application in a container

This pipeline ensures consistent, automated, and secure deployments, embodying DevOps best practices. By implementing this pipeline, you’ll save time, reduce errors, and deliver higher quality applications.

Throughout this four-part series, we’ve explored various approaches to CI/CD for Java applications, from basic Jenkins pipelines to advanced configurations with Kubernetes, Azure DevOps, and now Docker with webhooks. You can choose the approach that best fits your project’s needs and scale it as your requirements evolve.