Building an End-to-End CI/CD Pipeline for Java Applications - Part 2: Using Docker and Kubernetes

The second part of our end-to-end CI/CD series, focusing on extending your pipeline with Kubernetes for container orchestration and deployment.

CD Jenkins Docker Kubernetes
Building an End-to-End CI/CD Pipeline for Java Applications - Part 2: Using Docker and Kubernetes

Table of Contents

Share This Post

Introduction

In this tutorial, we’ll walk through the process of creating a robust end-to-end Continuous Integration and Continuous Deployment (CI/CD) pipeline for a Java application. This is the second part of our E2E CI/CD series, focusing on Jenkins integration with Docker and Kubernetes for CD. We’ll leverage industry-standard tools to automate building, testing, and deploying applications efficiently.

Code Repository

The sample code for this project is available at Spring PetClinic GitHub Repository.

Prerequisites

Before starting this project, make sure you have the following prerequisites:

  1. Jenkins: Ensure Jenkins is installed and running on a server. Jenkins Installation Guide
  2. Kubernetes: Set up a Kubernetes cluster. Kubernetes Installation Guide
  3. Docker: Install Docker on your server. Docker Installation Guide
  4. Java: Ensure your Java application is built and ready for deployment.
  5. Maven: Install Maven or use the Maven version bundled with Jenkins.

Setting up Jenkins

For this we need to use an EC2 instance of type t2.large so that we have enough resources to run the Jenkins server. And also to install other services like Docker, SonarQube and Trivy.

Steps to install Jenkins on an EC2 instance:

  1. SSH into the EC2 instance first install java
sudo apt update -y
sudo apt install openjdk-17-jdk -y or sudo apt install default-jre
java -version
  1. Download and install Jenkins
sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins

Don’t forget to open port 8080 in security group to access Jenkins.

  1. Open the Jenkins URL in the browser and get the initial admin password by following the instructions in the browser.

  2. Install suggested plugins.

  3. Set up the admin user.

  4. Once Jenkins is set up, you can access the Jenkins dashboard.

Two important plugins you need to install:

  1. JDK –> Eclipse Temurin
  2. SonarQube Scanner
  3. Trivy

Next we need to configure it so go to Global Tool Configuration:

  1. JDK –> Add JDK –> Name it as jdk11 –> Install automatically –> Install from Adoptium. –> Click on 11 version.
  2. Maven –> Add Maven –> Name it as maven3 –> Install automatically –> Add the path of the maven on your server.–> Use the version 3.8.6
  3. SonarQube Scanner –> Add SonarQube Scanner –> Name it as sonarqube scanner –> And then add the path of the sonarqube scanner on your server.
  4. Trivy –> Add Trivy –> Name it as trivy –> And then add the path of the trivy on your server.

Next is to create a pipeline job:

  1. Go to Dashboard –> New Item –> Enter the name of the project –> Select Pipeline –> OK
  2. First choose the discard old builds –> And then choose the number of builds to keep. For me it is 2.
  3. Next choose the SCM as Git.
  4. Add the repository URL.
  5. Add the credentials if any.
  6. Next choose the branch as main.
  7. In the pipeline section add the following code:

Setting up Docker

We are installing docker in jenkins server/ec2 instance.

Steps to install Docker on the EC2 instance:

sudo apt-get update
sudo apt-get install docker.io -y
sudo systemctl start docker
sudo docker run hello-world
sudo systemctl enable docker
docker --version
sudo usermod -a -G docker $(whoami)
newgrp docker

Setting up Kubernetes

For this we need at least 2 EC2 instances of type t2.medium. One will be master node and other will be worker node. Follow this link and create it:

https://mrcloudbook.com/ultimate-guide-setting-up-a-kubernetes-cluster-with-master-and-worker-nodes-on-ubuntu-24-04-for-optimal-performance/

Here’s a summary of the steps to set up a Kubernetes cluster:

Setting up the Kubernetes Master Node

  1. Update and Install Required Packages on Master Node

    sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl

  2. Add Kubernetes Repository

    sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

  3. Install Kubernetes Components

    sudo apt update
sudo apt install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

  4. Disable Swap

    sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

  5. Configure Containerd

    cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system

  6. Install Containerd

    sudo apt install -y containerd
sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml
sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml
sudo systemctl restart containerd

  7. Initialize Kubernetes Cluster

    sudo kubeadm init --pod-network-cidr=10.244.0.0/16

  8. Configure kubectl

    mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

  9. Deploy Network Plugin (Flannel)

    kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

Setting up the Kubernetes Worker Node

  1. Update and Install Required Packages on Worker Node

    sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl

  2. Add Kubernetes Repository

    sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

  3. Install Kubernetes Components

    sudo apt update
sudo apt install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

  4. Disable Swap

    sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

  5. Configure Containerd

    cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

sudo sysctl --system

  6. Install Containerd

    sudo apt install -y containerd
sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml
sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml
sudo systemctl restart containerd

  7. Join Worker Node to the Cluster

    Run the join command provided from the master node. It will look something like:

    sudo kubeadm join <master-ip>:6443 --token <token> --discovery-token-ca-cert-hash <hash>

Integrating Kubernetes with Jenkins

Once your Kubernetes cluster is set up, you need to integrate it with Jenkins:

1. Install Required Jenkins Plugins

  1. Go to Jenkins dashboard → Manage Jenkins → Manage Plugins → Available
  2. Search for and install these plugins:
    • Kubernetes
    • Kubernetes CLI
    • Kubernetes Credentials

2. Configure Kubernetes Credentials in Jenkins

  1. Go to Jenkins → Manage Jenkins → Manage Credentials → Jenkins → Global credentials → Add Credentials
  2. Select “Kubernetes Configuration” as the credential type
  3. Enter:
    • ID: kubernetes-config
    • Description: Kubernetes Configuration
    • Kubeconfig: Copy the content from ~/.kube/config on your Kubernetes master node
    • Click “OK” to save

3. Test the Kubernetes Connection

Create a simple Jenkins pipeline job to test the connection:

pipeline {
    agent any
    stages {
        stage('Test Kubernetes Connection') {
            steps {
                withKubeConfig([credentialsId: 'kubernetes-config']) {
                    sh 'kubectl get nodes'
                }
            }
        }
    }
}

Creating Deployment Files for Kubernetes

Now, let’s create the necessary Kubernetes YAML files for deploying our Java application:

1. Create Deployment YAML

Create a file named deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: petclinic
  labels:
    app: petclinic
spec:
  replicas: 2
  selector:
    matchLabels:
      app: petclinic
  template:
    metadata:
      labels:
        app: petclinic
    spec:
      containers:
        - name: petclinic
          image: ${DOCKER_IMAGE}:${IMAGE_TAG}
          ports:
            - containerPort: 8080
          resources:
            limits:
              memory: "512Mi"
              cpu: "500m"
            requests:
              memory: "256Mi"
              cpu: "250m"

2. Create Service YAML

Create a file named service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: petclinic-service
spec:
  selector:
    app: petclinic
  ports:
    - port: 80
      targetPort: 8080
  type: LoadBalancer

3. Create Ingress YAML (Optional)

If you’re using an Ingress controller, create a file named ingress.yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: petclinic-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    - host: petclinic.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: petclinic-service
                port:
                  number: 80

Building the Complete CI/CD Pipeline with Kubernetes Deployment

Now let’s create a complete Jenkins pipeline that builds, tests, and deploys our application to Kubernetes:

pipeline {
    agent any

    tools {
        maven "maven3"
        jdk "jdk11"
    }

    environment {
        DOCKER_IMAGE = "your-dockerhub-username/petclinic"
        IMAGE_TAG = "${BUILD_NUMBER}"
        SONAR_URL = tool name: 'sonarqube-scanner'
    }

    stages {
        stage('Git Checkout') {
            steps {
                git branch: 'main', url: 'https://github.com/spring-projects/spring-petclinic.git'
            }
        }

        stage('Build & Test') {
            steps {
                sh 'mvn clean package'
            }
        }

        stage('SonarQube Analysis') {
            steps {
                withSonarQubeEnv('sonarqube') {
                    sh '''
                    ${SONAR_URL}/bin/sonar-scanner \
                      -Dsonar.projectKey=petclinic \
                      -Dsonar.projectName=petclinic \
                      -Dsonar.sources=src/main \
                      -Dsonar.tests=src/test \
                      -Dsonar.java.binaries=target/classes
                    '''
                }
            }
        }

        stage('Build & Push Docker Image') {
            steps {
                script {
                    withDockerRegistry([credentialsId: 'docker-hub', url: '']) {
                        sh "docker build -t ${DOCKER_IMAGE}:${IMAGE_TAG} ."
                        sh "docker push ${DOCKER_IMAGE}:${IMAGE_TAG}"
                    }
                }
            }
        }

        stage('Security Scan') {
            steps {
                sh "trivy image ${DOCKER_IMAGE}:${IMAGE_TAG}"
            }
        }

        stage('Deploy to Kubernetes') {
            steps {
                script {
                    withKubeConfig([credentialsId: 'kubernetes-config']) {
                        sh "envsubst < deployment.yaml | kubectl apply -f -"
                        sh "kubectl apply -f service.yaml"
                        // Optional: apply ingress configuration
                        // sh "kubectl apply -f ingress.yaml"
                    }
                }
            }
        }

        stage('Verify Deployment') {
            steps {
                withKubeConfig([credentialsId: 'kubernetes-config']) {
                    sh "kubectl rollout status deployment/petclinic"
                    sh "kubectl get services petclinic-service"
                }
            }
        }
    }

    post {
        success {
            echo "Pipeline executed successfully!"
        }
        failure {
            echo "Pipeline execution failed!"
        }
    }
}

Understanding Key Components

Dockerfile

Your application will need a Dockerfile. Here’s a sample for a Spring Boot application:

FROM openjdk:11-jre-slim
WORKDIR /app
COPY target/*.jar app.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "app.jar"]

Kubernetes Resources

The pipeline deploys three Kubernetes resources:

  1. Deployment: Manages the pods running your application
  2. Service: Exposes your application to the network
  3. Ingress: (Optional) Routes external HTTP traffic to your service

Environment Substitution

The pipeline uses environment variables in the Kubernetes deployment file:

  • ${DOCKER_IMAGE}: The Docker image name (from Jenkins environment)
  • ${IMAGE_TAG}: The image tag (build number)

The envsubst command replaces these variables with their actual values before deployment.

Monitoring and Troubleshooting

Checking Deployment Status

kubectl get deployments
kubectl get pods
kubectl get services

Viewing Application Logs

kubectl logs -l app=petclinic

Debugging Deployment Issues

kubectl describe deployment petclinic
kubectl describe pod <pod-name>

Scaling the Application

You can scale your application horizontally by updating the replicas in the deployment.yaml or using kubectl:

kubectl scale deployment petclinic --replicas=5

Conclusion

In this tutorial, we’ve created a complete CI/CD pipeline for Java applications using Jenkins and Kubernetes. This pipeline:

  1. Checks out code from Git
  2. Builds and tests the application
  3. Analyzes code quality with SonarQube
  4. Builds and pushes a Docker image
  5. Scans the image for vulnerabilities
  6. Deploys the application to Kubernetes
  7. Verifies the deployment

In the next part of this series, we’ll explore integrating this pipeline with Azure DevOps for a more comprehensive CI/CD solution.

Table of Contents

Share This Post

Amodh BH | sitemap