Building an End-to-End CI/CD Pipeline for Java Applications - Part 6: Full-Stack Application with Frontend, Backend, and Database

Introduction

In this tutorial, we will learn how to set up a CI/CD pipeline for a full-stack Java application that consists of frontend, backend, and an embedded H2 database. This approach demonstrates how to handle more complex applications that involve multiple components working together.

Code Repository

The sample code for this project is available at Secret Santa Generator GitHub Repository.

Prerequisites

Before starting this project, make sure you have the following prerequisites:

1. Jenkins: Ensure Jenkins is installed and running on a server.
2. SonarQube: Ensure SonarQube is installed and running on a server.
3. Docker: Install Docker on your server.
4. Java: Ensure your Java application is built and ready for deployment.
5. Maven: Install Maven or use the Maven version bundled with Jenkins.
6. OWASP Dependency Check Plugin: Install the OWASP Dependency Check Plugin in Jenkins.
7. Trivy: Install Trivy for vulnerability scanning.

Setting up Jenkins

For this project, we need to use an EC2 instance of type t2.large to ensure sufficient resources for running Jenkins and other services like Docker, SonarQube, and Trivy.

Steps to install Jenkins on an EC2 instance:

1. SSH into the EC2 instance and first install Java:

sudo apt update -y
sudo apt install openjdk-11-jdk -y
java -version

2. Download and install Jenkins:

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins

Don’t forget to open port 8080 in security group to access Jenkins.

3. Open the Jenkins URL in the browser and get the initial admin password by following the instructions in the browser.

4. Install suggested plugins.

5. Set up the admin user.

6. Once Jenkins is set up, you can access the Jenkins dashboard.

Installing Required Jenkins Plugins

You need to install the following plugins:

1. JDK –> Eclipse Temurin
2. SonarQube Scanner
3. OWASP Dependency Check Plugin
4. All Docker plugins
5. Maven

To install these plugins:

1. Go to Manage Jenkins → Manage Plugins → Available
2. Search for each plugin and install it
3. Restart Jenkins after installation

Configuring Jenkins Global Tools

Configure the global tools by going to Manage Jenkins → Global Tool Configuration:

1. JDK: Add JDK → Name it as jdk11 → Install automatically → Install from Adoptium → Click on 11 version.
2. Maven: Add Maven → Name it as maven3 → Install automatically → Use version 3.8.6
3. SonarQube Scanner: Add SonarQube Scanner → Name it as sonarqube scanner → Install automatically

Understanding the Full-Stack Application

The Secret Santa Generator application consists of three main components:

1. Frontend: User interface built with Thymeleaf templates
2. Backend: Spring Boot REST APIs
3. Database: H2 in-memory database

This type of application requires a comprehensive pipeline that can handle building and testing all components together.

Creating the Pipeline Job

1. Go to Dashboard → New Item → Enter the name of the project (e.g., “SecretSanta-CI-CD”) → Select Pipeline → OK
2. First choose the discard old builds → And then choose the number of builds to keep. For this example, we’ll use 2.
3. Next choose the SCM as Git.
4. Add the repository URL: https://github.com/jaiswaladi246/secretsanta-generator.git
5. Choose the branch as master.
6. In the pipeline section, add the following pipeline script:

pipeline {
    agent any

    tools{
        maven 'maven3'
        jdk 'jdk11'
    }

    environment{
        SONARQUBE_HOME = tool name: 'sonarqube'
    }

    stages {
        stage('Checkout Code') {
            steps {
               git url: 'https://github.com/jaiswaladi246/secretsanta-generator.git', branch: 'master'
            }
        }

        stage('Compile Code') {
            steps {
                sh 'mvn clean compile'
            }
        }

        stage('Run Tests') {
            steps {
                sh 'mvn test'
            }
        }

        stage('Sonar Analysis') {
            steps {
                withSonarQubeEnv('sonarqube') {
                    sh ''' $SONARQUBE_HOME/bin/sonar-scanner -Dsonar.projectKey=secretsanta-generator -Dsonar.qualitygate.wait=true -Dsonar.qualitygate.timeout=600000 -Dsonar.url=http://172.31.24.10:9000 -Dsonar.login=admin -Dsonar.java.binaries=.
                    '''
                }
            }
        }

        stage('OWASP Dependency Check') {
            steps {
                dependencyCheck additionalArgs: '-DskipTests --scan ./', odcInstallation: 'DP'
                dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
            }
        }

        stage('Maven Install') {
            steps {
                sh 'mvn clean install -DskipTests=true'
            }
        }

        stage('Build Docker Image') {
            steps {
               script{
                withDockerRegistry([credentialsId: 'docker-hub-credentials', toolName: 'docker']) {
                    sh 'docker build -t jaiswaladi246/secretsanta-generator:latest .'
                    sh 'docker push jaiswaladi246/secretsanta-generator:latest'
                }
               }
            }
        }

        stage('Scan Docker Image with Trivy') {
            steps {
                sh 'trivy image jaiswaladi246/secretsanta-generator:latest'
            }
        }

        stage('Deploy the image to a container') {
            steps {
                script{
                    withDockerRegistry([credentialsId: 'docker-hub-credentials', toolName: 'docker']) {
                        sh 'docker stop secretsanta-generator || true'
                        sh 'docker rm secretsanta-generator || true'
                        sh 'docker run -d --name secretsanta-generator -p 8080:8080 jaiswaladi246/secretsanta-generator:latest'
                    }
                }
            }
        }
    }

    post {
        success {
            echo 'Pipeline completed successfully!'
        }
        failure {
            echo 'Pipeline failed!'
        }
    }
}

Understanding the Pipeline for a Full-Stack Application

Let’s examine the components of our pipeline:

1. Checkout Code:

   • Pulls the latest code from the Git repository with both frontend and backend components.

2. Compile Code:

   • Maven compile builds the Java backend and ensures all dependencies are resolved.

3. Run Tests:

   • Executes unit and integration tests for all components.
   • This includes tests for REST APIs and the database layer.

4. SonarQube Analysis:

   • Performs code quality and security analysis for both frontend and backend code.

5. OWASP Dependency Check:

   • Scans for vulnerabilities in all dependencies.
   • Especially important for a full-stack application with more dependencies.

6. Maven Install:

   • Builds the entire application including the frontend Thymeleaf templates.
   • Creates the JAR file with embedded Tomcat server.

7. Build Docker Image:

   • Creates a Docker image that includes the entire application.
   • Pushes the image to Docker Hub.

8. Scan Docker Image with Trivy:

   • Scans the Docker image for vulnerabilities.

9. Deploy the Docker Container:

   • Stops and removes any existing container.
   • Runs a new container with the latest image.

Special Considerations for Full-Stack Applications

When building a CI/CD pipeline for a full-stack application, there are a few special considerations:

1. Database Management

The H2 database used in this application is an in-memory database that is embedded within the Spring Boot application. This means:

• The database schema is created automatically when the application starts
• The database is initialized with sample data through Hibernate
• No separate database deployment is needed

For applications with persistent databases, you would need to add stages for:

• Database schema migration
• Data seeding
• Backup and restore procedures

2. Frontend and Backend Integration

Since the frontend (Thymeleaf templates) is packaged together with the backend in this Spring Boot application, they are built and deployed together. For applications with separate frontend and backend components, you would need:

• Separate build stages for frontend and backend
• Separate testing stages for each component
• Separate deployment stages

3. End-to-End Testing

For a full-stack application, end-to-end testing is crucial. You might want to add:

• Selenium or Cypress tests to verify the frontend functionality
• API tests to verify the backend functionality
• Integration tests to verify the communication between components

Adding Docker Hub Credentials

Before running the pipeline, add Docker Hub credentials to Jenkins:

1. Go to Manage Jenkins → Manage Credentials → Global
2. Click on Add Credentials
3. Choose Username with password
4. Enter your Docker Hub username and password
5. Add ID as docker-hub-credentials and description as Docker Hub

Setting Up Docker

Install Docker on the Jenkins server/EC2 instance:

sudo apt-get update
sudo apt-get install docker.io -y
sudo systemctl start docker
sudo docker run hello-world
sudo systemctl enable docker
docker --version
sudo usermod -a -G docker $(whoami)
sudo usermod -a -G docker jenkins
newgrp docker
sudo systemctl restart jenkins

Setting up SonarQube

Set up SonarQube on the Jenkins server using Docker:

docker run -d --name sonarqube -p 9000:9000 sonarqube:lts-community

Access SonarQube by navigating to http://<jenkins-server-ip>:9000 in your browser.

The default username and password are both: admin

To create a token for connecting Jenkins with SonarQube:

1. Go to Administration tab
2. Then go to security tab
3. Click on users
4. Click on update token
5. Create a token named jenkins and save it

Now add this token in Jenkins credentials:

1. Go to Manage Jenkins → Manage Credentials → Global
2. Click on Add Credentials
3. Select Secret text as the type
4. Add the token you created in SonarQube
5. Add ID as sonar and description as sonar

Setting up Trivy

Install Trivy on the Jenkins server for container vulnerability scanning:

sudo apt-get install wget apt-transport-https gnupg lsb-release
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy

Understanding the H2 Database Configuration

In the Secret Santa Generator application, H2 is configured in the application.properties file:

# H2 Database Configuration
spring.datasource.url=jdbc:h2:mem:santadb
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=password
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.h2.console.enabled=true
spring.h2.console.path=/h2-console

This configuration:

• Creates an in-memory database named santadb
• Enables the H2 console for debugging at /h2-console
• Uses Hibernate to automatically create tables based on entity classes

The benefit of using H2 for this pipeline is that it requires no external database setup, making the deployment process simpler.

Verifying the Deployment

After the pipeline completes successfully:

1. Access the application at http://<jenkins-server-ip>:8080
2. You should see the Secret Santa Generator application’s landing page
3. To access the H2 console for database inspection, navigate to http://<jenkins-server-ip>:8080/h2-console

Testing the Application

Once deployed, test the application by:

1. Creating a new Secret Santa group
2. Adding participants to the group
3. Generating Secret Santa assignments
4. Verifying that the assignments are correctly stored in the H2 database

Challenges and Troubleshooting

When working with a full-stack application in CI/CD, you might encounter these common issues:

1. Memory Issues: Full-stack applications require more memory, especially when running tests. Ensure your Jenkins server has sufficient memory.

2. Port Conflicts: If you’re deploying multiple applications, you might encounter port conflicts. Use different ports or Docker networking to resolve this.

3. Frontend Build Failures: JavaScript and CSS builds can be sensitive to Node.js versions and dependencies. Ensure you’re using compatible versions in your build environment.

4. Database Integration: Even with H2, you might face issues with database schema migrations or test data. Use Spring Boot’s testing features to manage test databases.

Conclusion

In this tutorial, we’ve created a comprehensive CI/CD pipeline for a full-stack Java application with frontend, backend, and H2 database components. This approach:

1. Ensures all application components are built, tested, and deployed together
2. Maintains consistency between frontend and backend changes
3. Simplifies the deployment process with containerization
4. Includes security scanning at multiple levels

By following this guide, you now have the knowledge to implement CI/CD for more complex Java applications that include multiple tiers and components.

This concludes our six-part series on building end-to-end CI/CD pipelines for Java applications. Throughout this series, we’ve covered various aspects of CI/CD implementation, from basic pipelines to complex deployments involving multiple components and platforms. We hope this knowledge helps you implement robust CI/CD practices in your own projects.