Building an End-to-End CI/CD Pipeline for Node.js Applications - Part 2: Using CircleCI

Introduction

In this tutorial, we will learn how to set up a Continuous Integration and Continuous Deployment (CI/CD) pipeline for a Node.js application using CircleCI. CircleCI offers a cloud-based CI/CD platform that automates the build, test, and deployment processes. This approach provides an alternative to Jenkins (which we covered in Part 1), with its own set of advantages, including easier setup and maintenance.

Code Repository

The sample code for this project is available at Clone Website GitHub Repository.

Prerequisites

Before starting this project, make sure you have the following prerequisites:

1. GitHub Account: Your Node.js application code should be hosted on GitHub.
2. CircleCI Account: Create a free account at CircleCI by signing up with your GitHub account.
3. Docker Hub Account: To store and distribute Docker images.
4. EC2 Instance: AWS EC2 instance (or equivalent) for deployment.

Setting Up CircleCI

1. Sign Up for CircleCI:

   • Go to CircleCI and sign up using your GitHub account.
   • Authorize CircleCI to access your GitHub repositories.

2. Add Your Project:

   • Once logged in, go to the “Projects” tab in the CircleCI dashboard.
   • Find your repository in the list and click on “Set Up Project”.
   • You’ll be guided to add a configuration file to your repository.

Creating the CircleCI Configuration

The heart of CircleCI is the configuration file that defines your pipeline. Create a file named .circleci/config.yml in the root of your repository:

version: 2.1
jobs:
  run_tests:
    docker:
      - image: cimg/node:18.16.0
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: npm install --save
      - run:
          name: Run tests
          command: npm test

  build_docker:
    docker:
      - image: cimg/node:18.16.0
    steps:
      - checkout
      - setup_remote_docker:
          docker_layer_caching: false
      - run:
          name: Build Docker Image
          command: |
            export TAG=0.2.<<pipeline.number>>
            docker build -t amodh-2002/clone_website:$TAG .
            docker tag amodh-2002/clone_website:$TAG amodh-2002/clone_website:latest            
      - run:
          name: Push Docker Image
          command: |
            echo $DOCKER_PASSWORD | docker login -u $DOCKER_USERNAME --password-stdin
            export TAG=0.2.<<pipeline.number>>
            docker push amodh-2002/clone_website:$TAG
            docker push amodh-2002/clone_website:latest            

workflows:
  version: 2
  build_and_deploy:
    jobs:
      - run_tests
      - build_docker:
          requires:
            - run_tests

Understanding the CircleCI Configuration

Let’s break down this configuration:

Jobs

1. run_tests:

   • Uses a Node.js Docker image
   • Checks out the code
   • Installs dependencies
   • Runs tests

2. build_docker:

   • Uses the same Node.js Docker image
   • Sets up remote Docker execution environment
   • Builds a Docker image with a tag that includes the pipeline number
   • Pushes the image to Docker Hub

Workflow

The build_and_deploy workflow defines the sequence of jobs:

1. First runs the tests
2. Then builds and pushes the Docker image, but only if the tests pass

Setting Up Environment Variables in CircleCI

To push images to Docker Hub, you need to securely store your Docker Hub credentials in CircleCI:

1. Go to your project in CircleCI
2. Click on “Project Settings” (found in the top-right of your project’s page)
3. Select “Environment Variables” from the sidebar
4. Add the following variables:
   • DOCKER_USERNAME: Your Docker Hub username
   • DOCKER_PASSWORD: Your Docker Hub password

Deploying to EC2

After your image is built and pushed to Docker Hub, you’ll need to deploy it to your EC2 instance. You can do this manually or extend the CircleCI configuration to handle deployment.

Manual Deployment

SSH into your EC2 instance and run:

docker pull amodh-2002/clone_website:latest
docker stop nodejs-app || true
docker rm nodejs-app || true
docker run -d --name nodejs-app -p 80:80 amodh-2002/clone_website:latest

Automated Deployment with CircleCI

For a fully automated deployment, you can extend your CircleCI configuration to include a deployment job:

deploy_to_ec2:
  docker:
    - image: cimg/base:stable
  steps:
    - add_ssh_keys:
        fingerprints:
          - "xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx"
    - run:
        name: Deploy to EC2
        command: |
          ssh -o StrictHostKeyChecking=no $SSH_USER@$SSH_HOST 'docker pull amodh-2002/clone_website:latest && \
          docker stop nodejs-app || true && \
          docker rm nodejs-app || true && \
          docker run -d --name nodejs-app -p 80:80 amodh-2002/clone_website:latest'          

To use this:

1. Add your EC2 SSH key to CircleCI in Project Settings → SSH Keys
2. Add the following environment variables:
   • SSH_USER: The username for SSH access (e.g., ubuntu or ec2-user)
   • SSH_HOST: Your EC2 instance’s public IP or DNS
3. Add this job to your workflow:

workflows:
  version: 2
  build_and_deploy:
    jobs:
      - run_tests
      - build_docker:
          requires:
            - run_tests
      - deploy_to_ec2:
          requires:
            - build_docker

Testing Your CircleCI Pipeline

After setting up your configuration file and environment variables:

1. Commit and push your changes to GitHub
2. Go to the CircleCI dashboard
3. Select your project
4. You should see your workflow running

Advantages of CircleCI Over Jenkins

CircleCI offers several advantages compared to Jenkins:

1. Zero Maintenance: CircleCI is a cloud-based service, so you don’t need to maintain servers.

2. Faster Setup: Getting started with CircleCI is simpler and requires less configuration.

3. Built-in Docker Support: CircleCI has native support for Docker without requiring additional plugins.

4. Parallel Execution: Free accounts get multiple concurrent builds, which can speed up your pipeline.

5. Easy Configuration: YAML-based configuration that lives with your code.

Optimizing Your CircleCI Pipeline

Here are some best practices to optimize your CircleCI pipeline:

1. Caching Dependencies

Add caching to speed up builds by reusing npm dependencies:

steps:
  - checkout
  - restore_cache:
      keys:
        - v1-dependencies-{{ checksum "package.json" }}
        - v1-dependencies-
  - run: npm install
  - save_cache:
      paths:
        - node_modules
      key: v1-dependencies-{{ checksum "package.json" }}

2. Reusing Configuration

Use CircleCI orbs to simplify your config and reuse code:

version: 2.1

orbs:
  node: circleci/[email protected]
  docker: circleci/[email protected]

jobs:
  build_and_test:
    executor: node/default
    steps:
      - checkout
      - node/install-packages
      - run:
          name: Run tests
          command: npm test

3. Workflow Optimization

Split your workflow into parallel jobs where possible to speed up execution:

workflows:
  version: 2
  build_and_deploy:
    jobs:
      - lint
      - test_unit
      - test_integration
      - build_docker:
          requires:
            - lint
            - test_unit
            - test_integration

Security Considerations

When implementing CI/CD pipelines with CircleCI, consider these security best practices:

1. Environment Variables: Always use environment variables for sensitive information.

2. Contexts: Use CircleCI contexts to share environment variables across projects.

3. Branch Protection: Set up branch protection rules in GitHub to require passing CI checks before merging.

4. Vulnerability Scanning: Add container scanning to your pipeline:

security_scan:
  docker:
    - image: aquasec/trivy
  steps:
    - run:
        name: Scan Docker image
        command: trivy image amodh-2002/clone_website:latest

Verifying Your Deployment

After deploying your application, verify it’s running correctly:

1. Access your EC2 instance’s public IP in a web browser
2. The Node.js application should be accessible
3. Check the logs to ensure everything is working as expected:

docker logs nodejs-app

Troubleshooting Common Issues

1. Docker Hub Authentication Failure

If you encounter authentication issues with Docker Hub:

• Verify your environment variables are set correctly
• Ensure your Docker Hub credentials are valid
• Check if you have reached Docker Hub’s rate limits

2. SSH Connection Problems

If the automated deployment can’t connect to your EC2 instance:

• Verify your SSH key is added correctly to CircleCI
• Check that your EC2 security group allows SSH access
• Ensure the SSH_USER and SSH_HOST environment variables are correct

3. Tests Failing

If your tests are failing in CircleCI but pass locally:

• Check for environment-specific dependencies
• Verify that all required environment variables are set
• Check for differences between your local Node.js version and CircleCI’s

Conclusion

In this tutorial, we’ve set up a comprehensive CI/CD pipeline for a Node.js application using CircleCI. We’ve covered:

1. Setting up CircleCI with your GitHub repository
2. Creating a workflow to test, build, and deploy your application
3. Securing sensitive information with environment variables
4. Implementing best practices for efficient pipelines

CircleCI provides a powerful, easy-to-use platform for automating your development workflows. By implementing the practices covered in this tutorial, you can ensure your Node.js applications are built, tested, and deployed consistently and efficiently.

In the next part of this series, we’ll explore how to enhance our Jenkins pipeline from Part 1 with Trivy for container security scanning, taking our CI/CD process to the next level of security compliance.