Ansible: The Ultimate Guide - Part 2 (Advanced Topics)

Ansible Modules

Modules are the building blocks of Ansible tasks, performing actions on managed nodes.

Types of Ansible Modules

1. System Modules: Manage OS-level tasks (e.g., apt, yum, user).
2. Cloud Modules: Manage cloud resources (e.g., AWS EC2 with ec2_instance).
3. Network Modules: Configure networking devices (e.g., ios_config, net_ping).

Using a Module

Run an ad-hoc command with the ansible CLI:

ansible all -i inventory.yml -m ping

In this command, all is the group name and ping is the module name.

Ansible is agentless and includes most modules out of the box. However, you can download additional modules using Ansible Collections:

ansible-galaxy collection install community.general

Roles and Tasks

Roles help you organize playbooks into reusable components.

What is a Role?

A role contains structured directories for tasks, handlers, variables, and templates:

roles/
  webserver/
    tasks/
    handlers/
    vars/
    templates/

Creating a Role

Use the ansible-galaxy command:

ansible-galaxy init my_role

This command generates a directory structure like this:

my_role/
├── README.md
├── tasks
│   └── main.yml
├── handlers
│   └── main.yml
├── meta
│   └── main.yml
├── defaults
│   └── main.yml
├── vars
│   └── main.yml
├── files
└── templates

Directory Structure Overview

• README.md: A markdown file for documentation about the role.
• tasks/main.yml: The main file where you define the tasks for the role.
• handlers/main.yml: Handlers that can be triggered by tasks in the role.
• meta/main.yml: Metadata for the role, such as dependencies.
• defaults/main.yml: Default variables for the role.
• vars/main.yml: Variables specific to the role.
• files/: Directory for files that can be used in your tasks.
• templates/: Directory for Jinja2 templates that can be used in your tasks.

Note: This doesn’t create inventory file and ansible.cfg file out of the box - we have to create them manually.

What is a Task?

A task is a single action in a playbook:

tasks:
  - name: Create a directory
    file:
      path: /tmp/example
      state: directory

Roles can be shared across projects. Store them in a version control system or share them via Ansible Galaxy.

Handlers and Notifications

Handlers are special tasks triggered by notifications.

What are Handlers?

Handlers perform tasks like restarting services when changes occur. They run only when notified.

Example

tasks:
  - name: Update configuration
    template:
      src: config.j2
      dest: /etc/myapp/config
    notify: Restart myapp

handlers:
  - name: Restart myapp
    service:
      name: myapp
      state: restarted

What the above code does is:

• The tasks section defines a task to update the configuration file.
• The notify directive in the task triggers the Restart myapp handler.
• The handlers section defines a handler that restarts the myapp service.

Multiple tasks can notify the same handler. The handler will run only once, even if notified multiple times.

Variables and Facts

Variables and facts allow dynamic configurations in playbooks.

Using Variables

Define variables in playbooks:

vars:
  app_name: myapp
  app_port: 8080

tasks:
  - name: Print app details
    debug:
      msg: "The app {{ app_name }} is running on port {{ app_port }}"

Here app_name and app_port are variables and msg is the message that will be displayed. It is using Jinja2 templating to display the variables.

Using Facts

Facts are gathered from managed nodes automatically:

tasks:
  - name: Display OS info
    debug:
      msg: "The OS is {{ ansible_os_family }}"

You can create custom facts as JSON or key-value pairs in /etc/ansible/facts.d/ on the managed node.

Ansible Vault

Ansible Vault is a feature of Ansible that allows you to encrypt sensitive data such as passwords, API keys, and other confidential information. This enables you to manage sensitive data securely within your playbooks and roles without exposing it in plaintext.

Encrypting Files

Encrypt a file:

ansible-vault encrypt secrets.yml

• What Happens:

  • This command encrypts the contents of the file secrets.yml.
  • When you run this command, you will be prompted to enter a password. This password will be used to encrypt the file and will also be required to decrypt it later.
  • The original secrets.yml file will be replaced with an encrypted version.
  • The encrypted content will look something like this when viewed:

    $ANSIBLE_VAULT;1.1;AES256
    613030636165626663303839373863306532383934393835343831303235633932373768313936663864393936663336312e343530310a646235616336313066383537373933383965636133666563316131366361383033323730663237313138306433333265653835663934646166373138396631623431383931386438643261373865623939316136356230323335663963613661646132386537393165643963323133303961626265373231306530656164353036303234666431626639306132653132306465363035386134626535313764396331393965
    

Using Vault in Playbooks

Reference encrypted variables:

vars_files:
  - secrets.yml

• This allows you to securely utilize the sensitive data contained in secrets.yml without exposing it directly in your playbook.

• What Happens:

  • Ansible reads the secrets.yml, decrypts it using the provided vault password, and loads the variables into the playbook context.
  • You can reference these variables anywhere in your playbook (e.g., in tasks, templates).

You can use different passwords for different vault files with --vault-id:

ansible-playbook playbook.yml --vault-id dev@prompt --vault-id prod@prompt

• What Happens:

  • The --vault-id option allows you to specify different vault IDs and their corresponding passwords for different encrypted files.
  • In this example:
    • dev@prompt: This instructs Ansible to ask for a password to decrypt the files associated with the dev vault ID.
    • prod@prompt: This instructs Ansible to ask for a different password for files associated with the prod vault ID.

Ansible Collections

What Are Ansible Collections?

Ansible Collections are packages that bundle various Ansible components together, such as:

• Modules: These are reusable code snippets that perform specific tasks (e.g., installing software).
• Plugins: These extend Ansible’s functionality (e.g., connection plugins, lookup plugins).
• Roles: These are a way to organize playbooks and tasks around specific functionalities.

This bundling makes it easier to manage and share Ansible functionality across different projects and environments.

Installing a Collection

To get started using a collection, you often need to install it, especially if it’s not part of the default Ansible distribution.

Example Command:

ansible-galaxy collection install community.docker

• What Happens:
  • This command tells Ansible to download and install the community.docker collection from Ansible Galaxy (the community repository).
  • After running this command, you will have access to all the modules, plugins, and roles contained within the community.docker collection.

Using a Collection

Once a collection is installed, you can reference its modules and other components in your playbooks.

Example Playbook Reference:

- name: Use Docker module
  community.docker.docker_container:
    name: my_container
    image: nginx

• What Happens:
  • In the playbook, you’re using the docker_container module from the community.docker collection.
  • Here’s what each part does:
    • - name: Use Docker module: This is a description of what this task does.
    • community.docker.docker_container: This fully qualified name tells Ansible to look for the docker_container module in the community.docker collection.
    • The name: my_container and image: nginx lines specify the parameters for creating a Docker container named my_container using the nginx image.

Creating Your Own Collections

You can create your own collections to package up your Ansible roles and modules.

Creating a Collection:

ansible-galaxy collection init my_namespace.my_collection

• What Happens:
  • This command creates a new directory structure for your collection named my_collection under the namespace my_namespace.
  • The structure allows you to organize your files properly.

Basic Structure Created:

my_namespace/
└── my_collection/
    ├── README.md
    ├── galaxy.yml
    ├── plugins/
    │   ├── modules/
    │   └── ...
    └── roles/

Advanced Ansible Concepts

Ansible Loops and Conditionals

Loops and conditionals in Ansible allow you to execute tasks iteratively or based on specific conditions.

Loops

• Ansible provides various ways to repeat tasks over multiple items using constructs like with_items, loop, or other specialized looping mechanisms.

Example: Using loop for repetitive tasks

- name: Install multiple packages
  apt:
    name: "{{ item }}"
    state: present
  loop:
    - apache2
    - mysql-server
    - php

• Explanation: This task installs three packages (apache2, mysql-server, and php) by looping through the items in the list.

Advanced Loop Constructs

• Use with_items for older versions or loop for modern playbooks.
• Combine with dictionaries for complex loops:

  - name: Create multiple users
    user:
      name: "{{ item.name }}"
      uid: "{{ item.uid }}"
    loop:
      - { name: "user1", uid: 1001 }
      - { name: "user2", uid: 1002 }
  

Conditionals

Conditionals use the when keyword to control task execution based on certain conditions.

Example: Task with a condition

- name: Install Apache only on Debian-based systems
  apt:
    name: apache2
    state: present
  when: ansible_os_family == "Debian"

• Explanation: This task installs Apache only if the target system belongs to the Debian family.

Combining Loops and Conditionals

- name: Install packages conditionally
  apt:
    name: "{{ item }}"
    state: present
  loop:
    - apache2
    - nginx
  when: ansible_os_family == "Debian"

If the condition evaluates to false, the task is skipped.

Ansible Templates (Jinja2)

Templates in Ansible, written in Jinja2, allow dynamic content generation based on variables, loops, and conditionals.

Creating a Template

1. Save a template file with a .j2 extension (e.g., nginx.conf.j2).
2. Use placeholders for variables inside the template.

Example: A basic template nginx.conf.j2:

server {
  listen {{ port }};
  server_name {{ server_name }};
}

Playbook to use the template

- name: Deploy Nginx configuration
  template:
    src: nginx.conf.j2
    dest: /etc/nginx/sites-available/default
  vars:
    port: 80
    server_name: example.com

• Explanation: This task dynamically generates an Nginx configuration file with values from variables and copies it to the target host.

Templates can include Jinja2 constructs like loops and conditionals:

{% for user in users %}
user {{ user }};
{% endfor %}

Error Handling in Ansible

Ansible offers mechanisms to gracefully handle errors during playbook execution.

Common Error Handling Techniques

1. Ignore Errors: Use ignore_errors: yes to allow playbook execution to continue even if a task fails.

   - name: Attempt to restart a non-existent service
     service:
       name: nonexistent_service
       state: restarted
     ignore_errors: yes
   

2. failed_when: Define custom failure conditions.

   - name: Fail the task if a condition is met
     command: /bin/false
     failed_when: result.rc != 0
   

3. Retries and Delays: Use retries and delay with until to retry tasks.

   - name: Wait for service to start
     shell: systemctl status apache2
     register: result
     retries: 5
     delay: 10
     until: result.rc == 0
   

You can notify a handler if a task fails and ignore the error to proceed.

Ansible Dynamic Inventory

Dynamic inventory allows Ansible to interact with dynamic environments like AWS, GCP, or Kubernetes by fetching inventory details at runtime.

Setting Up Dynamic Inventory

1. Install the required inventory script or plugin (e.g., AWS).
2. Configure your ansible.cfg to use it.

Example: Using AWS Dynamic Inventory

# ansible.cfg
[defaults]
inventory = ./aws_ec2.yaml

aws_ec2.yaml Configuration:

plugin: aws_ec2
regions:
  - us-east-1
filters:
  instance-state-name: running

• Explanation: This configuration fetches all running instances in the us-east-1 region.

If the inventory script fails, no hosts will be available, and tasks will be skipped.

Ansible Galaxy

Ansible Galaxy is a community hub for sharing and downloading reusable roles and collections. It simplifies the management of commonly used configurations.

Using Ansible Galaxy

1. Installing a Role:

   ansible-galaxy install geerlingguy.apache
   

   • Explanation: This command installs the geerlingguy.apache role from Ansible Galaxy.

2. Using the Installed Role:

   - name: Use Apache Role
     hosts: webservers
     roles:
       - geerlingguy.apache
   

3. Creating Your Own Role:

   ansible-galaxy init my_custom_role
   

   • Explanation: Initializes a new role structure in the my_custom_role directory.

You can customize downloaded roles by editing the role files or overriding their variables in your playbook.

Using Ansible in Real-world Scenarios

Automating Software Installation

Automating software installation with Ansible ensures that required packages and applications are installed uniformly across all target hosts. This eliminates manual setup errors and saves time.

Example: Installing a Web Server

- name: Install and start Apache Web Server
  hosts: webservers
  become: yes # This is used to run the task with elevated privileges
  tasks:
    - name: Install Apache
      apt:
        name: apache2
        state: present
    - name: Start and enable Apache
      service:
        name: apache2
        state: started
        enabled: yes

Explanation:

• apt module: Ensures the apache2 package is installed.
• service module: Starts the Apache service and enables it to run on boot.
• Outcome: Apache will be installed and running on all hosts in the webservers group.

Ansible ensures the desired state (present). If Apache is already installed, the task will skip.

Configuring Servers

Server configuration involves setting up services, managing files, and ensuring the server is ready for specific workloads. Ansible simplifies these tasks by making them repeatable and consistent.

Example: Configuring SSH Access

- name: Configure SSH Access
  hosts: all
  become: yes
  tasks:
    - name: Create a new user
      user:
        name: devops
        shell: /bin/bash
        state: present
    - name: Copy SSH key to the new user
      authorized_key:
        user: devops
        key: "{{ lookup('file', '/path/to/public_key.pub') }}"
    - name: Restrict root SSH login
      lineinfile:
        path: /etc/ssh/sshd_config
        regexp: "^PermitRootLogin"
        line: "PermitRootLogin no"
      notify: Restart SSH
  handlers:
    - name: Restart SSH
      service:
        name: sshd
        state: restarted

Explanation:

• user module: Creates a new user (devops).
• authorized_key module: Adds an SSH public key to the devops user for secure login.
• lineinfile module: Updates the SSH configuration to restrict root login.
• Handler: Restarts the SSH service if the configuration is changed.

Ansible can configure all servers in the inventory at once or a subset using tags or host groups.

Managing Cloud Infrastructure

Ansible enables you to manage cloud resources across platforms like AWS, Azure, or GCP. It uses modules specific to each cloud provider to automate tasks such as creating instances, setting up networking, or provisioning storage.

Example: Creating an EC2 Instance in AWS

- name: Launch EC2 Instance
  hosts: localhost # This is used to run the task on the local machine
  tasks:
    - name: Provision an EC2 instance
      amazon.aws.ec2_instance:
        name: "web-server"
        key_name: "my-aws-key"
        instance_type: "t2.micro"
        image_id: "ami-0c55b159cbfafe1f0"
        region: "us-east-1"
        state: present
        tags:
          Name: "Ansible-managed-instance"

Explanation:

• amazon.aws.ec2_instance module: Manages AWS EC2 instances.
• Key parameters:
  • name: Name of the instance.
  • key_name: The SSH key for secure access.
  • state: Ensures the instance is created (present).
• Outcome: A new EC2 instance named “web-server” is provisioned in AWS.

The task ensures the desired state. If the instance exists, no changes are made.

Continuous Integration/Continuous Deployment (CI/CD)

Ansible integrates seamlessly into CI/CD pipelines to automate the deployment of applications. By defining deployment tasks in playbooks, you can reduce deployment time and ensure consistency.

Example: Automating Application Deployment

- name: Deploy Application
  hosts: app_servers
  become: yes
  vars:
    app_source: "https://github.com/example/app.git"
    app_dir: "/var/www/app"
  tasks:
    - name: Install Git
      apt:
        name: git
        state: present
    - name: Clone the application repository
      git:
        repo: "{{ app_source }}"
        dest: "{{ app_dir }}"
    - name: Install application dependencies
      command: npm install
      args:
        chdir: "{{ app_dir }}"
    - name: Restart application service
      service:
        name: app_service
        state: restarted

Explanation:

• git module: Clones the application code from a Git repository.

• command module: Runs npm install to install dependencies.

• service module: Restarts the application service after deployment.

• Outcome: The latest version of the application is deployed and ready to use.

• Use Ansible playbooks as part of your pipeline script in Jenkins or GitHub Actions.

• Example Jenkins pipeline snippet:

  stage('Deploy') {
    steps {
      sh 'ansible-playbook deploy_app.yml'
    }
  }
  

Best Practices and Tips for Using Ansible

Structuring Ansible Projects

A well-organized Ansible project ensures scalability, maintainability, and clarity, especially for larger projects involving multiple playbooks and roles.

Recommended Directory Structure:

project_name/
├── ansible.cfg
├── inventory/
│   ├── production
│   ├── staging
├── group_vars/
│   ├── webservers.yml
│   ├── dbservers.yml
├── roles/
│   ├── webserver/
│   │   ├── tasks/
│   │   │   └── main.yml
│   │   ├── templates/
│   │   └── vars/
│   │       └── main.yml
├── playbooks/
│   ├── site.yml
│   ├── db_setup.yml
└── README.md

Explanation:

• ansible.cfg: Central configuration for the project. Define default paths, inventory, and privilege escalation settings.
• inventory: Stores separate inventories for different environments (e.g., production, staging).
• group_vars: Contains variables scoped to specific groups of hosts, like webservers or dbservers.
• roles: Encapsulates tasks, templates, and variables for reusability.
• playbooks: Contains playbooks targeting specific tasks or overall site setup.

This structure separates concerns, keeps the project manageable, and adheres to Ansible best practices.

Optimizing Performance in Ansible

Efficient playbooks save time, reduce overhead, and make deployments smoother.

Best Practices for Optimization:

1. Reduce SSH Connections: Use gather_facts: false if facts are not needed, as it reduces time spent gathering data from hosts.

   - name: Optimized Playbook
     hosts: all
     gather_facts: false
   

2. Use async for Parallel Tasks: Execute time-consuming tasks asynchronously.

   - name: Run a long task in parallel
     command: /usr/bin/long_script.sh
     async: 300
     poll: 0
   

3. Leverage Fact Caching: Cache facts to avoid re-gathering them on every run.
   Add this in ansible.cfg:

   [defaults]
   fact_caching = jsonfile
   fact_caching_connection = /path/to/cache
   

Tasks using async or batch-size execute out of sequence but improve overall speed for independent tasks.

Security Considerations in Ansible

Securing your Ansible setup is crucial, especially when dealing with sensitive data or production environments.

Using Ansible Vault:

Encrypt sensitive data such as passwords or private keys.

ansible-vault encrypt secrets.yml
ansible-vault decrypt secrets.yml
ansible-vault edit secrets.yml

Example: Encrypted Inventory Variables:

group_vars/
└── dbservers.yml  # Encrypted with Ansible Vault

Secure Permissions:

Ensure files like ansible.cfg, inventories, and playbooks have restrictive permissions to prevent unauthorized access:

chmod 600 ansible.cfg inventory/production

You can decrypt vault files during pipeline execution using a secure password store or environment variables.

Troubleshooting Ansible Playbooks

Debugging issues in playbooks is essential for smooth deployments.

Common Techniques:

1. Enable Debug Mode: Use the -vvv flag to increase verbosity.

   ansible-playbook playbook.yml -vvv
   

2. Use the debug Module: Print variable values or check conditions.

   - name: Debug variable value
     debug:
       var: ansible_facts['hostname']
   

3. Dry-run with check_mode: Preview changes without making them.

   - name: Test playbook without applying changes
     hosts: all
     tasks:
       - name: Dry-run task
         copy:
           src: /tmp/file
           dest: /etc/file
         check_mode: yes
   

Example: Conditional Error Handling:

- name: Task with error handling
  command: /bin/false
  ignore_errors: yes
  register: result

- name: Debug the error output
  debug:
    var: result.stderr

Explanation:

• ignore_errors: Ensures the playbook continues even if the task fails.
• Outcome: Error details are captured in the result variable for debugging.

Check SSH connectivity or timeout settings in ansible.cfg. Increase the timeout value if needed:

[defaults]
timeout = 60

Why Ansible is a Great Tool for Automation

1. Agentless Architecture

One of the biggest advantages of Ansible is that it is agentless.

• Unlike other tools (e.g., Puppet or Chef), Ansible does not require a special agent to be installed on target machines.
• It uses SSH (or WinRM for Windows systems) to communicate with hosts, making it lightweight and easy to implement.

Command Example:

ansible all -i inventory.txt -m ping

• What it does:
  Pings all hosts defined in inventory.txt to verify connectivity.
• Outcome:
  Displays whether Ansible can successfully reach and manage the hosts.

2. Simple and Human-Readable YAML Syntax

Ansible uses YAML, which is both easy to write and read. This reduces the learning curve for beginners.

Example Playbook:

- name: Install Apache Web Server
  hosts: webservers
  tasks:
    - name: Ensure Apache is installed
      apt:
        name: apache2
        state: present

YAML is intuitive and declarative, making the automation logic clear even to those with limited programming experience.

3. Cross-Platform Compatibility

Ansible works seamlessly with Linux, Windows, and cloud platforms like AWS, Azure, and GCP.

A single tool can manage on-premise servers, cloud instances, and containers, reducing the need for multiple automation tools.

4. Extensibility through Modules and Roles

Ansible’s rich library of built-in modules and support for custom modules allows it to cater to a variety of use cases.

Example:

• Using the yum module for installing packages on RHEL-based systems:

  - name: Install Git
    yum:
      name: git
      state: latest
  

Modules abstract away complex tasks, making automation faster and more reliable.

5. Strong Community Support

The vibrant Ansible community offers:

• Ready-to-use roles on Ansible Galaxy.
• Active forums and GitHub repositories for collaboration and troubleshooting.

Beginners can leverage community-made roles for faster implementation.

Final Thoughts and Future of Ansible

1. Ansible’s Current Role in Automation

Ansible has become the go-to tool for IT automation due to its simplicity, flexibility, and powerful features. From provisioning infrastructure to orchestrating CI/CD pipelines, Ansible plays a critical role in modern DevOps workflows.

Its agentless model, cross-platform compatibility, and extensive community make it ideal for diverse use cases.

2. Upcoming Features and Roadmap

The Ansible development team actively works on new features to meet evolving automation needs.

Event-Driven Automation

• Upcoming Ansible versions focus on event-driven automation, where tasks are triggered automatically based on specific system events.
• This eliminates the need for manual playbook execution, reducing latency and improving response times.

Enhanced Cloud-Native Capabilities

• Expect tighter integrations with Kubernetes, OpenShift, and other cloud-native tools.
• Users can seamlessly automate container deployments, service scaling, and orchestration tasks.

Improved Performance and Scalability

• Continuous improvements to Ansible’s execution engine promise better performance for managing large infrastructures.

3. The Future of Automation

As organizations increasingly adopt DevOps and cloud-first strategies, automation tools like Ansible will remain essential for managing complex infrastructures.

Ansible’s adaptability positions it to handle emerging trends like:

• Edge Computing: Automating edge device configurations and deployments.
• AI-Driven Automation: Leveraging AI to suggest optimal playbook logic and detect inefficiencies.

Key Takeaways

References/Resources

• Official Documentation: Ansible documentation
• Online Tutorials and Courses: Ansible tutorials
• Blogs and Articles: Ansible blogs