Wednesday 6 July 2022

Introduction to Ansible


Ansible helps:
  • automate the deployment
  • manage automation
  • orchestrate
  • manage configurations
  • deploy applications
  • provision/deprovision
  • deliver continuously
  • secure and comply
on
  • firewalls
  • load balancers
  • applications
  • containers
  • virtualization platforms
  • servers
  • clouds
  • storage
  • network devices
  • ...
 
Ansible is:
  • Infrastructure as Code (IaC) tool that allows using a single central location (Ansible control node) to monitor and control a large number of remote servers (hosts)
  • set of tools for orchestration, configuration management, deployment, and task execution mainly by using playbooks
  • used for configuring Linux and Windows operating systems over the ssh protocol 
 

Ansible VS Bash - When to use one over another?

 
 
 
 
The benefit of using Ansible is that commands are idempotent, this means that we can re-run commands and if the effect we desire is already there nothing will change, this is an improvement over creating our own scripts that will probably have unforeseen consequences when we run them multiple times. 
 

Ansible VS Terraform - When to use one over another?

Use Terraform to provision resources and then Ansible to install software on them.

 
Announcements of the latest releases: Ansible Announcements - Google Groups

 

Ansible Core 

 
Ansible Core components:
  • CLI - includes ansible, ansible-playbook, ansible-doc and other tools 
  • language - YAML, for crating set of rules for developing playbooks
  • framework - allows collections to be installed and operated from Ansible Hub and Ansible Galaxy
  • functions - includes conditionals, blocks, includes, loops and other imperatives 

 

Ansible Playbooks

 

  • lists of tasks we want our remote hosts to perform
  • describe the desired state for operating system properties like files, services, filesystems etc...
  • can be saved and reused, allowing to automate complex processes
  • They are written as YAML documents. 
Playbook example:

playbook.yaml:

# Taken from Red Hat's "Ansible Basics: An Automation Technical Overview" Udemy course
---
- name: Install and start Apache
  hosts: web
  become: yes

  tasks:
    - name: httpd package is present
      yum:
        name: httpd
        state: latest

    - name: latest index.html file is present
      template:
        src: files/index.html
        dest: /var/www/html/

    - name: httpd is started
      service:
        name: httpd
        state: started

 
A valid yaml file starts with triple dash (---).

Ansible playbook consists of:
  • plays
    • single playbook can contain multiple plays
  • modules
    • tasks have 1:1 correlation with modules e.g. yum module, template module, service module
    • task name is optional but recommended
    • module has parameters e.g. yum has name and state (in the example above this instructs yum to install the latest httpd)
  • plugins

Running Playbooks


Playbooks is interpreted and run against one or more hosts, task by task. 
The order of the tasks defines the execution.
In each task the module does the actual work.
 
Output messages are coloured:

A task executed as expected, no change was made.
A task executed as expected, making a change.
A task failed to execute successfully. 

To run playbooks we can use ansible-playbook or Ansible Navigator

If we run playbook for the first time, we'll see in the output that host has been changed for each task:

 
PLAY [play_name] 

TASK [task0]
ok: [node1]
 
TASK [task1]
changed: [node1]
 
TASK [task1]
changed: [node1]
 
TASK [task1]
changed: [node1]
 
PLAY RECAP
node1: ok=4 changed=3 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

 
Anisble follows the concept of idempotence: it knows what the state of the host should be (based on the playbook) and if it's already in that state, it will do nothing. So, next time we run the same playbook, nothing would be changed (provided that no manual changes were made on the host):

PLAY [play_name] 

TASK [task0]
ok: [node1]
 
TASK [task1]
ok: [node1]
 
TASK [task1]
ok: [node1]
 
TASK [task1]
ok: [node1]
 
PLAY RECAP
node1: ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0


This shows that Ansible is not just configuration management but also policy enforcement tool. We can schedule e.g. daily hosts & inventory audit by scheduling daily execution of this playbook via automation controller and each time it is run, Ansible will check if the state of the hosts is compliant with policies set in playbooks. If someone manually makes changes on hosts, that would be detected and related tasks would be executed reverting those changes to the desired state defined in the playbook.
 
 

Plays

  • they describe what is being automated
  • top level specification for a group of tasks
  • full list of its attributes
  • we must specify: 
    • hosts where play will be executed on (simultaneously, in parallel)
    • control behaviour (fact gathering, privilege level). E.g. privilege escalation is defined by become. yes means that Ansible will become sudo-er by default.
  • single playbook that executes on different hosts can contain multiple plays
Example:

- name: Install and start Apache
  hosts: web
  become: yes

Modules

  • "tools in the toolkit"
  • parametrized components with internal logic, representing a single step to be done
  • they "do" things in Ansible
  • each task is 1 to 1 with a module
  • can use any language but usually Python or Powershell (for Windows)
  • there are Ansible built-in modules (part of ansible-core and included in all Ansible installations), community modules, modules of 3rd party providers (AWS, Cisco, Google, ...) etc...: full list of modules
  • here are some built-in modules:

Example: template task is using template (ansible.builtin.template) module

    - name: latest index.html file is present
      template:
        src: files/index.html
        dest: /var/www/html/

 

Plugins

  • the "extra" bits that allow us to change behaviour or do extra things outside Ansible
  • technically, each module is a plugin
  • pieces of code that augment Ansible's core functionality
  • Ansible uses a plugin architecture to enable a rich, flexible and expandable feature set

Example #1: become plugin - for privilege escalation that can be done on a play or module level

 become: yes

Example #2: filter plugins - allow us to change the variable output of a particular task into a JSON, YAML or CSV

{{ some_variable | to_nice_json }}
{{ some_variable | to_nice_yaml }}

 

Ansible Inventory

  • systems/hosts or groups of hosts that a playbook runs against
  • Ansible works against multiple systems in an inventory
  • List of systems in your infrastructure that automation is executed against
  • Groups can be defined so certain playbooks are executed only on those hosts within the group and not on other hosts, within other groups.
  • Inventory is usually file based; could be a DB, GitHub
  • Can have multiple groups
  • Can have variables for each group or even host
    • Good practice is that these variables are used only to connect to devices e.g. (encrypted) username, password, type of connection (SSH, API, ..)
       
  • From Safespring | Säkra, lokala molntjänster
When working with static hosts in a data center, inventories are often also static textfiles maintained manually or semi-manually. However, inventories can also be dynamic, i.e. provided by scripts. 
When working with OpenStack, it is possible to use inventory scripts that queries the OpenStack API directly and produces a complete inventory of all instances with metadata, all the group memberships and so on, but oftentimes these scripts take a long time to run, and they generally need to run every time you run a playbook, thus making playbook runs orders of magnitude more time-consuming than static inventories. Also, they can put a heavy load on the OpenStack APIs if the inventory is frequently queried.

 
Example:
 
[web]
webserver1.example.com
webserver2.example.com
 
[db] 
dbserver1.example.com

[switches]
leaf01.internal.com
leaf02.internal.com
 
 
Use [group_name] format to define a group of hosts. e.g. web, db and switches are groups of hosts.

Example #2: static inventory includes systems with IP addresses as well as FQDN (Fully Qualified Domain Name).

[myservers]
10.42.0.3
10.42.0.24
10.42.0.101
host.example.com
 
Example #3 shows:
  • pinning a host to a particular IP address by using ansible_host keyword. Without this, DNS is used by default.
  • group variables by using [group_name:vars] construct
    • all is an implicitly included group, default group that every host is in it

[app1srv]
appserver01 ansible_host=10.42.0.3
appserver02 ansible_host=10.42.0.24
 
[web] 
webserver01

[web:vars]
apache_listen_port=8080
apache_root_path=/var/www/mywebdocs/

[all:vars]
ansible_user=ansible
ansible_ssh_private_key_file=/home/ansible/.ssh/id_rsa

Example #4 shows groups of groups, defined via [group_name:children] construct. They can also contain standalone hosts:

[nashville]
bnaapp01
bnaapp02
 
[atlanta]
atlapp03
atlapp04
 
[south:children] 
atlanta
nashville
hsvapp05

 

Ansible Roles

  • reusable automation actions
  • reusable playbooks
  • groups of tasks that usually get executed together
  • a reusable structure that groups our automation tasks and variables 
  • they are written once and can be shared
  • can be stored in collection

Example:

- name: Install and start Apache
  hosts: web
  become: yes
  roles:
     - common
     - webservers 
 

Ansible Collections

  • the way to share a content for Ansible
  • simplified and consistent content delivery
  • data structure containing automation content:
    • modules
    • playbooks
    • roles
    • plugins
    • docs
    • tests
  • a file structure (tarball) for sharing content that community agreed
  • collection examples: Cisco IOS, Rist IOS, NetApp collection
  • there are public collections available for 100+ platforms e.g. AWS, CheckPoint, Google, Cisco, IBM, Microsoft, ...
  • can be created within organization e.g. network toolkit
  • standard unit of automation
  • allow we work asynchronously from the Ansible releases
  • collections can have dependencies on other collections


Ansible project structure:

nginx_core
    MANIFEST.json
    playbooks
        deploy-nginx.yml
        ...
    plugins
    README.md
    roles
        nginx
            defaults
            files
                ...
            tasks
            templates
                ...
        nginx_app_protect
        nginx_config
     

deploy-nginx.yml (collection):
 
---
-   name: Install NGINX Plus
    hosts: all
    tasks:
        - name: Install NGINX 
          include_role:
                name: nginxinc.nginx
          vars:
                nginx_type: plus
        - name: Install NGINX App Protect
          include_role:
                name: nginxinc.nginx_app_protect
         vars:
                nginx_app_protect_setup_license: false
                nginx_app_protect_remove_license: false
                nginx_app_protect_install_signatures: false 

 


Automation Execution Environments

 

Collections might have different dependencies e.g. Python version, APIs, SDKs, OS system libraries, particular Ansible version. Automation Execution Environments (AEE) keep runtime environment, collections and dependencies aligned by packaging all components together in a cloud-native way.

AEE = Universal Base Image (Collections + Libraries + Ansible Core) 

There are minimal AEEs available, that we can use to build our own AEEs on top of.

Execution Environment builder is used to build images.

Automation Content Navigator is used for running AEEs.  


Ansible Automation Execution


2 types:
  • local
    • for automating Network Devices / API endpoints (e.g. AWS)
    • module code is executed locally on the control node
    • this is run as parallel processes on the control node
    • Ansible Automation Platform can be installed as a cluster and scale this further
       
  • remote
    • for executions on Linux/Windows hosts
    • code is executed on the end system
    • module code is copied to the managed node, executed, then removed
    • during execution, these remote agents are sending back to control node JSON blobs with information on what action it performed, success of actions etc...
       

Configuration File


  • ansible.cfg file
  • basic configuration for Ansible
  • can be in multiple locations, with different precedence

 

Ansible Variables

 

Example:


---
- name: variables demo playbook
  hosts: localhost
  vars:
     var_one: Hello
     var_two: world
     # example of variable unpacking (double curly braces):
     var_three: "{{var_one}}, {{var_one}}!"
  tasks:
  - name: print out var_three
   # debug module is handy for printing variable values (or any other messages)
   debug:
     msg: "{{ var_three}}" # "Hello, world!"


Ansible Facts

 

They are variables with standard (pre-defined) names whose values are coming from hosts themselves (local host or remote, managed nodes), e.g. host name, or host IPv4 address:

---
- name: Output localhost facts
  hosts: localhost
  tasks:
  - name: Prints localhost facts
    debug:
      msg: The default IPv4 address of {{ ansible_fqdn }} is {{ ansible_default_ipv4.address }}

To see run this playbook:
 
$ ansible-playbook facts_demo.yaml
...
TASK [Prints Ansible facts] ***********************************************************************************************************************************************************
ok: [localhost] => {
    "msg": "The default IPv4 address of bojans_computer is 11.186.34.126"
}
...

Built-in module ansible.builtin.setup module – Gathers facts about remote hosts — Ansible Documentation collects all facts from a local Linux system:

This module is automatically called by playbooks to gather useful variables about remote hosts that can be used in playbooks. It can also be executed directly by /usr/bin/ansible to check what variables are available to a host. Ansible provides many facts about the system, automatically.

"automatically called by playbooks" means that each playbook implicitly has this line added in yaml:

gather_facts: true

For custom platforms there are different modules, e.g. for Cisco devices, there is cisco.ios.ios_facts module – Module to collect facts from remote devices. — Ansible Documentation.
 

To see values of all facts from a local host we need to specify setup module and set host pattern to localhost:

$ ansible -m setup localhost

To see values of all facts from all hosts:

$ ansible -m setup all

Facts can be used for host audits e.g. to check hardware attributes for vulnerabilities, uptime etc...They can also be used for creating dynamic reports e.g. like web page which shows attributes of all hosts in a system (see example network-automation/ansible_inventory_report: This repo contains an Ansible networking inventory report playbook (inventory.yml). This playbook exports hostname, platform, mgmt0 IP address and code version to a HTML file. The jinja2 template used for the website can also highlight the version of code if it doesn't match a desired version.). 

---

References

No comments: