System Comparison

This is an example of a playbook designed to compare two systems ensure they are in sync. For example comparing a Primary node to its DR counterpart. It does this by putting the output through md5sum and comparing the two systems.

It requires a hosts.ini file that contains groups of two hosts and a variable with the commands used to validate a system.

Hosts

[Chef_Infra_Server]
chefinfra_main.example.com
chefinfra_dr.example.com
[Chef_Infra_Server:vars]
command='(/usr/bin/knife cookbook list -c /home/chef_admin/chef/.chef/knife.rb && /usr/bin/knife group show clients -c /home/chef_admin/chef/.chef/knife.rb && /usr/bin/knife group show admins -c /home/chef_admin/chef/.chef/knife.rb && /usr/bin/knife vault list --mode client -c /home/chef_admin/chef/.chef/knife.rb) | sort'

[App_Server]
app_server_main.example.com
app_server_dr.example.com
[App_Server:vars]
command='md5sum /app/payara/micro/deploy/*'

[Red_Hat_Satellite]
satellite_main.example.com
satellite_dr.example.com
[Red_Hat_Satellite:vars]
command='hammer status | grep -i version | sort'

system_comparison.yml

---

- name: Start of system_comparison playbook
  hosts: "{{ env }}"
  vars:
  gather_facts: True
  become: True
  become_flags: '-i'
  ignore_unreachable: True
  ignore_errors: True
  tasks:

    - name: Get file md5sums or command outputs
      shell: "{{ command }}"
      register: md5sum

    - name: Take output and convert to checksum.
      debug:
        msg: "{{ ( md5sum | default('empty') ).stdout_lines | checksum }}"

- name: Generate and send email
  hosts: localhost
  ignore_errors: False
  tasks:

    - name: Email report of different versions of software between environments
      become: False
      mail:
        host: mail.example.com
        from: notify@ansible.example.com
        to: "{{ other_email | default('Automation <auto@example.com>') }}"
        subject: "System Comparison Report {{ ansible_date_time.date }}"
        body: "{{ lookup('template', 'system_comparison.j2') }}"
        secure: never
      delegate_to: localhost
      run_once: true

...

system_comparison.j2

The comparison report gives a PASSED output if the md5sums match and FAILED if not. It will also give a FAILED if one of the systems is not accessible, but it will identify the system as not accessible to help call out that backup system is not ready or even built.

Comparison report between servers.
-----------------------------------

{% for list in groups %}
{% if 'all' not in list %}
{% set count = [0] %}
{% for host in groups[list] %}
{% if hostvars[host]['md5sum']['stdout_lines'] is defined %}
{% if hostvars[groups[list][0]]['md5sum']['stdout_lines'] is defined %}
{% if hostvars[groups[list][0]]['md5sum']['stdout_lines'] | checksum == hostvars[host]['md5sum']['stdout_lines'] | checksum %}
{% else %}
{% if count.append(count.pop() + 1) %}{% endif %}
{% endif %}
{% else %}
{% if count.append(count.pop() + 1) %}{% endif %}
{% endif %}
{% else %}
{% if count.append(count.pop() + 1) %}{% endif %}
{% endif %}
{% endfor %}

{{ list }} environment {% if count >= [1] %}FAILED{%else%}PASSED{%endif%}
---------------------------------------------------------------------
{% for host in groups[list] %}
{% if hostvars[host]['md5sum']['stdout_lines'] is defined %}
{% if hostvars[groups[list][0]]['md5sum']['stdout_lines'] is defined %}
{% if hostvars[groups[list][0]]['md5sum']['stdout_lines'] | checksum == hostvars[host]['md5sum']['stdout_lines'] | checksum %}
{# PASSED #} {{ host.split(".")[0] }} has checksum {{ hostvars[host]['md5sum']['stdout_lines'] | checksum }}
{% else %}
{# FAILED #} {{ host.split(".")[0] }} has checksum {{ hostvars[host]['md5sum']['stdout_lines'] | checksum }}
{% endif %}
{% else %}
{% if hostvars[host]['md5sum']['stdout_lines'] is defined %}
{# FAILED #} {{ host.split(".")[0] }} has checksum {{ hostvars[host]['md5sum']['stdout_lines'] | checksum }}
{% else %}
{# FAILED #} {{ host }} not checked (Server either doesn't exist or needs to be turned on.)
{% endif %}
{% endif %}
{% else %}
{% if hostvars[host]['md5sum']['stdout_lines'] is defined %}
{# FAILED #} {{ host.split(".")[0] }} has checksum {{ hostvars[host]['md5sum']['stdout_lines'] | checksum }}
{% else %}
{# FAILED #} {{ host }} not checked (Server either doesn't exist or needs to be turned on.)
{% endif %}
{% endif %}
{% endfor %}
{% endif %}
{% endfor %}

Variables

env - Intended to always be all but this selects the list of servers from the host.ini to be used.
other_email - Used to override the default email address.

Commands

By default the play will email Automation <auto@example.com>.

ansible-playbook -i hosts.ini system_comparison.yml -e "env=all"

But you can override the defaults using the other_email variable.

ansible-playbook -i hosts.ini system_comparison.yml -e "env=all other_email=<EMAIL@ADDRESS1>,<EMAIL@ADDRESS2>"