Comparing Microsoft Copilot vs Google Gemini Pro for installing Nagios using Ansible

I’ve been tinkering with the current generation of AI for a while now. I am amazed by the image generation capabilities of Copilot. I’ve already used generative AI to suggest blog names for my hobby blog and to generate a logo for it as well.

This got me thinking. How good is generative AI at more IT related tasks and what would be a good test?

There is no point in choosing something obscure, so I need something old and well known to give the AI a chance. You can’t get better known than Nagios in the system monitoring space and it is certainly old enough to have garnered plenty of food for the AI to learn from.

My test is going to be installing Nagios on an Ubuntu 22.04 host using Ansible. I chose Ansible because I’m familiar with it and would normally use it to configure a new host.

Test setup

The queries were executed on a Lenovo Thinkpad running Windows 11. The scripts were tested on an old Lenovo Thinkpad X230 running Ubuntu 22.04, Ansible 2.12.10 and LXD hosting the Ubuntu 22.04 container where Nagios is installed.

Ansible

Ansible is a powerful tool for ensuring a host is in an appropriate state. It can install software, create or alter configuration files among many other things.

I have used the query Write an ansible playbook for installing nagios on an ubuntu 22.04 host with both Copilot and Google Gemini Pro.

Microsoft Copilot

I launched the query using the preview Copilot in Windows sidebar installed in Windows 11.

The query produced the Ansible playbook listed below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

---
- name: Install Nagios
  hosts: nagios
  become: yes
  vars:
    nagios_version: 4.4.6
    nagios_user: nagios
    nagios_group: nagcmd
    nagios_admin_pass: changeme
  tasks:
    - name: Install dependencies
      apt:
        name: "{{ item }}"
        state: present
        update_cache: yes
      loop:
        - apache2
        - php
        - libapache2-mod-php
        - libgd-dev
        - libmcrypt-dev
        - libssl-dev
        - snmp
        - libnet-snmp-perl
        - gettext
        - nagios-plugins
        - nagios-nrpe-plugin

    - name: Create nagios user and group
      user:
        name: "{{ nagios_user }}"
        group: "{{ nagios_group }}"
        system: yes
        create_home: yes

    - name: Download Nagios Core
      get_url:
        url: "https://github.com/NagiosEnterprises/nagioscore/releases/download/nagios-{{ nagios_version }}/nagios-{{ nagios_version }}.tar.gz"
        dest: "/tmp/nagios-{{ nagios_version }}.tar.gz"
        mode: 0644

    - name: Extract Nagios Core
      unarchive:
        src: "/tmp/nagios-{{ nagios_version }}.tar.gz"
        dest: "/tmp"
        remote_src: yes

    - name: Configure Nagios Core
      command: "{{ item }}"
      args:
        chdir: "/tmp/nagioscore-nagios-{{ nagios_version }}"
      loop:
        - ./configure --with-command-group={{ nagios_group }}
        - make all
        - make install
        - make install-init
        - make install-commandmode
        - make install-config
        - make install-webconf

    - name: Enable Apache modules
      apache2_module:
        name: "{{ item }}"
        state: present
      loop:
        - authz_groupfile
        - auth_digest

    - name: Set nagiosadmin password
      command: htdigest -c /usr/local/nagios/etc/htdigest.users Nagios4 nagiosadmin <<< "{{ nagios_admin_pass }}"

    - name: Start and enable Apache and Nagios services
      service:
        name: "{{ item }}"
        state: started
        enabled: yes
      loop:
        - apache2
        - nagios

I executed the script as generated by Copilot using the instructions provided. The image below shows the results.

The script failed because when registering the nagios user, the group specified nagcmd does not exist.

Google Gemini Pro

I launched the query using the Gemini app.

The Ansible playbook generated by Google Gemini is listed below:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104

---
- hosts: nagios_servers  # Target group from inventory
  become: yes

  vars:
    nagiosadmin_password: "your_strong_password"  # Change this!

  tasks:
    - name: Install prerequisites
      apt: 
        name: 
          - build-essential
          - libgd-dev
          - libapache2-mod-php
          - openssl
          - libssl-dev
          - unzip
          - apache2
        state: present

    - name: Create Nagios user and group
      group:
        name: nagios
        state: present
      user:
        name: nagios
        group: nagios
        create_home: no

    - name: Download Nagios Core
      get_url:
        url: https://assets.nagios.com/downloads/nagioscore/releases/nagios-4.4.10.tar.gz  # Update version as needed
        dest: /tmp/nagios-4.4.10.tar.gz

    - name: Extract Nagios Core
      unarchive:
        src: /tmp/nagios-4.4.10.tar.gz
        dest: /tmp
        copy: no

    - name: Configure Nagios Core
      shell: ./configure --with-httpd-conf=/etc/apache2/sites-enabled --with-nagios-group=nagios --with-command-group=nagcmd
      args:
        chdir: /tmp/nagios-4.4.10  # Update path if version changes

    - name: Compile Nagios Core
      shell: make all
      args:
        chdir: /tmp/nagios-4.4.10

    - name: Install Nagios binaries, init script, and sample config files
      shell: make install
      args:
        chdir: /tmp/nagios-4.4.10

    - name: Install Nagios web interface
      shell: make install-webconf
      args:
        chdir: /tmp/nagios-4.4.10

    - name: Create Nagios admin account (nagiosadmin)
      shell: htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
      args:
        chdir: /tmp/nagios-4.4.10
        # You'll be prompted for the password

    - name: Install and enable Nagios command file
      shell: make install-commandmode
      args:
        chdir: /tmp/nagios-4.4.10

    - name: Install Nagios config files
      shell: make install-config
      args:
        chdir: /tmp/nagios-4.4.10

    - name: Install Nagios example service and host definitions
      shell: make install-init
      args:
        chdir: /tmp/nagios-4.4.10

    - name: Enable Apache modules
      apache2ctl:
        name: "{{ item }}"
        state: enabled
      with_items:
        - rewrite
        - cgi

    - name: Verify Nagios configuration
      command: /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

    - name: Start and enable Nagios service
      service:
        name: nagios
        state: started
        enabled: yes

    # Optional: Install Nagios Plugins
    - name: Download Nagios Plugins
      get_url:
        url: https://nagios-plugins.org/download/nagios-plugins-2.4.4.tar.gz # Update as needed
        dest: /tmp/nagios-plugins-2.4.4.tar.gz
      # ... (Similar installation steps as Nagios Core) 

One thing that I immediately notice is that the Nagios version number is hard coded throughout the Gemini generated script. In the Copilot script, the Nagios version is stored in a variable at the top of the script. This makes the Copilot script much easier to maintain when you need to upgrade to a new version of Nagios. Simply go to the top of the script, find the nagios version number variable and enter the desired Nagios version number. With the Gemini script, you have to go through the script and find all of the places that the Nagios version number is used. This makes it very easy to miss one of the references and potentially end up installing either the wrong version number or fail to install any version of Nagios.

The Gemini script also installs the Nagios core plugins and marks them as optional. The Copilot generated script does not mention plugins at all. Which is understandable as my query didn’t mention plugins either. But, given how Nagios is architected, it doesn’t do very much without plugins so kudos to Gemini for installing the core add-ons.

The Gemini output could be improved further by providing an example host file and the instructions how to use it. The Copilot instructions are much easier to follow for a beginner because all of the steps are clearly outlined with examples and sample commands. The Gemini instructions are nearly there but leave out crucial details like an example hosts file and the specific command for executing the playbook using the example hosts file.

I executed the script generated by Gemini and it failed with the error displayed below.

The script is invalid. Kudos to Gemini for creating the nagios group but if you wish to group commands together you need to put them under a block. Another problem is that I can’t find any evidence for a module called apache2ctl. There is one called community.general.apache2_module which appears to do the same thing.

Conclusion

I suspect your view of these very capable tools will differ based upon how you are using the tool.

As a sysadmin wishing to learn how to use Ansible, I think you will find either of these tools to be a great aid in helping you learn. Both tools provide further reading at the end of their output. I doubt either tool is going to guide a complete beginner through how to use Ansible to install Nagios. But, if you’ve done some reasearch before then I think these tools will help you a lot in speeding up your existing learning.

If you are a less experienced Ansible user wanting to get things done rather than as a learning tool then you will appreciate how Copilot works.

As an experienced sysadmin these tools are going to help you a lot. I think both tools would be helpful to an experienced Ansible user shortening the time to get a good enough script which can then be tweaked and improved. Where generative AI tools will be invaluable is removing that horrible feeling of starting a project with a blank page. Both these tools provide a starting point that cuts out the blank page and the struggle to get started. I think Copilot on this test provided a particularly good starting point upon which an experience Ansible sysadmin could then improve.

If you think these tools are drop in replacements for a sysadmin, then I think you will be disappointed. Neither of the playbooks worked. If you are expecting a tool that just works first time, you are going to be disappointed. You still need to know what you are doing. The tools will save you some time or they’ll help you learn faster. That’s about as good as this generation of tools is going to be.

If I had to choose between the two, I’d go with Copilot. I think the script is superior as well as the instructions being clearer.

P.S. The post thumbnail was generated by Copilot in answer to the query create an image for a blog post comparing the performance of Copilot vs Google Gemini 😉