AIX Patch Management with Ansible

Leading enterprises today use Red Hat Ansible Automation Platform to provision, configure, manage, secure and orchestrate hybrid IT environments. A common misconception is that Ansible is just used to manage the Linux operating system. This is a false belief. Ansible supports Linux, Windows, AIX, IBM i and IBM z/OS environments. This blog will help AIX system administrators get started with Ansible on AIX, and introduce a patching use case.

Ansible Content Collections

When Ansible Automation Platform was released, Ansible Content Collections became the de facto standard for distributing, maintaining and consuming automation content. The shift to Collections increased community participation and has exponentially increased the number of stable and supported Ansible modules. Modules delivered via Collections rather than packaged with Ansible Core have resulted in a faster release cadence for new modules.

Let us explore the IBM provided Ansible Collection for AIX. It is important to note that many of the Ansible modules for the Linux operating system will also work on AIX (in addition to the IBM provided AIX modules), making the use cases for Ansible on AIX very broad.

Ansible and AIX, why?

The AIX operating system has been around for 35 years and is used to run business-critical applications. Historically, AIX systems were managed using the tools that ship with AIX, complimented by shell scripts written by AIX system administrators. The problem with this approach is that these scripts can become extremely complex over the years, and often wind up being held together with “duct tape and zip ties”.

As enterprises move to a modern, enterprise-wide automation strategy with Ansible Automation Platform, extending automation to AIX is a great method to simplify and develop consistency in the way AIX systems are supported, all while using the same automation tools that can be used across the enterprise.

Ansible Concepts

First let us cover some basic Ansible concepts that will be used in the example. Further information can be found on the Ansible documentation site (https://docs.ansible.com).

Playbooks, which are ordered lists of tasks and variables that are performed against an inventory of hosts.

Tasks are a single unit of action in Ansible, which calls a module.

Modules are code that Ansible executes. Each module could be something like copying a file to using NIM to trigger an AIX update.

Roles are repeatable bundles of tasks that are contained in a specific directory structure.

Variables within Ansible are called like this “”.

Task delegation is how tasks can be delegated to another host in the inventory, other than the host that the Ansible run is targeted against.

Getting started with Ansible

In this example, I’m using a Fedora Linux 34 workstation, so I’m going to use the dnf package manager to install Ansible:

$ sudo dnf install -y ansible

Once Ansible is installed, I’m going to install the ibm.power_aix Collection:

$ ansible-galaxy collection install ibm.power_aix

When Ansible is installed, a default inventory file /etc/ansible/hosts is created. At this point, I’m going to include in the inventory the hosts used in this example:

nim01 is our AIX 7.2 NIM Master which is functional and has an lpp_source defined.
bruce is an AIX 7.2 NIM client registered to the nim01 NIM master.
freddie is an AIX 7.2 NIM client registered to the nim01 NIM master.
$ cat /etc/ansible/hosts
nim01 ansible_host=10.0.0.5 ansible_user=root
bruce ansible_host=10.0.0.6 ansible_user=root
freddie ansible_host=10.0.0.7 ansible_user=root

I’m now going to connect to all the systems over SSH as “root”. The usual practice is to have a service account with “sudo” access, however for this example I will use “root” in our lab environment. Using the ssh-copy-id command, I can distribute my SSH public key to the AIX servers.

$ ssh-copy-id root@nim01
$ ssh-copy-id root@bruce
$ ssh-copy-id root@freddie

The next step is to use the Ansible ping module to check that I can connect to the three hosts in our inventory.

$ ansible -m ping all

PLAY [Ansible Ad-Hoc] ************************************************************************************************************************************************************************************************************************

TASK [ping] **********************************************************************************************************************************************************************************************************************************
ok: [nim01]
ok: [freddie]
ok: [bruce]

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
nim01 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
bruce : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
freddie : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Ansible needs “python” to be installed on the AIX systems, and ideally the “yum” package manager should also be configured on AIX. If your AIX systems do not have these packages installed, or is a vanilla installation of AIX, IBM provides an Ansible Role to “bootstrap” an AIX system and manage it.

The playbook below uses the IBM provided role to prepare an AIX system for Ansible automation.

cat aix_bootstrap.yml
—

– name: Prep AIX for Ansible
hosts: all
vars:
pkgtype: yum
collections:
– ibm.power_aix
roles:
– power_aix_bootstrap

The following example demonstrates running the playbook; however, I can see that the hosts Ansible is running against already have “python” and “yum” installed, so there is no need for any changes to be made to these hosts.

$ ansible-playbook aix_bootstrap.yml

PLAY [Prep AIX for Ansible] ******************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [bruce]
ok: [freddie]
ok: [nim01]

TASK [ibm.power_aix.power_aix_bootstrap : Fail if pkgtype not specified] *********************************************************************************************************************************************************************
skipping: [nim01]
skipping: [bruce]
skipping: [freddie]

TASK [ibm.power_aix.power_aix_bootstrap : Fail if download_dir not specified] ****************************************************************************************************************************************************************
skipping: [nim01]
skipping: [bruce]
skipping: [freddie]

TASK [ibm.power_aix.power_aix_bootstrap : Fail if target_dir not specified] ******************************************************************************************************************************************************************
skipping: [nim01]
skipping: [bruce]
skipping: [freddie]

TASK [ibm.power_aix.power_aix_bootstrap : Fail if rpm_src not specified] *********************************************************************************************************************************************************************
skipping: [nim01]
skipping: [bruce]
skipping: [freddie]

TASK [ibm.power_aix.power_aix_bootstrap : Fail if yum_src not specified] *********************************************************************************************************************************************************************
skipping: [nim01]
skipping: [bruce]
skipping: [freddie]

TASK [ibm.power_aix.power_aix_bootstrap : Bootstrap yum] *************************************************************************************************************************************************************************************
included: /home/tholloway/.ansible/collections/ansible_collections/ibm/power_aix/roles/power_aix_bootstrap/tasks/yum_install.yml for nim01, bruce, freddie

TASK [ibm.power_aix.power_aix_bootstrap : Check for existence of yum] ************************************************************************************************************************************************************************
changed: [bruce]
changed: [nim01]
changed: [freddie]

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
nim01 : ok=3 changed=1 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0
bruce : ok=3 changed=1 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0
freddie : ok=3 changed=1 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0

Now that the platforms meet the required minimum components, I am now ready to automate AIX operations.

Running an AIX Update using NIM and Ansible

First off, I’ll use a simple playbook to see what “oslevel” our NIM master and NIM clients are on, before I start.

$ cat aix_oslevel_check.yml
—

– name: AIX oslevel checking playbook
hosts: all
tasks:

– name: Gather LPP Facts
shell: “oslevel -s”
register: output_oslevel

– name: Print the oslevel
debug:
msg: “{{ ansible_hostname }} has the AIX oslevel of {{ output_oslevel.stdout }}”

Running that playbook delivers the below result. I can see that bruce and freddie are a service pack behind.

$ ansible-playbook aix_oslevel_check.yml

PLAY [AIX oslevel checking playbook ] *****************************************************************************************************************************************************************************************************************

TASK [Gather LPP Facts] **********************************************************************************************************************************************************************************************************************
changed: [freddie]
changed: [bruce]
changed: [nim01]

TASK [Print the oslevel] *********************************************************************************************************************************************************************************************************************
ok: [nim01] =>
msg: nim01 has the AIX oslevel of 7200-05-02-2114
ok: [bruce] =>
msg: bruce has the AIX oslevel of 7200-05-01-2038
ok: [freddie] =>
msg: freddie has the AIX oslevel of 7200-05-01-2038

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
nim01 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
bruce : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
freddie : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

To ensure all systems are operating on the same OS level, I need to download the latest service pack. It should define an “lpp_source” on our NIM master. Make sure that the name of the “lpp_source” matches the example below, or the Ansible module will not detect the “oslevel”.

$ cat aix_download.yml
—

– name: AIX Patching Playbook
hosts: nim01
vars:
oslevel: 7200-05-02
nim_lpp_source: 7200-05-02-2114-lpp_source
collections:
– ibm.power_aix
tasks:

– name: Download AIX Updates
nim_suma:
action: download
download_dir: “/export/nim/lpp_source”
lpp_source_name: “{{ nim_lpp_source }}”
oslevel: “{{ oslevel }}”
targets: ‘bruce, freddie’

Next step is to run the download playbook. It will download the required updates from IBM Fix Central and define an “lpp_source” on the NIM master:

$ ansible-playbook aix_download.yml

PLAY [AIX Patching Playbook] *****************************************************************************************************************************************************************************************************************

TASK [Download AIX Updates] ******************************************************************************************************************************************************************************************************************
changed: [nim01]

Now I can run a patching playbook, which will make use of the “alt_disk” and “nim” Ansible modules. The playbook is going to perform the following tasks:

Remove any existing “altinst_rootvg” “alt_disk_copy” that is left on the AIX system.
Create a new “alt_disk_copy” clone of the root volume group to a spare disk as a backup.
Run an application stop script.
Run the AIX update via task delegation to the NIM master.
Reboot.
Run an application start script.
—

– name: AIX Patching Playbook
hosts: bruce,freddie
vars:
nim_lpp_source: 7200-05-02-2114-lpp_source
nim_master: nim01
collections:
– ibm.power_aix
tasks:

– name: Cleanup any existing alt_disk_copy
alt_disk:
action: clean

– name: Create an alt_disk_copy for backup
alt_disk:
targets: hdisk1

– name: Stop Application
shell: /usr/local/bin/stop.sh

– name: Run AIX Update
nim:
action: update
lpp_source: “{{ nim_lpp_source }}”
targets: “{{ ansible_hostname }}”
delegate_to: “{{ nim_master }}”

– name: Reboot
reboot:
post_reboot_delay: 180

– name: Start Application
shell: /usr/local/bin/start.sh

Now I will run the playbook and patch the NIM client systems “bruce” and “freddie”:

$ ansible-playbook aix_patching.yml

TASK [Cleanup any existing alt_disk_copy] ****************************************************************************************************************************************************************************************************
changed: [bruce]
changed: [freddie]

TASK [Create an alt_disk_copy for backup] ****************************************************************************************************************************************************************************************************
changed: [bruce]
changed: [freddie]

TASK [Stop Application] **********************************************************************************************************************************************************************************************************************
changed: [bruce]
changed: [freddie]

TASK [Run AIX Update] *************************************************************************************************************************************************************************************************************************
changed: [bruce -> 10.0.0.5]
changed: [freddie -> 10.0.0.5]

TASK [Reboot] ********************************************************************************************************************************************************************************************************************************
changed: [freddie]
changed: [bruce]

TASK [Start Application] *********************************************************************************************************************************************************************************************************************
changed: [bruce]
changed: [freddie]

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
bruce : ok=7 changed=6 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
freddie : ok=7 changed=6 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Next, I will run the aix_oslevel_check.yml playbook again and see that the systems are all on AIX 7.2 TL5 SP2.

$ ansible-playbook aix_oslevel_check.yml

TASK [Print the oslevel] *********************************************************************************************************************************************************************************************************************
ok: [nim01] =>
msg: nim01 has the AIX oslevel of 7200-05-02-2114
ok: [bruce] =>
msg: bruce has the AIX oslevel of 7200-05-02-2114
ok: [freddie] =>
msg: freddie has the AIX oslevel of 7200-05-02-2114

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
nim01 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
bruce : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
freddie : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Conclusion

As you can see from this example, Ansible provides a lot of value in automating AIX operations. For additional information, see the documentation for the supported Collection available from Automation Hub. This Collection is also available to the community from Ansible Galaxy.