Description

I wanted tooling to make changes to an ansible git repository but found this not as straightforward as I had hoped. This solution takes care of the annoying problem of loading yaml and python writing it back out of its original order and so making the git commits hard to read for a single line change.

Solution is to use the ruamel.yaml library which preserves ordering. First time you use this on a file it might make some unexpected minor changes to whitespace but thats a bonus!

For the git commits use gitpython

pip install ruamel.yaml GitPython

Sample

This example loads the yaml file and adds the command line argument ipaddr to the ip_address_list. Internally ip_address_list is a python array. While simple it shows the test_dictionary maintains order so the git commits are sane human readable changes.

import socket
import ruamel.yaml
import argparse
from pprint import pprint

'''
Author      : Paul Errington 
Description : insert new ip to list
Status      : Working
Requirement : developed with python3
'''

# parse args
parser = argparse.ArgumentParser(description='--ipaddr 0.0.0.0')
parser.add_argument('--ipaddr', required=True, help='--ipaddr 0.0.0.0')
parser.add_argument('--yaml', required=True, help='--yaml test.yml')
args = parser.parse_args()


# Validate IP
try:
    socket.inet_pton(socket.AF_INET, args.ipaddr)
except AttributeError:  # no inet_pton here, sorry
    try:
        socket.inet_aton(args.ipaddr)
    except socket.error:
        raise ValueError('not a valid ip address')
    print(args.ipaddr.count('.') == 3)
except socket.error:  # not a valid args.ipaddr
    raise ValueError('not a valid ip address')



# Read yaml file and add ip to list 
with open(args.yaml, 'r') as stream:
    try:
        ansible_yaml = ruamel.yaml.load(stream, ruamel.yaml.RoundTripLoader, preserve_quotes=True)
    except yaml.YAMLError as exc:
        print(exc)

    try:
        ansible_yaml['ip_address_list'].append(args.ipaddr)
    except:
        ansible_yaml['ip_address_list'] = [args.ipaddr]

    pprint(ansible_yaml)

# write out the new yaml here we preserve the original order
stream = open(args.yaml, 'w')

yaml = ruamel.yaml.YAML()
yaml.indent(mapping=2, sequence=4, offset=2)
yaml.explicit_start = True
yaml.dump(ansible_yaml, stream)

# git commit
from git import Repo
import os

repo_path = os.getcwd()

repo = Repo(repo_path)
index = repo.index

print(repo.working_tree_dir)
print(repo.working_tree_dir + "/" + args.yaml)
index.add([args.yaml])
index.commit("ADDED IP " + args.ipaddr)

Execute

$ python add_ip.py  --ipaddr 192.168.0.2 --yaml ansible/test.yml

Git log showing the results

$ git log -p 
commit a8e91c3807ed6308dbd0437951db02220fb35a73 (HEAD -> master)
Author: paul errington <[email protected]>
Date:   Sun Nov 3 13:06:01 2019 +0000

    ADDED IP 192.168.0.2

diff --git a/ansible/test.yml b/ansible/test.yml
index 01c7471..97224d3 100644
--- a/ansible/test.yml
+++ b/ansible/test.yml
@@ -8,3 +8,4 @@ test_dictionary:
 
 ip_address_list:
   - 192.168.0.1
+  - 192.168.0.2

commit 4947a951ba89ac36ba8880526388df238b8de4e2
Author: paul errington <[email protected]>
Date:   Sun Nov 3 13:05:58 2019 +0000

    ADDED IP 192.168.0.1

diff --git a/ansible/test.yml b/ansible/test.yml
index a1cb045..01c7471 100644
--- a/ansible/test.yml
+++ b/ansible/test.yml
@@ -7,3 +7,4 @@ test_dictionary:
   key3: value3
 
 ip_address_list:
+  - 192.168.0.1

commit ae41c032d2dfce632ba79f0f7c5608fbbd2ebd3b
Author: paul errington <[email protected]>
Date:   Sun Nov 3 13:05:31 2019 +0000

    YAML

diff --git a/ansible/test.yml b/ansible/test.yml
new file mode 100644
index 0000000..a1cb045
--- /dev/null
+++ b/ansible/test.yml
@@ -0,0 +1,9 @@
+---
+
+test_dictionary:
+  description: we want to preserve order!
+  key1: value1
+  key2: value2
+  key3: value3
+
+ip_address_list:

Conclusion

This works well! For any production use just need to make sure to pull latest before making changes and finally push to origin. The script itself can be ran through a web interface as part of a multistage job. Next stage would be to actually run the playbook against the commit.