Mounting data onto your filesystem for fun and unfortunately no profit

A little more than a year ago I was working for a client that desired to have a simple way do do an inventory of their linux servers (running SLES). They had their networking DTAP configured to make every environment only available via a bastion host and only that host.
Luckily you can do a lot with Ansible in conjunction with ProxyCommands (see https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Proxies_and_Jump_Hosts) so reaching the servers was not really a problem and since Ansible's excellent setup module provides a wealth of information, I had all the ingredients I needed, I just needed to connect the dots.

Since I wanted to learn more about filesystems (and FUSE in particular) I thought it would be a nice exercise to try and "map" the collected data from Ansible's setup module onto a mountpoint to make it easy to grep and parse and all that jazz. Since the data format Ansible uses is JSON anyway, I thought it would be best to first focus on creating a script that can mount an arbitrary datastructure in JSON format, which is exactly what I did:

https://github.com/yhekma/datamounter

Using this script, you can mount your JSON files like so:

datamounter.py -c my_file.json /opt/infra_map

After which the contents of the json files are mounted.
So, when you have the following dictionary/json:

{'key1': {'key2': 'data1',
  'key3': ['something', 'something_else', 'and_another_thing']},
 'key4': 'bla'}

it gets mapped as:

mountpoint/
mountpoint/key4
mountpoint/key1
mountpoint/key1/key2
mountpoint/key1/key3
mountpoint/key1/key3/listitem_0
mountpoint/key1/key3/listitem_2
mountpoint/key1/key3/listitem_1

Where the contents of mountpoint/key1/key2 is "data1" and the content of mountpoint/key1/key3/listitem_0 is "something".

In a followup post I will describe the realtime mounting process in which we can configure the filesystem so that the values like free swap space or load get collected the moment you open the file for reading.

Comments

Post a Comment

Popular posts from this blog

Runing Saltstack in multi-master

Recently I ran into some problems when trying to use multiple saltmasters in combination with using the mine and thought I'd share my experiences. Background   Like puppet   The salt mine can be compared to the PuppetDB ; it's a place to store stuff like (custom) facts for use on other nodes/instances. The classic use-case for this is a monitoring setup like Nagios that need to be configured for each additional deployed service/instance with the particular info of that service/instance. So let's say you deploy a new MySQL instance and automatically slave it to an existing cluster, you want the ip address, database name and maybe some other stuff to be configured on the monitoring server. Puppet does this by something called " exported resources " which basically sends data to a central location (the PuppetDB) which can then be collected by other machines when puppet runs there. But different   Where Puppet uses a database (PostgreSQL by default)
Read more

Recursively update Composite Content Views in Satellite 6/Katello

The basic idea Satellite 6 (and Katello for that matter) have a new way of dealing with content, whether it being puppet modules, rpm's or docker images. Below I will focus on rpm's, which I think will be the use case for most people. A content view can contain one or more rpm repository's at a specific point in time but can consist of multiple versions. So let's say I create a content view named RHEL7_BASE on Monday containing 2 repositories I just synced:  rhel7_server and rhel7_epel.  Version 1 of that content view points to the packages as they are on that Monday. Now the next Friday I do a sync of my repositories so I get the latest versions and patches and whatnot, but note that version 1 of my RHEL7_BASE content view is unchanged, and any servers that are using this version will not have access to the new packages. In order to make these new packages available, I need to publish a new version of the view and promote this version to the environment that con
Read more

First!

Well, I finally bit the bullet and started a blog. In here I will try to share some problems and solutions I encountered/devised in my daily dealings with mainly Linux as a DevOps Linux Engineer.
Read more