Parsing Configuration Files With Augeas on Osquery

Whilst Osquery provides a huge amount of flexibility for retrieving lots of different pieces of information about a system. Sometimes however, there may be a specific part of the system you’re after that isn’t covered by Osquery’s included tables.

Augeas configuration parsing within Osquery

Within Osquery there’s a table called augeas it’s super helpful for parsing configuration files across Mac OSX and Linux. Important for tasks like compliance, auditing and checking the configuration of deployed services.

Augeas parses configuration files in their native formats and transforms them into a tree. This tree is then translated into a query-able table by Osquery.

If you’re unfamiliar with Osquery or don’t already have it installed checkout our guide here.

How Augeas works within Osquery

Augeas works by parsing different types of configuration files. However, this inevitably ends up becoming a headache due to the vast array of different configuration formats, structures and the syntax used to define these configurations for different services. This is where the concept of lenses come in.

Augeas Lenses

Lenses are essentially a configuration file for the Augeas engine to let it know how to handle different file formats and structures. As well as how to transform common attributes within those files.

Table results

The parsed tokens are returned as a large list which Osquery then transforms into a table with the following columns:

• node
  This is path with the document a parasble entity was found as described by the label
• label
  The name of the parsed node. Within a configuration file this will normaly be the key of the associated value.
• value
  The value of the parsed entity.
• path
  This is used during the WHERE clause of the SELECT statement to specify the file to be parsed. It is also returned as part of the query to show you the file that was parsed succesffuly if a wildcard are provided in said WHERE clause.

Using osqueryi

Augeas lenses are bundled automatically with Osquery (though its worth downloading the latest configuration files). On Linux they’re installed in /usr/share/osquery/lenses. On Mac OSX, lenses are installed in the /private/var/osquery/lenses directory. When running osqueryi you don’t need to specify the path to the lenses directory. Unless you’ve installed the Augeas lenses in a different directory.

osqueryi --augeas_lenses=/usr/share/osquery/lenses -v

Once you’ve got the Osquery shell we can start parsing files.

Supported files

You can see whats supported by checking to see what configuration files lie within the configuration folder:

ls -l /usr/share/osquery/lenses

You can then see how each lense works by opening it within a text editor.

Examples

I’m going to provide a few examples on Linux though most of these will be interoperable on Mac OSX as well.

Some of these queries also have dedicated tables within Osquery. However, these are just to give you an idea of how to use the augeas table. In most cases you’ll use Augeas to parse application specific configurations such as Nginx, Httpd, PostgreSQL, Mysql etc.

Using Augeas to check PermitRootLogin for SSH

For security of the system we can use Augeas to parse the systems sshd_config to ensure that the PermitRootLogin parameter is set to no. AS we don’t want to allow login of the root user via SSH.

osquery> SELECT label, value FROM augeas WHERE path = '/etc/ssh/sshd_config' AND label = 'PermitRootLogin';
+-----------------+-------+
| label           | value |
+-----------------+-------+
| PermitRootLogin | yes   |
+-----------------+-------+

You can also convert this query into a test to check for compliance on your hosts.

osquery> SELECT CASE WHEN COUNT(*) = 1 THEN 
  'PASSING' ELSE 'FAILING' END AS state 
FROM augeas WHERE path = '/etc/ssh/sshd_config' 
AND label = 'PermitRootLogin' 
AND value != 'yes';
+---------+
| test    |
+---------+
| FAILING |
+---------+

System DNS information (nameservers)

Whilst osquery has a table for this query. We can perform a simple one line query to check the value for a given label.

osquery> SELECT value FROM augeas WHERE path = '/etc/resolv.conf' AND label = 'nameserver';
+-------------+
| value       |
+-------------+
| 192.168.1.1 |
+-------------+

This can also be achieved with the Osquery table dns_resolvers .

osquery> SELECT * FROM dns_resolvers;
+----+------------+-------------+---------+---------+
| id | type       | address     | netmask | options |
+----+------------+-------------+---------+---------+
| 0  | nameserver | 192.168.1.1 | 32      | 524993  |
+----+------------+-------------+---------+---------+

Configured web server domains (httpd)

This query is a little more interesting as it first requires we perform a sub query in order to first identify the ServerName nodes — that form part of the document. Once we have all of those nodes we can then use that information to find the arg value of those nodes. Which will contain our domain names.

osquery> SELECT value FROM augeas WHERE node LIKE (
  SELECT node FROM augeas 
  WHERE path = '/etc/httpd/conf.d/example.conf' 
  AND value = 'ServerName'
) || '%' 
AND path = '/etc/httpd/conf.d/example.conf' 
AND label = 'arg';
+-------------+
| value       |
+-------------+
| example.com |
+-------------+

Parsing the sudoers file (/etc/suoders)

Whilst sudoers support is directly supported by Osquery it’s still a good example. As most of the results you’ll get from Augeas are hundreds of rows in length as each parsed token gets its own row. Which you’ll need transform into something more meaningful.

osquery> SELECT * FROM augeas WHERE path = '/etc/sudoers';
+-----------------------------+--------+------------+--------------+
| node                        | value  | label      | path         |
+-----------------------------+--------+------------+--------------+
| ..spec[1]/user              | root   | user       | /etc/sudoers |
| ..spec[1]/host_group        |        | host_group | /etc/sudoers |
| ..spec[1]/host../host       | ALL    | host       | /etc/sudoers |
| ..spec[1]/host../command    | ALL    | command    | /etc/sudoers |
| ..spec[1]/host../runas_user | ALL    | runas_user | /etc/sudoers |
.. (50+ more rows)

The node key will provide a helpful index as to the hierarchy that’s been successfully parsed. Using the SQLite engine we can group on these node at a certain point and translate the row data into a single or collection of columns using SQLite’s in-built transforms. In this example I’ve use the json_group_object transform to take the label and value for our data components in order to represent them as a JSON object. Which Osquery supports.

osquery> SELECT json_group_object(label, value) AS result 
FROM (
  SELECT *, SUBSTR(node, 0, INSTR(node, ']')+1) AS 'group' 
  FROM augeas WHERE path = '/etc/sudoers' 
  AND node LIKE '%/spec[%'
) AS rows GROUP BY rows."group";
+------------------------------------------------------------------+
| result                                                           |
+------------------------------------------------------------------+
| {"user":"root","host":"ALL","command":"ALL","runas_user":"ALL"}  |
| {"user":"%wheel","host":"ALL","command":"ALL","runas_user":"ALL"}|
+------------------------------------------------------------------+

You can also use the group_concat method to join columns whilst using an aggregate function. Using an = to seperate the keys values and a comma to seperate the key pairs.

osquery> SELECT group_concat(label || '=', value || ',') AS result FROM (
  SELECT *, SUBSTR(node, 0, INSTR(node, ']')+1) AS 'group' 
  FROM augeas WHERE path = '/etc/sudoers' 
  AND node LIKE '%/spec[%'
) AS rows GROUP BY rows."group";
+------------------------------------------------------------------+
| result                                                           |
+------------------------------------------------------------------+
| spec=root,user=,host_group=ALL,host=ALL,command=ALL,runas_user=  |
| spec=%wheel,user=,host_group=ALL,host=ALL,command=ALL,runas_user=|
+------------------------------------------------------------------+

However, the same query can be made directly in Osquery using the sudoers table with next to little effort :). Though you don’t get the breakdown of the rules details. Nor the rest of the configuration file.

osquery> SELECT * FROM sudoers WHERE header IN (
  SELECT username FROM users
);
+--------------+--------+----------------+
| source       | header | rule_details   |
+--------------+--------+----------------+
| /etc/sudoers | root   | ALL=(ALL)  ALL |
+--------------+--------+----------------+

Troubleshooting

There are a few things to note when using the augeas table within Osquery. Firstly, as of 4.5.1 you cannot specify the parsing template (lens) you wish to use. This is done automatically for you based on the default locations files are typically found in. Which means that for some Augeas configuration files they will not work. There is currently an open ticket in Osquery for this.

You can check the supported paths by looking for the filter or view filter within an Augeas configuration. They’re typically down at the bottom of the document and look like this:

(* View: filter *)
let filter = (incl "/etc/sudoers")
    . (incl "/usr/local/etc/sudoers")
    . (incl "/etc/sudoers.d/*")
    . (incl "/usr/local/etc/sudoers.d/*")
    . (incl "/opt/csw/etc/sudoers")
    . (incl "/etc/opt/csw/sudoers")
    . Util.stdexcl

No results returned

This is usually for one of 3 reasons.

• The file provided within the path is incorrect or isn’t supported as part of the filters within Augeas. Please see the comment above not being able to manually specify lenes.
• Augeas failed to parse the file successfully. For more complex files this can happen where the parser just simply cannot process the file. You can test this by removing parts of the file. You can also download the latest lens files which may account for a newer configuration parameter. Which may not be supported in the one you’re currently using.
• Augeas may have failed to initalise. You can check for startup errors within Osquery using the --verbose flag. It may also be that the specified folder for lenes does not exist. Which you can specify manually with:--augeas_lenses .

Invalid path expression

This is because the path you’ve provided in your query isn’t valid. You can use the LIKE verb within WHERE statements with the % wildcards. However, I would try and stick with full qualified paths where possible. Also check for non-printable characters that may have slipped in from a copy and paste.

augeas.cpp:46] An error has occurred while trying to query augeas: Invalid path expression

The /files prefix

In a lot of Augeas documentation you’ll notice the /files prefix is applied to the path of file you’re tyring to parse. Osquery pre-appens this automatically. So don’t add it.

Its all over!

Hopefully that’s given you a quick dive into how to use Augeas with Osquery. We’ll be following up on this topic in the near future. As well as how Zercurity uses Augeas at scale in production. However, that’s all for now. Please feel free to get in touch if you have any questions.