User-ID or EDL … why not both?

Original Photo by Jon Tyson

User-ID and External Dynamic Lists (EDL’s) are probably the most commonly used PAN-OS features to share external IP metadata with the NGFW. The Use cases for them are enormous: from blocking known DoS sources to forward traffic from specific IP addresses on low latency links.

Let’s take for instance the following logs generated by the SSH daemon in a linux host.

Won’t it be great to have a way to share the IP addresses used by the user xhoms with the NGFW so specific security policies are applied to traffic sourced by that address just because we now know the endpoint is being managed by that user?

The following Shell CGI script could be used to create an External Dynamic List (EDL) that would list all known IP addresses the account xhoms was used to login into this server.

That is great but:

  • When would addresses be removed from the list? At the daily log rollout?

To overcome these (and many other) limitations Palo Alto Networks NGFW feature a very powerful IP tagging API called User-ID. Although EDL and User-ID cover similar objectives there are fundamental technical differences between them:

  • User-ID is “asynchronous” (push mode), supports bot “set” and “delete” operations and provides a very flexible way to create groupings. Either by tagging address objects (Dynamic Address Group — DAG ) or by mapping users to addresses and then tagging these users (Dynamic User Group — DUG). User-ID allows for these tags to have a timeout in order for the PAN-OS device to take care of removing them at their due time.

Years ago it was up to the end customer to create its own connectors. Just like the CGI script we shared before. But as presence of PAN-OS powered NGFW’s grown among enterprise customers more application vendors decided to leverage the User-ID value by providing API clients out-of-the-box. Although this is great (for Palo Alto Network customers) losing the option to fetch a list from a URL (EDL mode) makes it more difficult to integrate legacy technologies that might be out there still in the network.

Creating EDL’s from User-ID enabled applications

Let’s assume you have an application that features a PAN-OS User-ID API client and that is capable of pushing log entries to the PAN-OS NGFW in an asynchronous way. Let’s assume, as well, that there are still some legacy network devices in your network that need to fetch that address metadata from a URL. How difficult it would be to create a micro-service for that?

One option would be to leverage a PAN-OS XML SDK like PAN Python or PAN GO and to create a REST API that extracts the current state from the PAN-OS device using its operations API.

GET https://<pan-os-device>/api
?key=<API-KEY>
&type=op
&cmd=<show><object><registered-ip><all></all></registered-ip></object></show>
HTTP/1.1 200 OK<response status="success">
<result>
<entry ip="10.10.10.11" from_agent="0" persistent="1">
<tag>
<member>tag10</member>
</tag>
</entry>
<entry ip="10.10.10.10" from_agent="0" persistent="1">
<tag>
<member>tag10</member>
</tag>
</entry>
<count>2</count>
</result>
</response>

It shouldn’t be that difficult to parse the response and provide a plain list out of it, right? Come back to me if you’re thinking on using a XSLT processor to pipe it at the output of a cURL command packing everything as a CGI script because we might have some common grounds ;-)

I’d love to propose a different approach though: to hijack User-ID messages by implementing a micro-service that behaves as a reverse proxy between the application that features the User-ID client and the PAN-OS device. I like this approach because it opens the door to other interesting use cases like enforcing timeout (adding timeout to entries that do not have it), converting DAG messages into DUG equivalents or adding additional tags based on the source application.

Components for a User-ID to EDL reverse proxy

The ingredients for such a receipt would be:

  1. a light http web server featuring routing (hijack messages sent to /api and let other requests pass through). GO http package and the type ReverseType fits like a glove in this case.

Fortunately for us there is the xhoms/panoslib GO module that covers 2 and 3 (OK, let’s be honest, I purpose-built the library to support this article)

With all these components it won’t be that difficult to reverse proxy a PAN-OS https service monitoring User-ID messages and exposing the state of the corresponding entries on a virtual endpoint named /edl that won’t conflict at all with the PAN-OS schema.

I wanted to prove the point and ended up coding the receipt as a micro-service. You can either check the code in the the GitHub repository xhoms/uidmonitor or run it from a Docker-enabled host. Take a look to the related panos.pan.dev tutorial for insights in the source code rationale

docker run --rm -p 8080:8080 -e TARGET=<pan-os-device> ghcr.io/xhoms/uidmonitor

Some payload examples

To get started you can check the following User-ID payload that registers the tag test to the IP addresses 10.10.10.10 and 10.10.20.20. Notice the different timeout values (100 vs 10 seconds)

POST http://127.0.0.1:8080/api/ HTTP/1.1
Content-Type: application/x-www-form-urlencoded
key=<my-api-key>
&type=user-id
&cmd=<uid-message>
<type>update</type>
<payload>
<register>
<entry ip="10.10.10.10">
<tag>
<member timeout="100">test</member>
</tag>
</entry>
<entry ip="10.10.20.20">
<tag>
<member timeout="10">test</member>
</tag>
</entry>
</register>
</payload>
</uid-message>

Just after pushing the payload (before the 10 second tag expires) perform the following GET request on the /edl endpoint and verify the provided list contains both IP addresses.

GET http://127.0.0.1:8080/edl/
?list=tag
&key=test
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Mon, 12 Apr 2021 10:07:50 GMT
Content-Length: 24
Connection: close
10.10.20.20
10.10.10.10

A few seconds (+10) later the same transaction should return only the IP address whose tag has not expired.

GET http://127.0.0.1:8080/edl/
?list=tag
&key=test
Content-Type: text/plain
Date: Mon, 12 Apr 2021 10:08:55 GMT
Content-Length: 12
Connection: close
10.10.10.10

Now let’s use a bit more complex payload. The following one register (login) two users mapped to IP addresses and then apply the user tag admin to both of them. As in the previous case, each tag has a different expiration value.

POST http://127.0.0.1:8080/api/ HTTP/1.1
Content-Type: application/x-www-form-urlencoded
key=<my-api-key>
&type=user-id
&cmd=<uid-message>
<type>update</type>
<payload>
<login>
<entry name="bar@test.local" ip="10.10.20.20" timeout="100"></entry>
<entry name="foo@test.local" ip="10.10.10.10" timeout="100"></entry>
</login>
<register-user>
<entry user="foo@test.local">
<tag>
<member timeout="100">admin</member>
</tag>
</entry>
<entry user="bar@test.local">
<tag>
<member timeout="10">admin</member>
</tag>
</entry>
</register-user>
</payload>
</uid-message>

Although the login transaction has a longer timeout on both cases, the group membership tag for the user “bar@test.local” is expected to timeout in 10 seconds.

Calling the /edl endpoint at specific times would demonstrate the first tag expiration.

GET http://127.0.0.1:8080/edl/
?list=group
&key=admin
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Mon, 12 Apr 2021 10:28:16 GMT
Content-Length: 24
Connection: close
10.10.20.20
10.10.10.10
...GET http://127.0.0.1:8080/edl/
?list=group
&key=admin

HTTP/1.1 200 OK
Content-Type: text/plain
Date: Mon, 12 Apr 2021 10:28:32 GMT
Content-Length: 12
Connection: close
10.10.10.10

Summary

Both EDL (because it provides “universal” coverage) and User-ID (because its advanced feature set) have their place in a large network. Implementing a reverse proxy between User-ID enabled applications and the managed PAN-OS device capable of hijacking User-ID messages have many different use cases: from basic User-ID to EDL conversion (as in the demo application) to advanced cases like applying policies to User-ID messages, enforcing maximum timeouts or transposing Address Groups into User Groups.

Principal Solutions Architect at @PaloAltoNtwks