User-ID and External Dynamic Lists (EDL’s) are probably the most commonly used PAN-OS features to share external IP metadata with the NGFW. The Use cases for them are enormous: from blocking known DoS sources to forward traffic from specific IP addresses on low latency links.
Let’s take for instance the following logs generated by the SSH daemon in a linux host.
Won’t it be great to have a way to share the IP addresses used by the user xhoms with the NGFW so specific security policies are applied to traffic sourced by that address just because we now know the endpoint is being managed by that user?
The following Shell CGI script could be used to create an External Dynamic List (EDL) that would list all known IP addresses the account xhoms was used to login into this server.
That is great but:
- When would addresses be removed from the list? At the daily log rollout?
- Are all listed IP addresses still being operated by the user xhoms?
- Is there any way to remove addresses from the list at user logout?
To overcome these (and many other) limitations Palo Alto Networks NGFW feature a very powerful IP tagging API called User-ID. Although EDL and User-ID cover similar objectives there are fundamental technical differences between them:
- User-ID is “asynchronous” (push mode), supports bot “set” and “delete” operations and provides a very flexible way to create groupings. Either by tagging address objects (Dynamic Address Group — DAG ) or by mapping users to addresses and then tagging these users (Dynamic User Group — DUG). User-ID allows for these tags to have a timeout in order for the PAN-OS device to take care of removing them at their due time.
- EDL is “universal”. Almost all network appliance vendors provide a feature to fetch an IP address list from a URL.
Years ago it was up to the end customer to create its own connectors. Just like the CGI script we shared before. But as presence of PAN-OS powered NGFW’s grown among enterprise customers more application vendors decided to leverage the User-ID value by providing API clients out-of-the-box. Although this is great (for Palo Alto Network customers) losing the option to fetch a list from a URL (EDL mode) makes it more difficult to integrate legacy technologies that might be out there still in the network.
Creating EDL’s from User-ID enabled applications
Let’s assume you have an application that features a PAN-OS User-ID API client and that is capable of pushing log entries to the PAN-OS NGFW in an asynchronous way. Let’s assume, as well, that there are still some legacy network devices in your network that need to fetch that address metadata from a URL. How difficult it would be to create a micro-service for that?
&cmd=<show><object><registered-ip><all></all></registered-ip></object></show>HTTP/1.1 200 OK<response status="success">
<entry ip="10.10.10.11" from_agent="0" persistent="1">
<entry ip="10.10.10.10" from_agent="0" persistent="1">
It shouldn’t be that difficult to parse the response and provide a plain list out of it, right? Come back to me if you’re thinking on using a XSLT processor to pipe it at the output of a cURL command packing everything as a CGI script because we might have some common grounds ;-)
I’d love to propose a different approach though: to hijack User-ID messages by implementing a micro-service that behaves as a reverse proxy between the application that features the User-ID client and the PAN-OS device. I like this approach because it opens the door to other interesting use cases like enforcing timeout (adding timeout to entries that do not have it), converting DAG messages into DUG equivalents or adding additional tags based on the source application.
Components for a User-ID to EDL reverse proxy
The ingredients for such a receipt would be:
- a light http web server featuring routing (hijack messages sent to
/apiand let other requests pass through). GO http package and the type ReverseType fits like a glove in this case.
- a collection of XML-enabled GO structs to unmarshal captured User-ID messages.
- a library featuring User-ID message payload processing capable of keeping a valid (non-expired) state of User-to-IP, Group-to-IP and Tag-to-IP maps.
Fortunately for us there is the xhoms/panoslib GO module that covers 2 and 3 (OK, let’s be honest, I purpose-built the library to support this article)
With all these components it won’t be that difficult to reverse proxy a PAN-OS https service monitoring User-ID messages and exposing the state of the corresponding entries on a virtual endpoint named
/edl that won’t conflict at all with the PAN-OS schema.
I wanted to prove the point and ended up coding the receipt as a micro-service. You can either check the code in the the GitHub repository xhoms/uidmonitor or run it from a Docker-enabled host. Take a look to the related panos.pan.dev tutorial for insights in the source code rationale
docker run --rm -p 8080:8080 -e TARGET=<pan-os-device> ghcr.io/xhoms/uidmonitor
Some payload examples
To get started you can check the following User-ID payload that registers the tag
test to the IP addresses
10.10.20.20. Notice the different timeout values (100 vs 10 seconds)
POST http://127.0.0.1:8080/api/ HTTP/1.1
Just after pushing the payload (before the 10 second tag expires) perform the following GET request on the
/edl endpoint and verify the provided list contains both IP addresses.
&key=testHTTP/1.1 200 OK
Date: Mon, 12 Apr 2021 10:07:50 GMT
A few seconds (+10) later the same transaction should return only the IP address whose tag has not expired.
Date: Mon, 12 Apr 2021 10:08:55 GMT
Now let’s use a bit more complex payload. The following one register (login) two users mapped to IP addresses and then apply the user tag
admin to both of them. As in the previous case, each tag has a different expiration value.
POST http://127.0.0.1:8080/api/ HTTP/1.1
<entry name="firstname.lastname@example.org" ip="10.10.20.20" timeout="100"></entry>
<entry name="email@example.com" ip="10.10.10.10" timeout="100"></entry>
Although the login transaction has a longer timeout on both cases, the group membership tag for the user “firstname.lastname@example.org” is expected to timeout in 10 seconds.
/edl endpoint at specific times would demonstrate the first tag expiration.
&key=adminHTTP/1.1 200 OK
Date: Mon, 12 Apr 2021 10:28:16 GMT
HTTP/1.1 200 OK
Date: Mon, 12 Apr 2021 10:28:32 GMT
Both EDL (because it provides “universal” coverage) and User-ID (because its advanced feature set) have their place in a large network. Implementing a reverse proxy between User-ID enabled applications and the managed PAN-OS device capable of hijacking User-ID messages have many different use cases: from basic User-ID to EDL conversion (as in the demo application) to advanced cases like applying policies to User-ID messages, enforcing maximum timeouts or transposing Address Groups into User Groups.