How to install syslog-ng on macOS
The complete and painless guide to making syslog-ng work on Mac
Overview
My goal was to send any kind of Mac logs to a big data platform (Devo). Mac already comes with syslogd, which is the Apple System Log server. It is basically a daemon that processes syslog messages but to be honest, it’s pretty old and basic. As a result, I decided to try syslog-ng.
To my surprise, the syslog-ng documentation discloses:
At present we are not supporting macOS syslog-ng on our official repository on GitHub. However, you can compile syslog-ng yourself following this guide.
That’s what I did. I compiled syslog-ng from source following the official documentation. But I encountered different obstacles that they didn’t address. In the end, I managed to make syslog-ng work successfully on my Mac. If you follow this guide, you will too.
Environment
- OS: macOS Catalina 10.15.7
- syslog-ng: 3.29.1
- Homebrew: 2.5.2
Installing Dependencies
I’m assuming you already have Homebrew installed and updated. If so, install the following packages:
- automake
- autoconf
- binutils
- glib
- autoconf-archive
- flex
- bison
- libtool
- pkg-config
- ivykis
- openssl
- pcre
$ brew install automake autoconf binutils glib autoconf-archive \
flex bison libtool pkg-config ivykis openssl pcreSetting Up Environment Variables
Apple provides an outdated bison package (2.3). We need to force our Mac to use the new one we just installed (3.7.2) by adding it to our PATH. Also, we require to extend the search path of pkg-config. Add these 2 lines to your .bash_profile:
export PATH=/usr/local/opt/bison/bin:$PATHexport PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/opt/openssl/lib/pkgconfig
As usual, to apply those changes, open a new terminal or just reload your profile by running: . ~/.bash_profile
Cloning GitHub repository
To get the latest version, we can clone it from the official GitHub repository:
$ git clone https://github.com/syslog-ng/syslog-ng.gitConfiguring
Because we cloned the repo, run this command to generate a configuration script:
$ ./autogen.shNow we could just run the configure script with core features, but the compilation would fail. You have to disable java support. These are the options I used:
$ ./configure --with-ivykis=system --disable-javaCompile and Install
$ make -j4
$ make installThis is the output from /usr/local/sbin/syslog-ng -V after installation:
syslog-ng 3 (3.29.1.107.g1040f91)
Config version: 3.29
Installer-Version: 3.29.1.107.g1040f91
Revision:
Compile-Date: Oct 4 2020 00:48:18
Module-Directory: /usr/local/lib/syslog-ng
Module-Path: /usr/local/lib/syslog-ng
Include-Path: /usr/local/share/syslog-ng/include
Available-Modules: timestamp,kvformat,appmodel,afprog,examples,cef,map-value-pairs,afsnmp,stardate,system-source,confgen,afuser,xml,disk-buffer,tfgetent,linux-kmsg-format,dbparser,json-plugin,add-contextual-data,pseudofile,affile,csvparser,basicfuncs,syslogformat,hook-commands,graphite,tags-parser,afstomp,http,secure-logging,mod-python,afsocket,cryptofuncs,azure-auth-header
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: off
Enable-TCP-Wrapper: off
Enable-Linux-Caps: off
Enable-Systemd: offShould we run it?
If you try to start the service with the official command: /usr/local/sbin/syslog-ng -F it won’t start due to this error:
syslog-ng: Error setting file number limit; limit='0'; error='Invalid argument'
[2020-10-04T00:48:20.072382] system(): Error detecting platform, unable to define the system() source. Please send your system information to the developers!; sysname='Darwin', release='19.6.0'The reason is that system() is not designed for macOS. At least not for now. So remove or comment that option in the default config file /usr/local/etc/syslog-ng.conf. Additionally, you can update the destination file, since Mac uses system.log instead of messages:
source s_local {
internal();
};destination d_local {
file("/var/log/system.log");
};log {
source(s_local);
destination(d_local);
};
I thought with these changes it’d finally start, but it did not. Therefore, I decided to investigate how the service was started in other systems, such as Linux. What I found is that it requires a PID. So I replicated that command and it worked! We have a winner:
$ sudo /usr/local/sbin/syslog-ng -F -p /var/run/syslog-ng.pidIt needs sudo because of /var/run/ permissions. Now if you display the processes with ps -ef | grep syslog-ng | grep -v grep you should see a couple:
0 37004 29803 0 12:17PM ttys000 0:00.06 sudo /usr/local/sbin/syslog-ng -F -p /var/run/syslog-ng.pid
0 37005 37004 0 12:17PM ttys000 0:00.07 /usr/local/sbin/syslog-ng -F -p /var/run/syslog-ng.pidThe process 37004 is the sudo command running in the foreground and the 37005 is the actual syslog-ng service.
Testing
The default config file comes with internal() as a source. This function sends syslog-ng stats every 10 minutes. As you saw in the ps output, I started the service at 12:17. Thus, system.log shows this line on time:
Oct 4 12:27:42 macbook syslog-ng[37005]: Log statistics; processed='src.internal(s_local#0)=1', stamp='src.internal(s_local#0)=1583385462', processed='global(internal_queue_length)=0', queued='global(scratch_buffers_count)=0', queued='global(scratch_buffers_bytes)=0', processed='global(msg_clones)=0', processed='center(received)=1', processed='center(queued)=1', processed='global(sdata_updates)=0', processed='destination(d_local)=1', processed='source(s_local)=1', processed='global(payload_reallocs)=0'Finally, syslog-ng is sending internal stats successfully to our Mac system log!
Foreground vs Background
Now, syslog-ng is running in the foreground. What if we close the terminal where we ran the sudo command? The service will shut down. To avoid this, you could try running it in the background:
$ sudo /usr/local/sbin/syslog-ng --process-mode=background -p /var/run/syslog-ng.pidCreating a Job Definition
At this point, syslog-ng is working fine, but what if we want to go one step further and make it behave like an Apple service? That’s where launchd comes into play. This is the service management framework in macOS. It handles agents and daemons. For our use case, we require a daemon because we want the service to be running in the background upon system start without user input. So let’s create a job definition.
Following standards, I created a file calledcom.syslog-ng.syslog-ng.plist. To be treated as a global daemon it has to be stored in /Library/LaunchDaemons/. This is the XML content:
Note: You can give any name to your job. However, just make sure the name matches the
Labelkey.
I enabled RunAtLoad to make launchd start the job as soon as it has been loaded. Also, having KeepAlive enabled will make the job run no matter what.
Run and Enjoy
With this one command our job will load and run:
$ sudo launchctl load /Library/LaunchDaemons/com.syslog-ng.syslog-ng.plistFor validation, execute this command to verify if the service is running:
$ sudo launchctl list | grep syslog-ngThe output displays three columns: PID, the last exit status, and the job’s name:
39313 0 com.syslog-ng.syslog-ngIf you have a similar output and you see syslog-ng stats coming into system.log… Congratulations!
Restarting the Job
launchd does not support reload or restart commands, so if you edit your file and need to apply changes, first stop the job and then start it back. To stop the job run:
$ sudo launchctl unload /Library/LaunchDaemons/com.syslog-ng.syslog-ng.plistWrapping Up
I created this guide to share my syslog-ng journey with all the Mac users out there. I hope you find it useful and time-saving. Let me know your experience with syslog-ng on Mac in the comments below, or if you have any problems, don’t hesitate to contact me. Happy logging!
