Remote log forwarding

IAM appliance integrates with SIEM software by forwarding its log via syslog protocol. Everything in IAM appliance goes to the rsyslog daemon which then distributes logs into separate logfiles.

This howto will show you how to enhance rsyslog configuration to also send logs over the network into your SIEM solution. You will configure simple UDP-based log forwarding there, but in reality you can use any configuration that rsyslog can handle.

UDP-based syslog is inherently insecure because it transfers logs over the network in plaintext. It should be used only within dedicated (or otherwise secured) network segments.

Another option is to use TCP-based syslog with TLS but then you have to configure message queue on the sender (on the IAM appliance). Otherwise, when the SIEM server becomes unreachable, even processes on the sending machine come to halt. Number of messages going through the syslog may be quite high and we recommend you perform sizing analysis beforehand.

Understanding rsyslog configuration

All major components of the IAM appliance use rsyslog for logging, namely:

  • The operating system itself.

    • This logging is the same as on any other ordinary Linux machine.

  • All containerized services.

    • Logging through docker to rsyslog. There is specific configuration in the rsyslog which separates logs into per-service logfiles.

Rsyslog configuration of IAM appliance services is stored in multiple files in the /etc/rsyslog.d/ directory:

[root@localhost rsyslog.d]# ll
total 28
-rw-r--r--. 1 root root 144 Jan 20  2021 01_enable_udp_listen.conf
-rw-r--r--. 1 root root 176 Sep  6  2021 10_cas.conf
-rw-r--r--. 1 root root 190 Sep  6  2021 10_czechidm.conf
-rw-r--r--. 1 root root 199 Sep  6  2021 10_czechidm-db.conf
-rw-r--r--. 1 root root 214 Sep  6  2021 10_directory-server.conf
-rw-r--r--. 1 root root 193 Sep  6  2021 10_web-proxy.conf

If you look, for example, into CAS logging configuration

[root@localhost rsyslog.d]# cat 10_cas.conf
# This configuration is governed by RPM package. All manual changes will be lost.
:programname, isequal, "cas" {
	action (type="omfile" file="/data/logs/cas/cas.log")
	stop
}

you can see that whenever program name equals cas, logs are directed to the logfile and then the processing of the message stops.

Configuring log forwarding

If you want to, for example, configure CAS logs to also go to the SIEM, you create new configuration 09_cas_fwd.conf with following contents:

[root@localhost rsyslog.d]# cat 09_cas_fwd.conf
:programname, isequal, "cas" {
	action (type="omfwd" protocol="udp" target="10.0.1.247" port="514")
}

Key points here are:

  1. This configuration is placed in 09_cas_fwd.conf which goes before 10_cas.conf. If it was loaded after 10_cas.conf, it would not work because the 10_cas.conf contains a stop clause.

  2. The configuration in 09_cas_fwd.conf does not contain a stop clause. Logs are sent over the network according to 09_cas_fwd.conf but also further processed which also directs them into the local logfile as is specified in the 10_cas.conf.

You can modify the log forwarding configuration completely to your liking but you must not edit configuration files that come with the IAM appliance.

After you configure the rsyslog, restart it for changes to take effect.

[root@localhost ~]# systemctl restart rsyslog

Testing

Rsyslog now starts to send logs over the network. If you configured standard UDP on port 514, you can easily test with running nc on the target machine. Example of forwarded logs (newlines added for clarity).

[root@siem ~]# nc -u -l 514
<30>Jul 20 15:43:12 localhost.localdomain cas[1017]: #033[m#033[32m2022-07-20 15:43:12,399 INFO [org.apereo.cas.ticket.registry.DefaultTicketRegistryCleaner] - <[0] expired tickets removed.>
<30>Jul 20 15:43:14 localhost.localdomain cas[1017]: #033[m#033[33m2022-07-20 15:43:14,615 WARN [org.apereo.cas.services.RegisteredServiceAccessStrategyUtils] - <Unauthorized Service Access. Service [applianceStatusConnectionTest] is not found in service registry.>
<30>Jul 20 15:43:24 localhost.localdomain cas[1017]: #033[m#033[33m2022-07-20 15:43:24,640 WARN [org.apereo.cas.services.RegisteredServiceAccessStrategyUtils] - <Unauthorized Service Access. Service [applianceStatusConnectionTest] is not found in service registry.>