Audit Trail

DSS includes an audit trail that logs all actions performed by the users, with details about user id, timestamp, IP address, authentication method, …

Viewing the audit trail in DSS

You can view the latest audit events directly in the DSS UI: Administration > Security > Audit trail.

Note that this live view only includes the last 1000 events logged by DSS, and it is reset each time DSS is restarted. You should use log files or external systems for real auditing purposes.

Audit trail log files

The audit trail is logged in the run/audit folder of the DSS data directory.

This folder is made of several log files, rotated automatically. Each file is rotated when it reaches 10 MB, and up to 10 history files are kept

Auditing to external systems

In order to provide non-repudiation characteristics, it is critical that the audit log files are sent outside of the DSS machine, on a remote system that cannot be accessed and tampered with from the DSS machine.

The audit system in DSS sends data using the log4j library. Therefore, all log4j appenders can be used to get audit out of the DSS machine. Configuring log4j is done by editing the resources/logging/dku-log4j.properties file. (See Logging in DSS for more information)

The following section includes examples of how to send audit to some external systems

Sending audit trail to Kafka

Apache Kafka is a distributed message queue, which can be used to get audit logs out of DSS

To enable audit queries to Kafka:

  • Add all jars from the Kafka distribution to the lib/java folder of your DSS datadir
  • Create or edit the resources/logging/dku-log4j.properties file (See Logging in DSS for more information)
  • Add the following lines
log4j.appender.AUDIT_KAFKA=org.apache.kafka.log4jappender.KafkaLog4jAppender
log4j.appender.AUDIT_KAFKA.BrokerList=kafka1:9092,kafk2:9092,kafka:9093
log4j.appender.AUDIT_KAFKA.Topic=dku-dss-audit
log4j.appender.AUDIT_KAFKA.layout=com.dataiku.dip.logging.JSONAuditLayout

log4j.logger.dku.audit= INFO, AUDIT_KAFKA
  • Replace dku-dss-audit by the name of your Kafka topic
  • Replace BrokerList by the bootstrap list of your Kafka brokers

Note

Your Kafka libraries must correspond to the version of your Kafka server