The full article was first available on The New Stack - read it here.
Audit logs are immutable records that describe a system’s changes over time. Each event and attempt that occurs should be captured in a log stating what happened, the time it occurred and the users that were involved.
Effective audit logging creates an audit trail that sequentially documents your system’s activity. This information is invaluable for compliance, debugging and investigation of security breaches.
This article will teach you what audit logs are and explain why they’re essential for production environments.
Audit logs are centrally stored records of the events that affect a system’s state or behavior. Each record is a historical footprint of a single change. It logs the event and contextual information about how it was initiated. Logs can also incorporate information about why the event occurred.
The following data fields are usually tracked as part of an audit log event:
An audit log entry for a user authorization check could look similar to the following:
An administrator who’s concerned about a spike in authorization failures could consult the audit log as part of their investigation. They’d see that Demo User tried to use a feature that required the CreateNewUser role, but the system denied their access. Without the audit log, there would be no record of this potentially problematic request.
Audit logs have a different purpose and intended audience when compared to the system logs written by your application’s code. Whereas those logs are usually designed to help developers debug unexpected technical errors, audit logs are primarily a compliance control for monitoring your system’s operation. They’re helpful to regulatory teams, system administrators and security practitioners who need to check that correct processes are being followed.
Audit logs also differ in the way they’re stored and retained. They’re usually stored for much longer periods than application logs, which are unlikely to be kept after an issue is solved. Because audit logs are a historical record, they could be retained indefinitely if you have the storage available. This enables you to investigate incidents that only come to light years after the events occurred.
The information within an audit log typically falls into one of three main categories:
These examples show how audit events directly relate to your system’s business functions. Anyone involved with your project should be able to understand the data in its audit trail. This accessibility contrasts with application logs that are often only useful to developers. As they focus on errors within the code, app logs tend to be technical, verbose and low-level.
Audit logs are a key tool for monitoring and investigating authentication and authorization patterns. Distinguishing between these terms is important so you know what each log includes and when to access them:
The information provided by an audit log entry will vary depending on whether the record relates to authentication or authorization. If it’s an authentication attempt, the request will have occurred in the context of a public session. Authorization logs, written by AuthZ services like Cerbos, will include the identity of the logged-in user.
Assessing both types of logs will expose the widest range of attacks. Brute force credential-guessing attempts will show up as failed authentications in the audit trail. Meanwhile, a spike in denied authorizations for a user could indicate that their API token has been intercepted by an attacker.
Audit logs should be written by your application each time a significant event occurs. At its simplest, an audit log is simply a file or database table where a new record is added as each event occurs. To generate an audit event, capture the information that’s relevant to your system, then label it with the type of event that’s occurred.
Audit logs can be stored in several different formats:
You’ve also got options for where audit events are sent to be indexed and consumed. Manually interacting with any of the methods above is clunky and error-prone. Setting up a dedicated log viewer provides convenient access to your records:
Choose a processing pipeline that’s resilient enough to handle audit logs. Occasional drops of an error log can be acceptable, but a failed audit log write could alter the outcome of a future compliance investigation. For example, delivering logs from your application to an HTTP webhook is risky if there’s no mechanism to retry the submission. Events sent while the network’s down would be irretrievably lost.
Audit logs enable you to maintain compliance by proving whether certain actions have been performed. Having the ability to retrieve the sequence of changes that a particular user’s initiated, or which occurred on a specific day, can be critical when you need to demonstrate you’ve met the requirements of regulatory frameworks such as ICO 27001 or SOC2.
Selecting technologies that produce audit logs will ensure you’ve always got the information you need on hand. For example, a PCI DSS audit might result in questions about who has accessed payment information and the context of their request. If you use Cerbos as your authorization platform, you could retrieve its audit logs and list out the “accept” decisions that relate to payment access.
Another side to audit logs and compliance is the ability to provide legal evidence that breaches have occurred. Without audit logs, breaches could go undetected for months or years, and any signs of their existence might be non-conclusive. A detailed audit trail informs when customers and regulators need to be notified. It can prove to insurers that a breach did occur or defend a lawsuit by demonstrating your system’s integrity.
Audit logs also allow you to reconstruct the events that led up to unexpected behavior or a security breach. After observing an issue, you can access the audit log and repeat the actions it’s recorded in a fresh development environment. This can help reproduce the bug and identify how attackers gained access. Audit logs don’t usually contain precise technical data, however, as they’re focused on answering business queries about who did what in the system. Application-level logs are still required to reveal the specific code paths that handled a user action.
Finally, a robust audit log implementation challenges you to identify the turnkey points in your system that have a bearing on your organization’s operations, both past and future. Each audit event increases accountability, provides visibility into changes and helps you recognize where risks are lurking. They demonstrate to developers, legal teams, customers and regulators that you’re proactively managing risks.
Audit logs create a historical record that’s maintained independently of your system’s current state. Administrators and compliance teams can use the audit logs to investigate user actions, spot suspicious activity and adhere to regulatory frameworks.
Comprehensive audit logs are especially critical for your application’s authorization requests. Cerbos’ open source authorization platform provides full visibility into incoming requests and their corresponding responses, explaining why specific allow or deny decisions were made. This equips you to sustain compliance and identify suspect behavior. Audit logs can be retrieved from the Cerbos admin API or written to a JSON file or standard output stream, ready to be forwarded to a log aggregation platform.
The full article was first available on The New Stack - read it here.
Book a free Policy Workshop to discuss your requirements and get your first policy written by the Cerbos team
Join thousands of developers | Features and updates | 1x per month | No spam, just goodies.