Don't fetch that. A developer's guide to pre- vs. post-filtering for authorization

So you’ve done the hard work. You’ve decoupled your application logic from your authorization logic, moving your policies to a centralized service. Your codebase is cleaner, and your security posture is stronger. But a new, critical question emerges: how do you actually filter the data you send to users?

You need to ensure that when a user asks for a list of "documents," "projects," or "invoices," they only see the ones they’re truly allowed to see. This brings you to a fork in the road. Do you fetch a broad set of data and filter it in your app, or do you filter it at the source? This is the core dilemma of post-filtering versus pre-filtering.

The "fetch-then-filter" trap. The post-filtering approach

The most common starting point is post-filtering. It’s conceptually simple:

1. Your application receives a request, say GET /expenses.
2. It queries the database: SELECT * FROM expenses;.
3. It then iterates through every single expense record and, one by one, asks the authorization service, “Can the current user view this specific expense?”
4. If the answer is "yes," it keeps the record. If "no," it's tossed out.

Why it's tempting: This approach feels straightforward. The logic is easy to follow, and for a small-scale application or a prototype, it gets the job done without much fuss.

Why it breaks down: In the real world, this model quickly becomes a performance and security nightmare.

• Massive over-fetching: You're pulling potentially thousands of records into your application's memory just to throw most of them away. This puts an unnecessary load on your database, clogs network bandwidth, and slows your application to a crawl.
• The pagination puzzle: How do you implement pagination? If a user asks for page 2 with 10 items, you can't just LIMIT 10 OFFSET 10. You might have to fetch 10,000 records just to find 10 that the user is allowed to see on that "page." It’s inefficient and unpredictable.
• A security risk: For a brief moment, sensitive data the user shouldn't access is sitting in your application's memory. Even if it's filtered before being sent, this momentary exposure can be an unacceptable risk in secure environments.

A smarter way. Pre-filtering with a query plan

What if, instead of asking for permission for each item, you could ask your authorization service, “What are the conditions for a user to see any item?”

This is the power of pre-filtering. It flips the script entirely:

1. Your application receives a request: GET /expenses.
2. It first asks the authorization service, "Generate a query plan for a user trying to view expenses."
3. The authorization service, instead of returning a simple "allow" or "deny," returns a set of conditions.
4. Your application takes these conditions and integrates them directly into the database query.
5. The database does what it does best—filtering—and returns only the data the user is authorized to see.

With a stateless authorization engine like Cerbos, this is done via a partial evaluation of your policies. For example, imagine a policy where users can view their own expenses, and managers can view all expenses in their department. If a manager named "sara" in the "engineering" department requests the list, Cerbos would return a query plan containing a filter. That filter would have a condition object representing the abstract syntax tree (AST) of the rules.

It would look like this:

{
  "filter": {
    "kind": "KIND_CONDITIONAL",
    "condition": {
      "expression": {
        "operator": "or",
        "operands": [
          {
            "expression": {
              "operator": "eq",
              "operands": [
                { "variable": "request.resource.attr.ownerId" },
                { "value": "sara" }
              ]
            }
          },
          {
            "expression": {
              "operator": "eq",
              "operands": [
                { "variable": "request.resource.attr.department" },
                { "value": "engineering" }
              ]
            }
          }
        ]
      }
    }
  }
}

Your application’s data layer can then traverse this structure to translate it into a native SQL WHERE clause: WHERE ownerId = 'sara' OR department = 'engineering'. The logic is generated dynamically based on the user and the policies in effect.

The clear wins:

• Maximum efficiency: The database only returns the precise data needed. No over-fetching, no wasted resources.
• Rock-solid security: Sensitive data the user isn't authorized to view never leaves the database. It’s never loaded into the application's memory.
• Pagination that just works: Since the filtering is done at the source, pagination is as simple and reliable as it should be.
• Built to scale: This approach is designed for the complexity of modern applications, handling large datasets and intricate rules without breaking a sweat.

The bulletproof hybrid. Pre-filter and verify

For applications where you absolutely cannot compromise, think high-security or mission-critical systems, there's an even more robust pattern: the hybrid approach.

It’s simple: you do both.

First, you use the highly efficient pre-filtering method with a query plan to fetch the initial dataset from the database. Then, as a final check, you run the results through the authorization PDP one last time before sending the data on its way.

This provides the ultimate safety net. It combines the performance of pre-filtering with the explicit verification of post-filtering, protecting against any subtle misconfigurations or complex edge cases in your policies.

Make the right choice for your architecture

While the initial simplicity of post-filtering is alluring, it’s a solution with a ceiling. It creates performance bottlenecks and security concerns that are difficult to engineer your way out of later.

For modern, scalable, and secure applications, pre-filtering is the clear winner. By pushing the authorization conditions down to the data layer, you build a system that is performant, secure by design, and ready for future growth. And for those who need absolute certainty, the hybrid approach offers truly bulletproof authorization.

If you want to dive deeper, check out Cerbos Hub, join one of our engineering demos or check out our in-depth documentation.