Governance stays with the data.

By Gil Salu, Cyaxios

We're in an age where LLMs pull data in from everywhere, reshaped at every stop along the way. Take something as ordinary as car auction data: the same rows can end up on a public results page or an internal operations desk, and those two audiences don't get to see the same thing.

The rule, and why names aren't enough

A bidder's identity, their ZIP code, the amount they bid, the reserve price the seller quietly set: none of that belongs on the public page. The lot number, the title, whether the lot is still open: that's fine to publish. The rule is simple to write down. It's much harder to keep true once the data starts moving, because rename a column, join it against a lookup table, group and count it, and it looks new. A system that decides what's safe by reading column names will wave a disguised column straight through.

tn-polars is a small, working answer to that, built directly on top of Polars, the dataframe library. Before any of this reaches an LLM, Polars does its usual groundwork: cleaning up records, putting them in order, filtering down to whatever's actually in play. What tn-polars adds runs in parallel and never shows up in the output itself — every time a column gets selected, renamed, derived, or aggregated, it remembers which raw input columns that column actually came from.

The query itself is usually written by a coding agent, not typed out by hand column by column. tn-polars ships as an installable agent skill, tn-polars-policy-compiler, that teaches an agent the lazy-first, lineage-aware way to write one: discover the schema, classify fields against the governance body, write the whole transform as a single chain, and release only at the end.

{
  "instruction": "Run internal live-auction monitoring and produce public lot results without exposing bidder data.",
  "do_not_use_for": "Disclosing bidder identity, bidder ZIP code, county or any other location derived from bidder ZIP code, bid amount, reserve price, or a public result selected using those values.",
  "policy": "tn://policy/auction-bids@1#sha256:abc123"
}

Before a single row moves, that rule gets compiled. An LLM reads it alongside the actual field names in the data and proposes a classification for each one — public, internal, or confidential — constrained to a schema that won't let it invent a field or a fourth category. The model only proposes; plain code checks the result against the original rule, the exact field names, and the three allowed labels before any of it becomes real policy.

{
  "fields": {
    "lot_id": "public", "lot_title": "public", "status": "public",
    "bid_amount": "internal", "bid_timestamp": "internal", "reserve_price": "internal",
    "bidder_id": "confidential", "bidder_zip": "confidential"
  }
}

That classification compiles into an enforcement point: a short Rego program, run as compiled WebAssembly at decision time. Two checks, both run against lineage, not names — does any column in the output trace back to something non-public, and did any filter use a non-public field to decide which rows survived, whether or not that field shows up in the final output.

deny contains "public release has non-public output" if {
    input.audience == "public"
    some column in input.columns
    column.classification != "public"
}

deny contains "public release uses a non-public row-selection source" if {
    input.audience == "public"
    some source in input.row_selection_sources
    data.tn_polars.policy.fields[source] != "public"
}

Watching it decide

Internal operations gets to see everything: bidder identity, ZIP code, the bid amount itself. That's allowed, because internal is exactly who this data was meant to reach.

frame.filter(pl.col("status") == "open") \
     .select("lot_id", "bidder_id", "bidder_zip", "bid_amount") \
     .release(..., to="internal")
# -> released

Now turn ZIP codes into county names, and it looks like a brand-new, harmless chart.

frame.with_columns(pl.col("bidder_zip").replace_strict(county_by_zip).alias("county")) \
     .group_by("county").agg(pl.len().alias("bid_count")) \
     .release(..., to="public")
# -> PolicyDenied: public release has non-public output

county is a new name. The rows are a count, not a raw ZIP. None of that matters — the lineage still says this column came from bidder_zip, so it carries the same restriction, and it's blocked from going public.

Same idea, a different disguise. This list of lots looks like something anyone could see.

frame.filter(pl.col("bid_amount") >= pl.col("reserve_price")) \
     .select("lot_id", "lot_title") \
     .release(..., to="public")
# -> PolicyDenied: public release uses a non-public row-selection source

Neither bid_amount nor reserve_price shows up in the output. But a private reserve price quietly decided which lots made this list. That decision follows the list. Blocked too.

What's left, the lot number, the title, the status, never touched anything private.

frame.select("lot_id", "lot_title", "status").release(..., to="public")
# -> released

That's the one table that's actually safe to share with the world. And once it's written, it doesn't travel alone.

{
  "file_sha256": "97926fa6c8e88b6c0203bdcdf9a027b3c828bf0d517617f22bfbd4c38146f03d",
  "policy_ref": "tn://policy/auction-bids@1#sha256:abc123",
  "opa_decision": { "allow": true, "reason": "release satisfies compiled TN policy", "id": "1659617249f456d3" },
  "lineage": { "lot_id": ["lot_id"], "lot_title": ["lot_title"], "status": ["status"] },
  "row_selection_sources": []
}

Policy, decision, lineage, hash, attached to the file, so the next system, or a human auditor, doesn't have to trust the release blindly. It can check. Change so much as one byte of the file after release, and the hash stops matching; the file is refused the next time anyone tries to read it.

That's the piece being built here — proof that travels with the data itself, past the one boundary where it's easiest to lose, rather than a rule that only holds at the instant someone hits publish.