# Data Provenance for AI Agents: What to Record
Use a practical data-provenance contract to connect AI-agent results to sources, runs, methods, reviewers, and evidence without copying sensitive payloads.
Data provenance is the structured history of where data came from, how it changed, and who or what
was responsible for producing it. For an AI-agent workflow, record the source and its version, the
run and method that transformed it, the responsible agent or reviewer, the resulting row, and a
safe evidence reference. Provenance makes a result inspectable; it does not prove the result is true.
A useful provenance record should answer five questions: Which result? Which source version? Which
activity produced it? Which agent, method, or reviewer was responsible? What evidence and decision
support the current value?
This guide turns those questions into a five-part **PROVE contract**:
1. **Pin** the result to stable identity.
2. **Retain** source identity and version.
3. **Observe** the generating activity.
4. **Version** the method and identify the responsible agent.
5. **Expose** evidence, review state, and corrections.
## In this guide
- [What is data provenance?](#what-is-data-provenance)
- [Data provenance vs data lineage](#provenance-vs-lineage)
- [Why AI agents need provenance](#why-ai-agents-need-provenance)
- [The PROVE contract](#prove-contract)
- [A practical provenance schema](#provenance-schema)
- [How to store provenance in Rowset](#provenance-in-rowset)
- [A worked example](#provenance-example)
- [Common provenance failures](#provenance-failures)
- [Data provenance FAQ](#data-provenance-faq)
## What is data provenance?
Data provenance is information about the origin and production history of data. It connects a
result to the source entities, processing activities, software or people, and derivation steps that
influenced that result.
The World Wide Web Consortium's PROV family defines a common model for exchanging provenance. Its
starting point has three concepts:
- an **Entity**, such as a source document, input row, or generated result
- an **Activity**, such as extraction, normalization, classification, review, or publication
- an **Agent**, such as a person, organization, software agent, or service responsible for an
activity or entity
The model also describes relationships such as an entity being generated by an activity, an
activity using an entity, and a result being derived from another entity
([W3C PROV-O, checked August 2026](https://www.w3.org/TR/prov-o/)). W3C's overview explains that
this history can support assessments of quality, reliability, and trustworthiness
([W3C PROV Overview, checked August 2026](https://www.w3.org/TR/prov-overview/)).
You do not need to implement the complete W3C ontology to make one operational dataset useful. You
do need to preserve the same basic relationships: result, source, activity, responsibility, and
derivation.
Provenance is evidence about production, not a truth certificate. A record can faithfully say that
an agent extracted a value from a particular page with method version `extract-v4`, while the page
itself is outdated or the extraction is wrong. Provenance helps a reviewer find and test the weak
link. It cannot replace source evaluation or result validation.
## What is the difference between data provenance and data lineage?
Terminology varies by system. In this guide, data lineage emphasizes movement among datasets, jobs,
and systems. Data provenance may also capture the source version, generating activity, responsible
agent, method, evidence, and decision behind an individual result.
| Record | Main question | Typical identity | Useful detail |
|---|---|---|---|
| Data lineage | Where did data flow? | dataset, table, job | upstream/downstream edges and run status |
| Data provenance | How was this result produced? | source version, activity, result | derivation, method, agent, evidence, review |
| Runtime trace | What did the agent execute? | trace, span, tool call | timing, requests, responses, errors, handoffs |
| Audit trail | What consequential action occurred? | event, actor, target | authorization, before/after state, outcome |
| Change history | What changed in this store? | resource and revision | fields changed, timestamp, actor label |
OpenLineage, for example, defines lineage events around datasets, jobs, and runs
([OpenLineage object model, checked August 2026](https://openlineage.io/docs/spec/object-model/)).
That is useful for pipeline-level flow. An operational agent may also need row-level answers:
which source item and revision produced `supplier:acme`, which prompt or parser version interpreted
it, and which reviewer accepted the value.
These records should connect through stable IDs instead of being forced into one giant log. A
`run_id` can link the provenance activity to a runtime trace. A `result_id` can link it to the
published row and its audit events. A `source_id` and `source_version` can link it back to the
captured evidence.
## Why do AI agents need data provenance?
AI agents combine deterministic tools, model judgment, changing sources, retries, and human
decisions. The final row often looks cleaner and more certain than the process that produced it.
Provenance preserves that uncertainty and context in queryable fields.
It helps with five practical jobs:
1. **Review:** a person can open the exact evidence and method behind a proposed value.
2. **Correction:** an agent can find every result derived from a source or method later found to be
wrong.
3. **Refresh:** a new source version can be compared with the version that produced the current row.
4. **Repeatability:** an operator can re-execute the bounded activity using the recorded inputs and
method version; equivalent output is not guaranteed.
5. **Accountability:** an investigation can connect the output to the agent, reviewer, approval,
and later state change.
NIST's Generative AI Profile describes provenance-data tracking as information about the origin and
history of content that can assist risk-management work across the AI lifecycle
([NIST AI 600-1, published July 2024](https://doi.org/10.6028/NIST.AI.600-1)). The amount of evidence
you retain should still match the consequence and sensitivity of the workflow. A public product
catalog and a private customer-risk decision should not use the same evidence policy.
Provenance also helps agents correct their own work safely. Without it, a request such as "rerun
everything produced by the broken parser" becomes a text search and a guess. With explicit
`method_version`, `activity_id`, and `result_id` fields, the agent can produce a bounded impact list
before proposing changes.
## Use the PROVE contract for agent data provenance
The PROVE contract in this guide is a minimum operational contract. Add domain-specific fields when
needed, but do not remove the relationships that make the history navigable.
### 1. Pin the result to stable identity
Give every published or proposed result a stable business key. The provenance record must point to
that exact key, not a title, current row position, or approximate text match.
For example, use `supplier:acme` or `claim:release-2026-08-25:postgres-support`, not "the Acme row."
A retry should update the same provenance relationship instead of creating another indistinguishable
result.
If the destination has no natural business key, use a generated stable ID and keep it unchanged.
The [index-column decision guide](/blog/choose-index-column-agent-rows) explains when to use a
business key and when generated `rowset_id` is safer.
### 2. Retain source identity and version
Record the source system, source-local item ID, and the version the agent actually saw. A current URL
alone is insufficient because its content can change.
Useful version signals include an API event ID, upstream revision, ETag, commit, publication time,
or content hash. Keep the original source value separate from any normalized value. If the source
does not expose a version, record the observation time and a permitted digest or snapshot reference
so the limitation is visible.
The [AI data-collection workflow](/blog/ai-data-collection) shows how to register approved sources,
checkpoint collection runs, and stage an observation envelope before publication.
### 3. Observe the generating activity
An activity is one bounded attempt to produce or change a result. Give it an `activity_id`,
`activity_type`, start and finish time, input references, result references, and status.
Do not overwrite a failed run with a successful retry. Keep both activities and relate the accepted
result to the successful one. The failed activity explains why a retry occurred; the successful
activity explains the current value.
Activities should describe work in controlled terms such as `extract`, `normalize`, `match`,
`classify`, `review`, and `publish`. A paragraph saying "the agent fixed the data" is not a
queryable activity record.
### 4. Version the method and identify the responsible agent
Separate the operational agent identity from the method it used. `agent_id = catalog-researcher`
answers who performed the activity. `method_version = supplier-extract-v4` answers which prompt,
mapping, ruleset, model configuration, or code version shaped the output.
If a human approves the result, record the reviewer and decision separately. An API-key label can
be a useful operational identity, but it is not proof that a particular natural person acted.
Avoid storing an entire system prompt or source payload in every row. Keep versioned methods in
source control or a protected registry and store a stable locator plus digest where verification
requires it.
### 5. Expose evidence, review state, and corrections
Store an evidence reference that an authorized reviewer can resolve. Add a short, permitted excerpt
or structured comparison only when it materially supports the decision.
Use explicit states such as `proposed`, `accepted`, `rejected`, `superseded`, and `needs_review`.
Bind the decision to the exact source version, method version, and proposed result. A later correction
should supersede the old assertion and explain why; it should not silently rewrite the history.
Evidence can be wrong, inaccessible, or incomplete. Record those conditions. `evidence_status =
missing` is more useful than an agent inventing a confident explanation for an unsupported value.
## What should a provenance assertion contain?
Use one row per result-to-activity-to-source assertion. A compact schema is easier for agents to
validate and join than nested narrative logs.
| Column | Purpose | Example |
|---|---|---|
| `provenance_id` | stable assertion identity | `prov:supplier:acme:country:v3` |
| `result_id` | exact proposed or published result | `supplier:acme` |
| `result_field` | optional field-level scope | `country_code` |
| `activity_id` | run or review that produced the result | `activity:catalog:20260825:18` |
| `activity_type` | controlled operation | `extract_then_review` |
| `source_system` | registered origin | `vendor_registry` |
| `source_item_id` | source-local stable identity | `vendor-771` |
| `source_version` | revision actually observed | `etag:9a781c` |
| `agent_id` | operational software identity | `catalog-researcher` |
| `method_version` | prompt, parser, ruleset, or code version | `supplier-extract-v4` |
| `evidence_ref` | protected locator, not necessarily raw content | `evidence://catalog/run-18/item-771` |
| `decision_status` | review state | `accepted` |
| `reviewer_id` | responsible reviewer when applicable | `ops-reviewer-2` |
| `recorded_at` | UTC observation timestamp from the trusted writer | `2026-08-25T06:20:00Z` |
| `supersedes` | prior assertion replaced by this one | `prov:supplier:acme:country:v2` |
Store activity timing, input references, retry relationships, and run status in
`provenance_activities`. Store source ownership, version method, sensitivity, and evidence policy
in `provenance_sources`.
Keep source and result identities separate. The same source item may support several result fields,
and one result may derive from several sources. When a result has multiple sources, create multiple
provenance assertions or a separate activity-input table rather than packing an ambiguous list into
one text cell.
For sensitive workflows, `evidence_ref` should point to protected storage with its own authorization
and retention policy. Copying full customer records, prompts, or tool payloads into a broad provenance
table increases exposure without necessarily improving review.
## How do you store data provenance in Rowset?
Use Rowset as the private structured record layer for provenance when trusted agents need to create,
inspect, search, and update the evidence contract through MCP or REST. Rowset does not collect source
data or runtime traces for you.
A practical design uses three datasets:
```text
provenance_sources index: source_id
provenance_activities index: activity_id
provenance_assertions index: provenance_id
```
`provenance_sources` defines the authorized origin, version method, purpose, sensitivity, and evidence
location policy. `provenance_activities` records bounded runs and reviews. `provenance_assertions`
connects exact result keys to source versions and activities.
Use these minimum headers before adding domain-specific fields:
```text
provenance_sources:
source_id, source_system, version_method, owner, sensitivity, evidence_policy
provenance_activities:
activity_id, activity_type, started_at, finished_at, agent_id, method_version, status
provenance_assertions:
provenance_id, result_id, activity_id, source_id, source_version, evidence_ref,
decision_status, reviewer_id, recorded_at, supersedes
```
Create Rowset relationships from `provenance_assertions.activity_id` to the activity dataset's
index and from `provenance_assertions.source_id` to the source dataset's index. Enable relationship
validation so non-blank references must resolve before the assertion is stored. The
[linked-dataset guide](/docs/link-datasets) documents the current MCP and REST relationship paths.
Add semantic column descriptions so the next agent does not confuse `source_version` with the time
the provenance row was written. Put controlled status values, evidence rules, and prohibited fields
in [dataset instructions](/blog/structure-dataset-instructions-ai-agents). Keep machine-readable
conventions in JSON metadata. The [schema-design guide](/docs/design-schema) documents current
semantic types, descriptions, metadata, and instructions.
Connect the agent through [hosted MCP](/docs/connect-mcp) when tool discovery is useful, or use the
[Dataset API](/docs/dataset-api) from an HTTP client. In both cases:
1. inspect the dataset and instructions
2. look up the stable index before writing
3. patch absolute final values by index
4. read the row back after an uncertain response
5. keep source and method secrets outside the dataset
Rowset datasets are mutable. They are not write-once storage, a cryptographically signed provenance
system, or a compliance ledger. Use a dedicated immutable or tamper-evident store when your assurance
requirements demand it, and keep stable references in Rowset for operational use. The
[AI-agent audit-trail guide](/blog/ai-agent-audit-trail) explains that boundary in more detail.
## Worked example: publish a supplier country code
Suppose an agent maintains a supplier catalog. It reads an approved registry and proposes
`country_code = DE` for `supplier:acme`.
### Register the source
Create `source_id = vendor-registry` with the canonical endpoint, allowed record scope, version
method, data owner, sensitivity, and evidence-retention policy. This is authorization context, not a
command embedded in source text.
### Record the activity
Create `activity:catalog:20260825:18` before extraction. Record the source version, agent ID,
`supplier-extract-v4`, start time, and intended destination. When the run finishes, set its status
and retain any error or retry relationship.
### Stage the assertion
Create `prov:supplier:acme:country:v3` with the exact supplier key, `country_code`, source item
`vendor-771`, observed ETag, activity ID, method version, and protected evidence reference. Mark it
`proposed` until deterministic validation and any required review pass.
### Accept and publish
The reviewer checks that the registry record really refers to the supplier, that the country value
uses the expected code system, and that no higher-priority approved source conflicts. The decision
changes to `accepted`, and the agent patches `supplier:acme` through its stable index. It then reads
the row back and records the verified result.
### Correct without erasing
If the registry later corrects the record, create a new assertion tied to the new source version.
Set `supersedes` to the prior provenance ID, review the proposed change, and update the supplier only
after acceptance. Retain the old assertion by workflow policy and dataset instructions so it remains
the explanation for the earlier value. Use immutable storage when that retention must be guaranteed.
## Common data-provenance failures
### Recording only a URL
A URL identifies a location, not the version the agent observed. Add the source-local ID and the
best available revision signal.
### Treating a model explanation as evidence
An explanation may summarize evidence, but it is generated output. Keep the protected source
reference and the deterministic checks that support the decision.
### Logging everything
Full prompts, source payloads, and tool arguments may contain credentials, personal data, licensed
content, or prompt-injection text. Store the smallest evidence needed for the review purpose and
apply a retention policy.
### Losing method versions
`model = current` and `prompt = latest` cannot reconstruct a past result. Use immutable version IDs
or content digests for prompts, rules, mappings, and code.
### Overwriting corrections
An in-place edit can destroy the prior explanation unless a separate history system preserves it.
Add a new assertion, retain the old row by policy, and link the new record with `supersedes`.
### Claiming provenance proves truth
Provenance can reveal that an output came from a stale source or an unreviewed method. That is useful
precisely because the record does not guarantee correctness.
## Data provenance FAQ
### What is data provenance in simple terms?
Data provenance is a record of where data came from and how it was produced. It connects a result to
its source, source version, processing activity, responsible agent or person, method, evidence, and
review decision.
### Is data provenance the same as data lineage?
Terminology varies by system. In this guide, lineage emphasizes movement among datasets, jobs, and
systems. Provenance may also capture the detailed origin, activity, responsibility, method, and
evidence behind a specific result.
### Does provenance prove that data is accurate?
No. Provenance makes the production history inspectable. Reviewers still need to assess source
quality, validate the transformation, resolve conflicts, and verify the final write.
### What provenance should an AI agent record?
At minimum, record a stable result ID, source ID and version, activity or run ID, agent ID, method
version, evidence reference, decision status, timestamp, and any superseded assertion.
### Can Rowset store data provenance?
Yes. Rowset can store private source, activity, and assertion datasets with stable indexes,
instructions, semantic columns, and MCP or REST access. Rowset does not automatically collect the
source evidence or make mutable rows into an immutable compliance ledger.
## Start with one result and one source
Do not begin by modeling every possible derivation in the organization. Choose one consequential
agent-produced field, record its exact source version and generating activity, and test whether a
reviewer can reconstruct the decision without searching logs by timestamp.
If that works, extend the same stable identities across collection, review, publication, and audit
events. The useful outcome is not more metadata. It is a result that an authorized person or agent
can inspect, correct, and safely reuse. Start with the [Rowset quickstart](/docs/quickstart), create
the three small datasets, and test one assertion from source lookup through destination read-back.