Query v2: annotated faithfulness
:::info Live in production
POST /query/v2 and POST /query/v2/stream are live in production beside /query since 23 Oct 2026, and /query/v2 is the route to build on going forward. They take the same authentication as /query: the X-API-Key header and the partner-signed JWT in Authorization: Bearer. /query is unchanged. Design: ADR-0051 and ADR-0052.
:::
/query withholds any sentence its faithfulness judge does not confirm, and refuses the whole answer when too little survives. /query/v2 runs the same agent with the same tools and scope, its own token-budgeted history window (see Conversations), and the same prompts except one: it asks the agent to write its answer as segments. After the agent finishes, v2 annotates instead of withholding:
- Citations. The agent returns an ordered list of segments, each with its
text, itskind(factorframing), the passages itcites, and alayout(paragraph,heading,list_item,continue,table_header,table_row). The server rendersanswerfrom them as Markdown, so every segment's offsets are exact. A fact segment that cites nothing is markeduncited. Every framing segment (a lead-in such as "Berikut rinciannya:", a greeting, "not found in the documents") is checked in one model call per answer that sees the question, with no word or digit rule in front of it; framing that states a fact needing a citation, or answers the question, is markeduncited, and so is every framing segment when that call fails. A framing segment that cites a passage is checked as a fact against it. - Faithfulness. A separate judge checks each cited segment against the passages it cites, shown with the same document identity (legal reference, section, number, year, instrument type, issuer) the agent saw, or for a portal or MCP result its dataset, source, period and the call that produced it. It also sees what the reader sees around the segment (its heading, its list's lead-in, its table header; a table row is checked whole), and never the question. The judge is strict: a segment is
supportedonly when its figure has the same scale (ribu, juta, miliar, triliun), unit, period, entity and condition as the source; the same value written differently (871.667.219.294 and Rp871,67 miliar) is still supported. The result is a mark on the segment; the segment is never removed. - Government Policy Guard. It reads the full answer, as on
/query. It is the only check that can replace the answer with a refusal.
The reader verifies a marked segment by opening its citation. A client must render the marks: an unsupported segment shown without its mark looks verified. The rules a client must follow are in Client contract.
Before publication, v2 may attempt one bounded citation correction for a detected
failure. A model may select a neutral introduction before a verified table or supply
additional existing citation labels. Already verified segments stay exact. The whole
candidate passes the same independent checks again; a failed or out-of-budget attempt
keeps the original answer and warning. There is no blanket exemption for factual
introductions and no automatic source assignment. Recall/file turns keep their own paths;
custom editorial language policies also skip correction so neutral copy cannot override them.
Synthesis metadata records a text-free v2_correction receipt; the JSON/SSE annotation
contract and final Policy Guard remain unchanged.
Request
The body is /query's body (see Query, Streaming Query, and Search APIs), offer_artifacts included: a requested file is judged by /query's file check, not annotated, and the reply names the file that was made or why it was not. If the Studio relay omits offer_artifacts, its server-stamped metadata.portal: "admin" opts in; an explicit offer_artifacts value still wins, and an omitted flag on other callers allows the existing model intent check to offer files for a requested file or chart. An unavailable intent check does not opt in. include_grounding and include_provenance are JSON-only, as on /query.
Every caller gets the annotations; no opt-in is needed. accept_annotations is accepted and ignored: it was a required opt-in from 21 to 23 Oct 2026, removed by product decision, and stays a known field so clients that still send it keep working. Use /query if the client cannot render the annotations.
curl -sS "$ACEH_API/query/v2" \
-H "X-API-Key: $ACEH_API_KEY" -H "Authorization: Bearer $PARTNER_JWT" \
-H "Content-Type: application/json" \
-d '{"query": "Berapa anggaran pendidikan Aceh tahun 2023?"}'
Prompt playground
POST /playground lets a verified administrator try one agent-editorial prompt without
publishing it. The body accepts the same fields as /query/v2 and adds the required
editorial_prompt string (1–2,000 characters):
curl -sS "$ACEH_API/playground" \
-H "X-API-Key: $ACEH_API_KEY" -H "Authorization: Bearer $ADMIN_JWT" \
-H "Content-Type: application/json" \
-d '{
"editorial_prompt": "Answer clearly and concisely in Bahasa Indonesia.",
"query": "Apa tugas utama Pemerintah Aceh dalam dokumen ini?",
"document_ids": ["<document UUID>"]
}'
The route executes the same query-v2 agent, retrieval, tools, access checks, budgets,
citations, annotate-v2 faithfulness checks, and Government Policy Guard. Only the
editorial prompt is replaced, for that request only. It requires both the shared API key
and a verified admin principal; it fails closed with 503 when verified identity is
disabled and is rate-limited per administrator.
The response keeps the query-v2 fields that still apply and adds
playground_run_id, published: false, and latency_ms. It deliberately omits
conversation_id and message_id: no conversation, message, memory, feedback, prompt
version, prompt label, or normal answer analytics is published. The server stores only a
small administrative receipt containing identifiers, status, counts, and timing—never the
candidate prompt, question, answer, or source-passage text. A successful run does not
change the prompt used by later /query or /query/v2 requests.
Response
Every /query field that keeps its meaning is kept: answer, conversation_id, message_id, trace_id, history_truncated, history_exchanges_in_view, citations, tool, tool_executed, fallback_used, fallback_reason, and the opt-in grounding, used_docs, tool_call_audit, model_call_log. Removed, because they describe a gate v2 does not have: claim_map, dropped_claims_count, dropped_claim_reasons, provenance_complete. Added: schema (always "annotate-v2", the contract this body follows), claims, framing and faithfulness. Kept with v1's meaning but counted from v2's verdicts: claims_supported, claims_total and grounding_rate, the claim grounding rate (Tingkat keterverifikasian klaim), so one badge renders from either route.
On v2, claims_supported counts supported claims and claims_total counts every claim except from_conversation, which restates the conversation, and uncited, the model's own statement with no source (show it as "Klaim model - tanpa sumber dokumen"); neither is checked against a source. An unsupported, unverified or invalid_citation claim lowers the rate. Leaving uncited out is the client's decision of 10 Nov 2026. grounding_rate is null when that total is 0, as on a refusal before any claim was checked. A Policy Guard block publishes no claims, but, as on v1, the rate still counts the claims checked before the block (counts only; no claim text). The same three keys are faithfulness.grounding (on the SSE annotations frame too, and stored in v2_faithfulness). It is not the faithfulness figure (ADR-0042); faithfulness.verification remains the answer-level banner state.
{
"schema": "annotate-v2",
"answer": "Berikut datanya. Anggaran pendidikan 2023 adalah Rp 5 miliar. Naik 9 persen dari 2022.",
"citations": [{"document_title": "Qanun Anggaran", "chunk_id": "…", "document_id": "…"}],
"fallback_used": false,
"fallback_reason": null,
"claims": [
{"text": "Anggaran pendidikan 2023 adalah Rp 5 miliar.", "citation_keys": ["P1"],
"citations": [{"citation_key": "P1", "citation": {"chunk_id": "…"}}],
"verdict": "supported", "unverified_reason": null, "invalid_citation_keys": [],
"kind": "fact", "layout": "continue",
"start": 17, "end": 61, "start_utf16": 17, "end_utf16": 61},
{"text": "Naik 9 persen dari 2022.", "citation_keys": ["P1"],
"citations": [{"citation_key": "P1", "citation": {"chunk_id": "…"}}],
"verdict": "unsupported", "unverified_reason": null, "invalid_citation_keys": [],
"kind": "fact", "layout": "continue",
"start": 62, "end": 86, "start_utf16": 62, "end_utf16": 86}
],
"framing": [
{"text": "Berikut datanya.", "layout": "paragraph",
"start": 0, "end": 16, "start_utf16": 0, "end_utf16": 16}
],
"faithfulness": {"mechanism": "annotate-v2", "judge_status": "complete",
"framing_check": "complete", "claims_total": 2,
"supported": 1, "unsupported": 1, "unverified": 0, "uncited": 0,
"invalid_citation": 0, "from_conversation": 0,
"unverified_reasons": {"deadline": 0, "budget": 0, "judge_unavailable": 0, "unparseable": 0,
"disputed": 0},
"framing_total": 1, "unlocated_total": 0, "verification": "partial",
"grounding": {"claims_supported": 1, "claims_total": 2, "grounding_rate": 0.5}},
"claims_supported": 1,
"claims_total": 2,
"grounding_rate": 0.5
}
verdict | Meaning | Required rendering |
|---|---|---|
supported | The judge found the cited passage supports the segment. | The only positive mark: "terverifikasi", with its citation. |
unsupported | The judge found the cited passage does not support it, and a recheck of that segment alone confirmed it (an unconfirmed rejection is unverified with reason disputed). | "Belum diverifikasi", for example "tidak didukung sumber"; its citation chip must not read as proof. |
unverified | The judge could not decide; unverified_reason says why. Never a pass. | "Belum diverifikasi". |
uncited | States a fact but cites nothing (including framing the framing check found checkable). | "Belum diverifikasi", for example "tanpa sumber". |
invalid_citation | Every passage it cites is one the agent was not shown. | "Belum diverifikasi"; the citation does not resolve. |
from_conversation | Restates this conversation (what the user said, or an earlier verified answer). | Its own neutral mark, "dari percakapan, bukan dokumen". Never styled as supported: a user's own premise echoed back is not confirmed by any document. |
Segments that assert nothing (a greeting, "Berikut ringkasannya:", a table's header row) are not claims: they are returned in framing, each with its text, layout and the same four offsets, and carry no verdict. Claims and framing together cover every character of answer except the layout the server writes between segments (whitespace, the ## and - markers, the table pipes and delimiter row); text outside every span would be counted in unlocated_total. answer contains no [P1] labels: a client places a claim's citation chips at its end, from its citation_keys. Every offset is a position in the rendered answer, end exclusive, and answer[start:end] is exactly the span's text, Markdown emphasis included:
| Field | Meaning |
|---|---|
start, end | Offsets in Unicode code points (Python string indices). |
start_utf16, end_utf16 | The same span in UTF-16 code units: use these as JavaScript string indices (answer.slice(start_utf16, end_utf16)). They differ from start/end after an emoji or any character outside the Basic Multilingual Plane. |
kind | The agent's own fact or framing. A framing claim is framing the framing check found checkable, or framing that cites a passage. |
layout | Where the segment sits: paragraph, heading, list_item, continue (same paragraph or item), table_header, table_row (cells separated by |). |
faithfulness.verification is the answer-level state, computed by the server:
verification | When |
|---|---|
full | At least one claim supported, every claim but uncited ones supported, and unlocated_total is 0. An uncited claim is shown as unsourced and does not count against full. |
partial | At least one claim supported, and anything else: a claim with another verdict (including from_conversation, but not uncited) or unlocated text. |
none | No claim supported, which includes a refusal, a greeting with nothing to check, and a turn where nothing was checked. |
framing_total counts the framing spans. unlocated_total counts the runs of answer text no claim or framing span covers; a refusal or notice, which carries no annotations, is one such run.
judge_status is complete, unavailable (at least one cited segment is unverified) or skipped (no cited segment). framing_check is the same for the one framing check: unavailable means every segment it had to check is shown uncited, never passed as framing.
unverified_reason is null except on an unverified claim, where it is one of deadline (the request's time ran out before the claim's judge batch returned; batches that had returned keep their verdicts), budget (the request's model calls were spent), judge_unavailable (the judge failed or timed out), unparseable (the judge gave no usable verdict for it) or disputed (the batched judge found the segment unsupported, but the recheck of that segment alone did not confirm it, so it is shown neither as supported nor as unsupported). A failed or timed-out recheck gives its own reason (deadline, budget, …). faithfulness.unverified_reasons counts them. The agent stops early enough to leave the judge its own share of the request's time and model calls.
invalid_citation_keys lists the passages a segment cites that the agent was not shown, or may not cite. A segment with at least one valid citation is judged against its valid ones and keeps that verdict; only a segment whose citations are all invalid is invalid_citation.
fallback_used says whether the API published a refusal, never whether it is trustworthy: false with verification: "none" is a published answer nothing verified. It is true for a refusal: policy_blocked (the Policy Guard), or one of /query's loop-level reasons (query_budget_exhausted, model_timeout, context_budget_exhausted, agent_output_limit (on /query/v2 only when a follow-up's re-asked forced tool round reaches its output ceiling; the could-not-answer-in-time notice is published), mcp_source_unavailable, source_out_of_scope, source_scope_unavailable, unparseable_completion). source_scope_empty is a pre-loop refusal described under Errors. A refusal has no claims. unparseable_completion is also the outcome when the agent's segments are still malformed after one regeneration; malformed output is never published.
context_budget_exhausted means the source material the question needed could not fit the model's context, so the answer was stopped before it was written and nothing was checked. Its faithfulness report carries "checked": false, on the JSON response and the SSE annotations frame alike: render it as "not checked", never as a failed or passed check. Every other response omits checked; do not read its absence as a pass. The notice text is fixed system copy, not a timeout message.
Reading a named document
When a question names a document and asks for one of its provisions, the answer checks that provision in the document itself before answering. For an amendment, it reads both the amending and the original regulation, and states an original wording only when a passage of the original shows it. Every part of a multi-part question is answered, or the answer says which part was not found. Each passage carries its stored page, so a question asking "di halaman berapa" gets the page.
Personal identifiers are masked
Mobile numbers, NIPs and other 15–20-digit identity numbers (NIK, KK) never appear in
answer, claims, grounding or used_docs. They are replaced with
[nomor HP disamarkan], [NIP disamarkan] or [nomor identitas disamarkan], including
values the user typed in the question. A place and date of birth in a labelled field
("Tempat/Tgl. Lahir: …") becomes [tempat/tanggal lahir disamarkan]. Office landlines, figures, decree numbers and
dataset IDs are unchanged. For a named person, the answer gives only their public role,
office and the document that names them; it declines salary, honorarium, home address
and other personal details.
Errors
The same status codes as /query for the same causes (400, 403, 404, 413, 502; 401 for a missing key; 422 for a malformed body), plus 409: a conversation belongs to the route that first answered in it. Continuing a /query conversation on /query/v2, or the reverse, returns 409. A conversation created empty with POST /conversations is claimed by whichever route answers first. Changing a conversation's sources or MCP selection does not release it.
Two causes differ: when the evidence the agent gathered no longer fits the model's context window, /query/v2 answers 200 with fallback_reason: "context_budget_exhausted" where /query answers 413. The same refusal answers when tools ran but none of their results fitted the context. A named selection with no indexed document answers 200 with fallback_reason: "source_scope_empty"; the API publishes fixed Indonesian refusal text before any tool runs or model call, including intent and trend classification, even on resumed recall questions. A selection the caller cannot access also has zero indexed documents and receives this refusal. A 404 from /query/v2 means the tools that ran returned nothing readable, which includes a search that failed or was stopped at the tool deadline. A 413 from /query/v2 carries fixed text, never the model server's error.
Streaming
POST /query/v2/stream sends the annotations before the answer text: tool.started / tool.finished while the agent works, then message.start, trace, citation, then either guard.blocked or passage, annotations, answer.delta, then done. The answer is buffered and guarded before the first frame, so every verdict is final when the text starts.
event: message.start
data: {"conversation_id": "…", "message_id": "…", "schema": "annotate-v2"}
event: citation
data: {"chunk_id": "…", "document_title": "Qanun Anggaran", …, "citation_key": "P1"}
event: passage
data: {"kind": "chunk", "chunk_id": "…", "citation_key": "P1", "text": "…", …}
event: annotations
data: {"mechanism": "annotate-v2", "claims_total": 2, "supported": 1, …,
"framing_total": 1, "unlocated_total": 0, "verification": "partial",
"grounding": {"claims_supported": 1, "claims_total": 2, "grounding_rate": 0.5},
"fallback_used": false, "fallback_reason": null,
"claims": [{"start": 17, "end": 61, "start_utf16": 17, "end_utf16": 61,
"kind": "fact", "layout": "continue", "citation_keys": ["P1"],
"verdict": "supported", "unverified_reason": null,
"invalid_citation_keys": []}, …],
"framing": [{"start": 0, "end": 16, "start_utf16": 0, "end_utf16": 16,
"layout": "paragraph"}]}
event: answer.delta
data: {"text": "Berikut datanya. Anggaran …"}
event: done
data: {"finish_reason": "stop", "history_truncated": false}
The annotations frame carries the faithfulness counts (including verification) at its top level, fallback_used and fallback_reason, and per claim and per framing span the offsets above, without the text: the answer.delta frames concatenate to exactly answer, so the offsets apply to the streamed text. Every citation frame and every chunk passage frame carries the citation_key it was shown under (a tool passage already did), so each of a claim's citation_keys joins to the source the reader opens to verify it; a key listed in invalid_citation_keys joins to nothing. The event is not /query/stream's faithfulness: that frame counts removed sentences, and a v1 client reading v2's counts under that name would see "nothing dropped". A fault after annotations ends the stream with an error frame; the marks for any text already received have arrived.
Client contract
A client of /query/v2 must follow these rules. They make "verified" the positive state, so no failure (a dropped frame, an unplaced span, a client that ignores a field) can look like a verified answer.
- Check the contract. Read
schema(JSON body, ormessage.starton the stream) and treat anything other than"annotate-v2"as unverified. - Mark verified text positively. Only a
supportedclaim is shown as verified ("terverifikasi"). Every other verdict, and anyanswertext outside every claim and framing span, is shown as "belum diverifikasi". Framing is not verified either; it needs no mark of its own, but it must not look like the verified mark. from_conversationis neutral. Show it with its own mark, "dari percakapan, bukan dokumen", never styled assupported.- Banner. Whenever
verificationis not"full", show an answer-level notice that stays visible with the answer, saying it is not fully verified. - No annotations, no trust. On the stream, an answer whose
annotationsframe was not received is rendered as "belum diverifikasi" in full, with the banner. fallback_usedis not trust. It signals a refusal; an answer without a fallback is only as verified asverificationsays.
Conversations
History selection and configuration are defined in ADR-0051’s conversation-history token budget. On v2, history_truncated also becomes true when a retained exchange is clipped, even if no whole exchange is dropped. history_exchanges_in_view still counts retained question/answer pairs.
Later turns see only what was verified: the history given to the model, the follow-up passages it re-reads, and long-term memory extraction use only supported (and framing or conversation) segments of an earlier /query/v2 answer. An earlier answer with no verified statement is the one exception, for the model's history only: it is quoted within the history budget after a note saying none of it was verified and it is not evidence, so the model knows what the reader saw instead of reading an empty answer. A linked follow-up carries documents forward for fresh, access-checked retrieval even when the earlier answer had no supported claims. This does not restore unsupported passages as evidence. One anchor is selected from a multi-document exchange; the v2 guard contract, rows v3–v3c defines selection priority, bounds and naming limitations. Search text stays the agent's own, with no titles added. The reader's own view of the conversation is the full stored answer.
Live data and chart requests
Ask for a named dataset with its geography, period and population breakdown. Discovery can continue beyond the first CKAN catalogue page; a catalogue miss, failed read and missing measure are different outcomes. BPS and portal series may disagree: a source substitution must identify the series rather than silently mix them.
A chart request can produce a downloadable PDF, XLSX or PPTX through the existing
artifact attachment fields. The file contains a real chart plus the cited data table.
Chart values come from that table; nulls are never invented as zeros. offer_artifacts: false disables first-turn file generation. Artifact support still requires the configured
file gate and object storage. No new frontend endpoint or database migration is required.
Public MCP citations after reload
JSON and streamed citations retain the producer's dataset URL, title and stable source
ID. The producer must include these with the final data-read result; discovery metadata
is not inherited. On GET /conversations/{id}, public MCP citations survive only when
the source remains enabled and public and the engine's successful execution receipt
binds that server, tool and source ID with visibility_mode: "public" at read time.
Private/profile sources, missing or legacy receipts and unsafe URLs remain withheld.
Changing a source from private to public does not release historical private evidence.
The SatuData producer update is still an external dependency; this engine change does
not add missing metadata to the connected server's response.