SAMPLE: a fictional business. Not a client of Adwaizer.
Document Answer Agent: handover report for Ottermoss Gear Rentals (sample)
Prepared by: Adwaizer AI Consulting Inc. · Engagement: Document Answer Agent · Report version: sample, October 2026
What this page is. This is the handover report a client keeps at the end of the Document Answer Agent build. Ottermoss Gear Rentals, its people, its help centre, every passage quoted below and every result are invented to show the format and depth; none of them exist, and any likeness to a real business is a coincidence. In a real build, every row points to the client's own documents, repository and test run.
How to read the numbers. Every number in this report is illustrative: invented for this sample, not measured. They are marked (illustrative) where they appear. The pass marks (recall@5 of 0.85 or higher, 10 cited questions, 3 declines) are the offer's acceptance test, not results. A real report shows the client's own run, and its numbers can be higher or lower.
1. What was built
Ottermoss Gear Rentals is a fictional outdoor-equipment rental shop (tents, canoes, winter gear) with a counter team and an online booking page. Customers ask the same questions by email and at the counter; staff look answers up in the shop's help centre. Ottermoss asked for an assistant that answers from the help centre only and shows where each answer comes from.
| Item | What was delivered |
|---|---|
| Documents covered | One format: the help centre's articles exported as HTML, (illustrative) 212 articles split into 1,486 passages. Each passage has a stable id (for example hc-rentals-014.p3) and a link back to its article |
| Not quoted | 4 articles on the client's "must not be quoted" list (staff pay, supplier terms) (illustrative count), excluded at ingest and listed in the handover note |
| Who uses it | Counter staff, through Claude Desktop with the MCP server (search, get_section, verify_citations); the shop's own booking page team, through the REST API (/v1/search, /v1/sections/{id}, /v1/research) |
| How an answer works | Search returns the best passages; the answer cites passage ids and quotes them word for word; the verifier checks every quote against the passage before the answer is shown; questions the help centre does not cover are declined |
| Where it lives | The client's own repository and the client's own model API account. Our copies of the documents were deleted at handover (handover checklist, item 7) |
2. Acceptance test
The client ran the offer's four checks on handover day with the run guide. All four passed (illustrative).
| # | Check (offer's pass mark) | Result (illustrative) | Pass |
|---|---|---|---|
| 1 | Retrieval eval on the golden set: recall@5 of 0.85 or higher | 0.90 on 30 golden questions (27 of 30 found in the top 5) | Yes |
| 2 | 10 golden questions: every claim shows a citation and a quote, and is marked verified or visibly "unverified" | 10 answers, 23 claims: 22 verified, 1 shown as "unverified", 0 hidden | Yes |
| 3 | 3 questions outside the documents: each declined, no citation invented | 3 of 3 declined, 0 citations | Yes |
| 4 | Every answer shows tokens, USD and seconds | 13 of 13 answers | Yes |
The golden set. Each golden question is one line: the question as a customer would ask it, and the id of the passage a person would point to. Ottermoss supplied 30 (illustrative); we drafted them on day 1 and the client corrected them.
Ten of the golden questions (illustrative results):
| Question | Expected passage | Rank found |
|---|---|---|
| Can I pick up a canoe the evening before my rental starts? | hc-rentals-014.p3 | 1 |
| What happens if I return a tent late? | hc-returns-002.p1 | 1 |
| Do you rent kids' snowshoes? | hc-winter-007.p2 | 2 |
| Is a life jacket included with a kayak? | hc-water-003.p1 | 1 |
| How much is the damage deposit on a canoe? | hc-deposits-001.p2 | 1 |
| Can I change my booking dates online? | hc-booking-009.p1 | 3 |
| Do I need to clean the stove before returning it? | hc-returns-005.p4 | 1 |
| Can someone else pick up my booking? | hc-rentals-021.p1 | 2 |
| Are pets allowed in rented tents? | hc-camping-011.p2 | 1 |
| What do I do if a zipper breaks on the trail? | hc-repairs-004.p1 | not in top 5 (one of the 3 misses) |
The misses, and what was fixed. On day 7 the first run scored (illustrative) 0.77. Two fixes brought it to 0.90: (1) article titles were added to each passage's search text, so "zipper" questions reached the repairs article; (2) two FAQ answers that the export had merged into one passage were split. The remaining 3 misses (illustrative) are questions worded very differently from any article; they are listed in the eval report so Ottermoss can add a sentence to the help centre if it wants.
3. Example answers
Three answers as staff see them, and one decline. The source passages are invented for this sample.
Answer 1. Question: "Can I pick up a canoe the evening before my rental starts?"
Yes, for a fee. Evening pick-up the day before is offered for canoes and kayaks when the shop has stock.
- Claim: evening pick-up the day before is possible for canoes. Citation
hc-rentals-014.p3, quote: "Canoes and kayaks can be picked up after 5 pm on the day before your rental starts, if available." Verified.- Claim: there is a fee. Citation
hc-rentals-014.p4, quote: "Early pick-up is charged as a half day." Verified.Tokens 2,940 · USD 0.011 · 3.8 s (illustrative)
Answer 2. Question: "What happens if I return a tent late?"
A late return is charged per day, and the shop holds the deposit until the gear is back.
- Claim: late returns are charged per day. Citation
hc-returns-002.p1, quote: "Gear returned after closing time on the due date is charged one extra day for each day it is late." Verified.- Claim: the deposit is held. Citation
hc-deposits-001.p4, quote: "We release your deposit once all items are back and checked." Verified.Tokens 3,105 · USD 0.012 · 4.1 s (illustrative)
Answer 3, with one claim shown as unverified. Question: "Do you rent kids' snowshoes?"
Yes. Children's snowshoes are rented in two sizes.
- Claim: children's snowshoes are available. Citation
hc-winter-007.p2, quote: "Youth snowshoes are available for children from about 20 kg." Verified.- Claim: there are two sizes. Citation
hc-winter-007.p2, quote: "Youth snowshoes come in two sizes." Unverified: this sentence is not in the passage, which names no number of sizes. The model's repair turn did not fix it, so the claim is shown marked "unverified" rather than hidden.Tokens 3,420 · USD 0.013 · 6.2 s, including the repair turn (illustrative)
This is the verifier working as designed: the reader sees which part to check. After this run Ottermoss decided whether to add the sizes to its help centre; the agent itself never fills the gap.
A decline. Question: "Which trail near the lake is safest for beginners?"
I can't answer that from Ottermoss's help centre: none of its articles covers trail choice or safety. Please ask a staff member.
No citations. Tokens 1,210 · USD 0.004 · 2.0 s (illustrative)
The decline rule: when no passage supports an answer, the agent says so and cites nothing; it never invents a citation or answers from general knowledge.
4. Cost per answer and limits
Cost per answer. Every answer reports tokens, USD and seconds, as in section 3. On the acceptance run (illustrative): 13 answers, median 3,000 tokens, USD 0.011 and 3.9 seconds per answer, and the most expensive was USD 0.013 (the answer with a repair turn). USD is computed from the model's per-token price table, which is in the repository and is updated by the client when the provider's prices change. The model cost is paid in Ottermoss's own API account. Search and the verifier make no model call and cost nothing per question.
Limits, as the offer and the kit state them.
- The agent quotes Ottermoss's documents. It does not judge whether they are correct or current; an out-of-date article gives an out-of-date answer, with its quote.
- One document format (the help-centre HTML export). PDFs, scanned pages that need OCR and handwriting are not read; another format is quoted separately.
- Passages on the "must not be quoted" list are excluded at ingest; adding to that list means re-running the ingest (run guide, section 3).
- No accuracy promise beyond the acceptance test. "Verified" means each quote is word-for-word text of the cited passage, not that the answer is right.
- Not legal, medical, financial or other professional advice.
- Hosting, monitoring and running costs after handover are not included.
5. Handover checklist
| # | Item | Status (illustrative) |
|---|---|---|
| 1 | Source code in Ottermoss's repository, with the ingest adapter, the API, the verifier and the MCP server | Done |
| 2 | Tests pass with the model mocked; gitleaks clean on every push | Done |
| 3 | Golden set (30 questions, illustrative) and the eval report: score, misses, fixes | Done |
| 4 | Per-answer cost report and the price table it uses | Done |
| 5 | One-page run guide, including claude mcp add for the MCP server and how to re-run the ingest | Done |
| 6 | Acceptance checklist (the four checks in section 2), run and signed by the client's named contact | Done |
| 7 | Our copies of the documents deleted; the API key stayed in Ottermoss's own account throughout | Done |
| 8 | 45-minute walkthrough call | Done |
| 9 | Handover note listing everything in the repository, the "must not be quoted" list and the 3 eval misses | Done |
If a check had failed, the fix window in the offer applies: fixed within 5 working days at no charge.
Sample report. Ottermoss Gear Rentals is fictional; any likeness to a real business is a coincidence. Adwaizer AI Consulting Inc. promises no accuracy or savings outcome beyond the acceptance test; every result above is illustrative.