Service Ops

The Refund Answer That Was Wrong for Six Weeks

A bot quoted the wrong return window to forty customers and nobody noticed, because confident answers look identical to correct ones. Citations are how you tell them apart.

Пока только на английском — китайская версия скоро появится.

A skincare brand we talked to discovered their support AI had been quoting customers a 60-day return window. Their actual policy is 30 days. The 60-day version came from a Black Friday promo two years earlier — the old policy PDF was still sitting in the knowledge base, and the retrieval happily served it. For six weeks, the bot confidently told about forty customers they had two months to change their minds.

Nobody on the team caught it. A customer did — on day 45 of her return clock, with screenshots.

Here's the part that should bother you: every one of those answers looked great. Polite, well-formatted, specific. If your AI doesn't show where its answers come from, some version of this failure is running in your support channel right now, and you have no mechanism for finding out.

Wrong answers don't look wrong

When a human agent isn't sure, you can usually tell. They hedge, they say "let me check," they ping a teammate. When a language model is wrong, it's wrong in exactly the same tone as when it's right. Fluency and accuracy are different axes, and the model only ever shows you fluency.

Customers won't flag it either. Think about who received the 60-day answer: people who got a better deal than the real policy. Nobody complains about that. Customers report answers that sound wrong or unhelpful — they have no way to detect answers that sound right and aren't. Which means confident-and-wrong stays invisible until money moves and someone pulls up screenshots.

The three places it bites hardest

Return and refund policies. These change more than any other document a support AI relies on. Promotions create temporary exceptions, exceptions become stray documents, stray documents outlive the promotion. An AI has no native concept of "current" — it has whatever the retrieval found.

Shipping-fee tables. "How much to ship to Alberta?" involves region tables, weight brackets, and carriers that reprice twice a year. An AI answering from last year's table is precise, specific, and wrong — the worst combination, because a specific number sounds far more authoritative than a vague one.

Warranty terms. Duration and coverage differ per product line. Ask about the wrong SKU and a model can blend two manuals into one answer that reads perfectly and is accurate for neither product.

Notice the pattern: all three are cases where the words are fine and the facts have versions. That's exactly the failure mode fluency can't save you from.

What a citation actually changes

A citation is a small UI element — a line under the answer that says something like "Source: Returns Policy, updated May 12." It changes three workflows completely.

Spot-checking drops from minutes to seconds. With a source attached, a reviewer reads the answer, glances at the cited passage, done — five seconds. Without one, verifying a single answer means searching your own document pile to figure out what the bot might have read. That's minutes per answer, which in practice means nobody does it, which in practice means zero review.

Disputes get an audit trail. When a customer says "your bot told me X," you can pull the conversation and see not just what it said but what it read. Then the fix is obvious: if the doc was wrong, fix the doc; if the doc was right and the answer wasn't, you've found a retrieval problem worth escalating to your vendor. Without citations, every dispute is archaeology.

Stale documents become visible. This is the one that would have saved the skincare brand. In week one, an agent skimming conversations sees "Source: BFCM 2024 Returns Promo" under an ordinary Tuesday answer, thinks why is that still alive, and deletes the doc. Citations turn silent rot into visible clutter — and visible clutter gets cleaned.

The routine that makes citations work

The citation is just the display layer. What it enables is a lightweight discipline:

  • One fact, one document. The return window lives in exactly one doc. If two documents state it, they will eventually disagree, and your AI's answer becomes a coin flip you can't see.
  • Date your policies in the body text. "Effective March 2026" inside the document, not just in the filename. It shows up in the citation preview, and stale versions announce themselves.
  • A ten-minute weekly spot-check. Pull ten AI conversations, read each answer against its cited source. Answer matches source: fine. Answer contradicts source: retrieval bug, report it. Source is outdated: doc bug, fix it now.
  • Make deletion part of policy changes. The day a policy changes, the checklist has two items: upload the new doc, delete the old one. Most teams only ever do the first, and orphaned versions are where six-week incidents come from.

None of this is heavy. All of it is impossible without knowing which document produced which answer.

What to ask your vendor

If you're evaluating AI support tools, "does it cite sources" belongs in your first five questions — ahead of anything about model choice. And press past the checkbox:

  • Does it cite the specific passage, or just wave at a document title?
  • Do agents see the sources inline in the conversation view, without leaving the thread?
  • When the AI can't find a source, does it say so and hand off — or does it improvise?

Lane.Chat renders sources under every AI answer, pointing at the exact knowledge-base passage the answer came from, and the agent sees them right in the conversation. The design goal is simple: any teammate can audit any answer in five seconds.

Because an AI answer you can't audit isn't automation. It's liability with good grammar.