The Auditor and the Scribe

Two of Biryar's agents produce words rather than numbers. One reads Amazon's warehouse records and drafts claims for stock that was lost and never paid for. The other reads buyers' messages and drafts replies with a small language model that runs on the business's own hardware. Neither of them sends anything, and that single decision shapes almost everything about how they are built.

The board is at the top because it would be easy to read the rest of this report as a description of working machinery. It is a description of finished machinery waiting for its inputs, written now because the design questions it answers are the interesting part, and they do not change when the data arrives.

Part AThe auditor

Money that goes missing in someone else's warehouse

Stock sent to Amazon for fulfilment lives in Amazon's buildings, and a small share of it is lost in transit, damaged in a warehouse, or never found after a customer return. Amazon's policy is to reimburse the seller for such units, in cash at an estimated value or with replacement units. Two of Amazon's reports describe the two sides of that promise. The inventory ledger records every movement of every unit, including the ones marked as lost or damaged, each with a reason code. The reimbursement report records what Amazon actually paid back. In principle the second covers the first. In practice the two drift apart, nobody on the seller's side reconciles them line by line, and the gap is money owed that nobody asks for.

Reconciling two independent records of the same events is one of the oldest procedures in auditing [1], and the auditor applies it in its plainest form. It reads every ledger event with a recoverable reason, damaged or lost in a warehouse, inbound, in transit or after a customer return, or destroyed, treats each as a claim the seller holds against Amazon, and then lets reimbursements pay those claims off.

The matching rule is deliberately conservative. Losses of the same product on the same day are collapsed into one claim. Reimbursements are applied oldest first, each to the oldest loss of the same product that is still owed and lies within 120 days of the reimbursement, and a single reimbursement can drain several losses, with every drained claim recorded so the trail can be audited. The rule matches on product, window and quantity, and pointedly not on the reason text, because Amazon's reimbursement reasons rarely echo the ledger's wording and trying to be clever there would produce false claims the owner could not trust. Whatever remains unpaid is owed, and a remaining balance becomes eligible for a drafted claim only once the loss is 45 days old, which gives Amazon time to pay on its own first. The scan looks back 540 days. All three windows are configuration rather than code, because Amazon's own claim rules have changed more than once and will change again.

The value of an owed unit is the owner's landed cost, the one number no Amazon report contains. It is the same dated cost that Part 5 of this series traces from supplier invoices, so a claim and a profit figure never disagree about what a unit was worth.

The weird-week detector

Beside the claims, the auditor watches for warehouses that start losing stock at an unusual rate, a sign of a process problem worth raising with Amazon even before any individual claim matures. For each warehouse it counts recoverable losses per day over the lookback window and scores each day with a robust z-score.

The median and the MAD are used because they resist outliers [2, 4]. With an ordinary mean and standard deviation, one bad week inflates the spread so much that the next bad week no longer looks unusual, which is precisely the case that matters. An early version had a subtler flaw. It built each warehouse's baseline only from days on which losses occurred, so a long quiet period followed by a sudden spike never accumulated enough history to be scored at all. Days without losses now count as zeros within the window, so silence is part of the baseline, as it should be.

Built to survive a long ledger

A seller's ledger over eighteen months can run to millions of rows for a large account, and Amazon delivers reports asynchronously, request, wait, download. The auditor is built for both facts even at this business's small scale. It streams reports rather than loading them whole, stores them in its own local database, and reconciles one product at a time, so memory never holds more than one product's history. Ingestion is idempotent. A re-pulled date range replaces what was there, and reimbursements are keyed on Amazon's own identifier, so repeated pulls converge rather than double count. Later runs ask only for days not yet seen. A report request still being prepared by Amazon when the process stops is remembered and resumed rather than requested again, every network call retries with exponential back-off on throttling and gives up at once on errors that retrying cannot fix, and a reconciliation interrupted halfway leaves a marked, unfinished run that the next pass can resume.

Until the live reader is switched on, the same pipeline runs from report files exported by hand from Seller Central, through the same parsers, so the file path and the live path produce identical events from identical rows.

A draft, never a filing

An eligible claim becomes a pending item in Biryar's approval queue, carrying the product, the warehouse, the date, the reason, the quantity and the amount, and a link to the prediction log entry that produced it, so the owner's eventual approval or rejection becomes a label for later learning. The text of the claim comes from a deliberately formulaic template, because the agents who process reimbursement requests work from structured facts, product, warehouse, date, quantity, reason, amount, and a well-ordered list of those facts is what gets a claim handled. The template is a replaceable component, and Part C describes how the scribe can supply more natural prose instead.

The auditor never files anything. Its configuration carries two separate safety switches, and both must be turned off before any claim could leave, and even then the interface for submitting a claim is intentionally left without an implementation, so a request to file answers that nothing is wired rather than pretending something was sent. Claims are filed by a person in Seller Central.

Part BThe scribe

Where buyer messages come from

Buyers write to sellers through Amazon's messaging system, and Amazon forwards each message to the seller's registered email address. Amazon's programmatic messaging interface sends messages but offers no way to read the inbox, so the scribe reads a mailbox. A parser recognises the forwarded emails by stable identifiers, the order number and the product's Amazon identifier, rather than by label text that varies between marketplaces and languages, and each message is paired with the order behind it, its dates, the amount paid and whether a return is still possible, because a reply that knows the ship date says something true and one that does not either guesses or stays vague.

A small model, on purpose

The drafts come from Qwen2.5-3B-Instruct [6], an openly available instruction-tuned model, quantised to four bits so that it needs about two gigabytes of graphics memory and runs comfortably on the four-gigabyte card of the development laptop, served by a local runtime that loads the model on demand. A larger model would write better prose and would not fit on the card. The trade is acceptable because the output is a draft that a person reads, and because a reply to a buyer needs to be short, correct and polite far more than it needs to be eloquent. The sampling temperature is kept low, at 0.3, so drafts stay conservative and predictable.

Running locally is a privacy decision as much as a cost one. Buyer messages contain names, addresses and order details, and the platform's rule is that business data and buyer data do not go to third-party models. The drafting agent can talk to three hosted language-model services, but it refuses to build any of them unless the owner has explicitly opted in, which has not happened, and even then sensitive numbers are removed before any text leaves. This follows the data-minimisation principle of European data protection law [7], applied in its strictest form, where by default the data does not leave the machine at all.

The prompt is a contract

The model receives the buyer's message, the order facts and a system prompt that reads more like a contract than an instruction. It may draft only. It may offer at most one of a short list of permitted actions, pointing to the tracking information already on the order page, offering a return label if and only if the order is still eligible for return, or escalating to a person within a day. It may not reveal costs, suppliers or wholesale prices. It may not apologise for an event the order data says has not happened. And it may not state any product fact it was not given, no specifications, ratings, measurements or compatibility claims, with a worked example of the right and wrong answer to a question about waterproofing and battery life. That last rule exists because language models produce fluent, plausible statements that are simply false, a failure studied at length as hallucination [8], and on a product page a false specification is a returned order and possibly a complaint.

There is a second reason to treat the prompt defensively. A buyer's message is text written by a stranger and placed inside the model's instructions. Research on systems that embed untrusted text in prompts has shown that such text can carry instructions of its own that the model then follows [9]. The scribe cannot fully prevent that, and it does not need to, because the worst a manipulated draft can do is sit in a queue in front of the owner. The decision not to send anything turns a security problem into a proofreading problem.

The flags deserve a note. They do not block a draft, they mark it, so that the owner's eye goes first to a sentence promising a refund of a specific amount or a replacement. Automation research has long documented that people supervising an automated system tend to accept its output without enough scrutiny, a tendency called automation bias [10]. A visible flag on exactly the sentences that commit money is a small, deliberate counterweight to it.

What a draft looks like

An illustrative exchange, written for this report. It shows the shape of the process, a declined specification question, a flagged commitment and a small edit, not a real buyer.

Learning from the owner's pen

Every draft is written to the platform's prediction log, and every decision the owner makes about it, sent unchanged, edited, or rejected, is recorded against it. The quality measure that follows is unusually honest. It is the share of drafts the owner sends without changing a word. That rate is the agent's primary metric, because an edit is the owner telling the system exactly where it was wrong.

Edits are also the training data. Each edited draft yields a pair, the text the owner preferred and the text the model wrote, which is the input of Direct Preference Optimisation, a method that tunes a language model directly on preferred and rejected answers without training a separate reward model [11]. It belongs to the same family of ideas as learning from human preference comparisons [12, 13], with the useful difference that here the preferences are not collected by asking anyone. They fall out of ordinary work. Training waits until at least thirty edited pairs exist, runs monthly, and would adapt the model with low-rank adapters on a four-bit base, the QLoRA recipe that makes fine-tuning feasible on a four-gigabyte card [14]. Like every model in Biryar, a retrained version ships in shadow and replaces the current one only when the owner promotes it.

The rate at which the system can learn is therefore bounded by how much the business writes. That is a sensible bound. A small shop with few messages needs few improvements, and the training data grows exactly as fast as the need for it.

Human in the loop, precisely placed

Parasuraman, Sheridan and Wickens proposed describing automation by how far it reaches into four stages, gathering information, analysing it, deciding and acting, and recommended that the level of automation be chosen separately for each [15]. The two agents in this report sit high on the first three and at zero on the last. They gather everything, analyse it, and propose a specific action with its full text, and the act itself, sending a message or filing a claim, stays with a person. The choice is not caution for its own sake. Amazon's rules put a person responsible for buyer communication, a claim filed in error costs credibility with the people who decide claims, and the cost of reading a short draft is small next to either.

Part CWhere the two meet

The auditor's claim text comes from a template, and the template is one implementation of a small interface for turning a structured claim into prose. The scribe implements the same interface with a second prompt, written for reimbursement claims, which is formal, cites the ledger event verbatim, avoids speculating about whose fault the loss was, and never asks for more units or more money than the ledger justifies. The auditor never imports a language model. The platform decides which implementation to hand it, so the deterministic template remains the default and the model's prose is an option that can be switched on, measured and switched off again without touching the auditor.

The division of labour is the general lesson of this report. Arithmetic belongs to deterministic code, which can be tested exhaustively and does not hallucinate. Language belongs to the model, which is good at tone and bad at facts. And the boundary between them is a narrow, typed interface that carries facts one way and words the other.

Honest marginsWhat remains

Switching on the report reader. The live adapter for Amazon's two reports exists and is tested against fakes, and several of its details, report type names, payload field names and the shape of the downloaded documents, are explicitly marked to be verified against Amazon's current documentation before first use. Until it runs, the weekly reimbursement job checks for a reader, finds none and stops.

The mailbox, and a quiet failure. The buyer mailbox currently refuses the system's credentials, every half hour. The job logs a warning each time and the scheduler records the run as completed, because the job is written never to fail a scheduler tick. That is the right behaviour for a job and the wrong signal for a person, and a repeated sign-in failure should surface on the platform's health view rather than only in a log.

The model behind the stub. The platform's configuration still selects the test stub for drafting, a safe default during development that should now give way to the local model already installed.

Language and currency. The prompts and the claim template were written when the business targeted an English-speaking marketplace, and some order fields are still labelled in dollars. German buyers will write in German, and the prompts should say in which language to answer.

Thin data, by nature. A young account with modest stock loses few units, and many losses are reimbursed by Amazon without any claim. The auditor's value will show over years rather than weeks, and its anomaly detector needs a month of history at a warehouse before it scores anything.

ReferencesSources

1. Arens, A. A., Elder, R. J., and Beasley, M. S. (2017). Auditing and Assurance Services. An Integrated Approach (16th ed.). Pearson.
2. Rousseeuw, P. J., and Croux, C. (1993). Alternatives to the median absolute deviation. Journal of the American Statistical Association, 88(424), 1273–1283.
3. Iglewicz, B., and Hoaglin, D. C. (1993). How to Detect and Handle Outliers. ASQC Quality Press.
4. Hampel, F. R. (1974). The influence curve and its role in robust estimation. Journal of the American Statistical Association, 69(346), 383–393.
5. Manning, C. D., Raghavan, P., and Schütze, H. (2008). Introduction to Information Retrieval. Cambridge University Press.
6. Qwen Team (2024). Qwen2.5 technical report. arXiv preprint arXiv:2412.15115.
7. European Parliament and Council (2016). Regulation (EU) 2016/679, General Data Protection Regulation, Article 5(1)(c). Official Journal of the European Union, L 119, 1–88.
8. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., and Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 1–38.
9. Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., and Fritz, M. (2023). Not what you've signed up for. Compromising real-world LLM-integrated applications with indirect prompt injection. Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 79–90.
10. Skitka, L. J., Mosier, K. L., and Burdick, M. (1999). Does automation bias decision-making? International Journal of Human-Computer Studies, 51(5), 991–1006.
11. Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. (2023). Direct preference optimization. Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36.
12. Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. (2017). Deep reinforcement learning from human preferences. Advances in Neural Information Processing Systems, 30.
13. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., et al. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35.
14. Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. (2023). QLoRA. Efficient finetuning of quantized LLMs. Advances in Neural Information Processing Systems, 36.
15. Parasuraman, R., Sheridan, T. B., and Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics, Part A, 30(3), 286–297.

Written from the agents' code, configuration and test suites and from the production server's logs on 6 October 2026. The exchange in Part B and the timelines in Figures 1 and 2 are illustrative. Biryar is a Ciya Labs project.