The first version of the marketplace was mostly a database and a set of import scripts. It collected business listings, attached them to cities and categories, and waited for a person to clean up the result. That worked at a small scale. Past 87,000 vendor listings, cleaning up by hand was a different problem.
The tempting solution was one large agent: give it a URL, let it browse, ask it to write the listing, translate it and send a message if something looks wrong. That design is easy to demo and difficult to trust. A timeout can leave half a listing behind. A hallucinated price can damage a business. A provider outage can turn a retry loop into a bill.
The system I built uses many small scheduled workers instead. Each worker has one job, a bounded input, an explicit output and a way to fail without stopping the batch. The constraints around those workers matter more than the models they call.
Start with a boring record
Every listing has a stable identity derived from its source and slug. Before an enrichment job begins, it checks whether that identity already exists and whether the particular enrichment step has already succeeded. The job can then be retried safely.
That idempotency flag is more valuable than a clever prompt. Workers get interrupted, and providers can return an error after accepting a request. A deployment may restart halfway through a batch. If the next run cannot tell what has already happened, the system will duplicate work or overwrite a better result with a partial one.
The pipeline keeps raw source data separate from generated fields. The source URL, original description, phone number and hours remain identifiable as imported material. Model output is stored as derived data with its own status. We can review or regenerate those fields and still tell them apart from the source.
One worker, one contract
The enrichment worker does not receive the whole marketplace. It receives one vendor and the fields that are eligible for enrichment. It can read a public website, extract services and contacts, and produce a structured result. It cannot publish arbitrary pages, send email or modify an unrelated vendor.
The output is validated before it reaches the application database. Required fields must have the expected types. URLs must look like URLs. Prices receive a stricter check because a plausible-looking invented number is worse than an empty field. If validation fails, the item is marked for review and the batch moves on.
The model path has a primary provider and fallbacks. A Claude call may be followed by GPT and then Gemini when the first provider fails or returns an unusable result. The fallback is not a second opinion. It is a way to keep a single provider outage from stopping the pipeline. Where disagreement matters, the item remains reviewable instead of being silently accepted.
Images make extraction harder. A vendor’s menu or price board often contains information that is missing from the HTML. The worker can pass supported images through a vision model, but we still check the extracted price. The model is allowed to say “not found.” That answer is preferable to filling a blank with a number that was never on the page.
The batch is the unit of failure
The scheduler processes a limited batch, and each item has its own success or failure state. One broken website, malformed image or rate-limit response should cost one item, not the whole run. The job records the reason, releases the connection and continues.
This sounds obvious until a system has a few thousand items waiting. A single Promise.all over the entire queue creates a failure domain the size of the queue. We use smaller groups with per-item isolation and provider limits. The batch is large enough to make progress and small enough to retry without guessing where the failure happened.
I also added a global cost-pause switch. A blunt tool, on purpose. If usage or provider pricing looks wrong, processing stops before the next batch. The marketplace can keep serving existing pages while enrichment waits. A switch that prevents a bill is more useful than a dashboard that explains the bill afterwards.
The same rule applies to scheduled content. Directory articles are generated from live marketplace data in bounded batches, then reviewed through the normal content workflow. The generator does not get permission to change vendor records. Content generation is a separate job because its failure should never block listing updates.
Email is where the gate matters most
Inbound email needs more judgment than enrichment. A message may ask to correct a phone number, claim a listing, complain about a price or remove a business entirely. The first version treated classification as the action. That was too much responsibility for one model call.
The current flow classifies the message and writes a pending row to a triage queue. It extracts the likely business, language and request type, then proposes what should happen. Most cases wait for a person in an admin queue with the original message, the classification and a draft reply.
There is one narrow automatic path: a clear removal request, one confident vendor match and zero live bookings. Before acting, the application checks those conditions again. It uses the same delisting operation as the manual admin flow, records the event and sends the confirmation in the requester’s language. If there are multiple matches, any bookings or any uncertainty, the row stays pending.
I want that automatic path to stay narrow. “The model is confident” is not a safety policy. We define the allowed cases through business conditions and check them again just before the irreversible part. Everything outside that subset becomes a human decision with useful preparation already done.
What I measure
On the dashboard, I watch the queue over time more closely than model confidence.
For enrichment, I watch completion rate by job type, retry rate, provider fallback rate and the age of the oldest pending item. For email, I watch classification disagreements, manual edits to drafts, automatic-path rejects and the number of cases that leave the narrow subset. A rising fallback rate usually means a provider problem. A rising edit rate usually means the input or the prompt contract changed.
I also sample generated listings against their sources. Aggregate success rates hide the failure that matters: a listing that is complete, well written and wrong about one important field. Sampling is slower than a green percentage, but it catches the errors users actually notice.
People still run the marketplace. They spend their time on exceptions and judgment, while small workers handle repeatable work within visible boundaries. Scheduled agents handle the volume. Validation, idempotency, cost controls and approval queues handle the uncertainty.
I keep returning to this design. An agent does not become reliable because it is given a larger prompt or a more impressive model. It becomes reliable when its job is small, its output is checked, its failure is contained and the system knows when to ask a person.
Sources and further reading
Making retries safe with idempotent APIs, by Malcolm Featonby, AWS. Useful background for preventing duplicate effects when batch work is retried. Idempotency does not, by itself, enforce spending limits.