Reliable lead intake

Scope and assumptions

The system starts when a form, referral or upload reaches the intake endpoint and ends when a CRM record is confirmed or an owned exception exists. Everything below is a proposal; nothing has been deployed.

Assumptions behind the illustrative figures: 900 submissions a month; each submission makes one CRM create call; 3% of those calls hit a rate limit or 5xx and the chained automation does not retry them; 1% of successful creates lose their response and the chained automation retries blindly. Lost ≈ 900 × 0.03 = 27; duplicates ≈ 900 × 0.01 = 9. The held-for-review estimate assumes 35% of submissions touch an existing account and about one in eight of those needs a person.

Architecture

form / referral / CSV upload
        │
        ▼
 intake endpoint ──► Postgres: submissions + jobs (one transaction)
        │                         │
   thank-you page                 ▼
                           worker (SKIP LOCKED)
                 ┌────────────┼──────────────┐
                 ▼            ▼              ▼
            enrichment   intent model   territory rules
                 └────────────┼──────────────┘
                              ▼
                  CRM adapter: create with event key
                              │
                   read back by event key
                  ┌───────────┴────────────┐
                  ▼                        ▼
              delivered              review queue (owner, deadline)

• Intake endpoint. A small web service (Django or FastAPI). It authenticates the source (form token, partner key, upload user), checks consent flags, validates required fields and writes the submission and its job in one transaction.
• Store. Postgres. A submissions table holds the immutable payload and the event key; a jobs table is the outbox; an attempts table records every external call.
• Worker. A process that claims jobs with SELECT … FOR UPDATE SKIP LOCKED, so several workers can run without taking the same job. A library such as Procrastinate works; so does forty lines of plain SQL.
• CRM adapter. The only code that talks to the CRM. It knows how to create, how to search by the event key stored in a custom field, and how to classify CRM errors.
• Review queue. A table plus a simple internal page, or a CRM task assigned to the queue owner. What matters is that every held item has a reason, an owner and a deadline.

Data model

The event key is derived from the source's own identifier when there is one (form submission ID, partner reference) and from a hash of source, email and timestamp window otherwise. The same person submitting twice within a few minutes produces two submissions with one identity check, not two leads.

States

received → processing → delivered, with three side states: retry_wait (a transient error, retry scheduled), held (a person must decide) and rejected (consent withdrawn, spam, or invalid source). unknown is a property of an attempt, not a submission state: an unknown attempt always triggers reconciliation before anything else runs.

Failure handling

Where the model fits

Intent classification is a small, closed task: map free text to one of eight categories (demo, pricing, partnership, support, job application, and so on) and quote the phrase that justifies it. A small hosted model with structured output is enough. The output is validated against the category list; anything else is discarded. I would log the prompt version with every decision so a later change in intent quality can be traced to a prompt change.

The model never sees consent status or other leads, and nothing it returns is required for delivery.

Evaluation

Before launch, the shadow run described above gives three numbers per day: accepted, delivered, held. The invariant to check is simple: accepted = delivered + held + rejected + in progress. Any submission outside those buckets is a bug.

For intent, a set of 200 historical messages labelled by the RevOps manager gives a baseline. Report per-category precision and recall, and the share of messages the model left empty. Intent can be wrong without harm; I would only use it for sorting, not routing, until those numbers are known.

Trade-offs I considered

• Keep the no-code tool and add error alerts. Cheapest. It fixes the silence but not the duplicate or the lost promise at acceptance time, because the tool's run is still the only record.
• A managed workflow engine (Temporal, a cloud workflow service). Gives durable execution for free, but it is a new platform for a team with one part-time maintainer. A Postgres outbox covers this volume.
• Write straight to the CRM from the form. Makes the CRM the acceptance store. It fails exactly when the CRM is unavailable, which is the case we need to handle.

Stack

Python web service and worker, Postgres, the CRM's REST API, the enrichment vendor's API, one small hosted model with JSON-schema output, and OpenTelemetry traces keyed by the event key. Hosting on the company's existing cloud account stays well inside the stated budget of a few hundred euros a month.

Security and operations

Store only the fields the CRM needs; purge raw payloads after the retention period. Partner keys are scoped to one source. Alerts fire on three things only: jobs older than 15 minutes in business hours, holds past their deadline, and a daily reconciliation mismatch. Everything else is a dashboard, not a page.