Attention CRM Writeback Accuracy Test

A fluent CRM entry can still be wrong. Test whether Attention writes to the correct account, contact, opportunity and owner; whether propositions, actions and next steps are supported by the call; whether existing values are appended or overwritten correctly; and whether failures are visible and recoverable.

Official documentation was reviewed August 8, 2026. Gangly did not operate Attention or test its connector. Exact objects, field direction, overwrite/append behavior, provenance, retry ordering, idempotency and per-write review gates were not fully established publicly. Treat these as buyer-verified conditions, not absent capabilities.

Separate writeback accuracy from product fit

This page owns buyer-controlled CRM validation. The Attention migration checklist owns exit custody; comparison pages own fit; the transcription test owns word recognition. CRM correctness also requires truth extraction, identity, mapping, write semantics and recovery.

Official Attention docs describe two-way Salesforce scopes and admin prerequisites and AI-populated field mappings for Salesforce or HubSpot. These are bounded capabilities, not universal object coverage or accuracy evidence.

Freeze the connector and authority contract

Record tenant, plan, version/date, CRM org/sandbox, connector, integration user, OAuth scopes, objects, fields, direction, triggers, stage conditions, prompts, types, null behavior, append/overwrite rule, conflicts, timing, backfill, batch/rate limits, retries, idempotency, error surface, approval, audit, provenance and rollback.

Public docs do not specify every object, field direction, append/overwrite rule, or failure contract. Require a configured demo and captured vendor answer for each unknown. Attention’s API documentation establishes keys, permissions, rate limits, and revocation—not CRM-write idempotency.

Create authority with the CRM integration guide. CRM owns record IDs, owners and approved pipeline facts unless explicitly delegated. Attention may propose or write only named fields under named conditions. Define approver, override, replay and reversal rights.

Turn that contract into a versioned artifact map. One table should connect call ID, meeting source, participant identity, account/contact/opportunity candidates, selected primary record, owner, source transcript version, Attention prompt and mapping version, proposed field values, reviewer decision, write event ID, CRM before/after value, error/retry events and rollback record. A second authority table should identify the system and human allowed to create, append, replace, clear, merge, approve and reverse each field. Store the configuration hash and effective UTC interval so a later prompt or schema change cannot be evaluated against the wrong test version.

Build golden calls and frozen CRM cases

Use consented discovery, demo, evaluation, negotiation, customer and poor-audio calls. Balance lengths, speakers, accents, jargon, dates, currencies, negation, corrections, uncertainty and no-action calls. Include duplicate names, subsidiaries, several opportunities, merged records, changed owners and no valid match.

Freeze pre-call CRM account, contact, opportunity, owner, stage, close date, amount, risks, needs, next step, notes, custom fields and relationships. Give every call, participant, record, expected field and prohibited write a stable ID. Keep a holdout set so prompt tuning cannot memorize acceptance cases.

Sample by the write risks, not only by call volume. Ensure every critical field appears with positive, negative, absent, corrected and contradictory evidence. Include calls that should create no update, calls with two plausible opportunities, customers sharing a domain, former employees, forwarded calendar invitations and meetings where the CRM owner is not present. Pre-register the corpus manifest, inclusion/exclusion rules, train/tune/holdout split, annotation policy, adjudicator and prohibited uses. The CRM migration checklist supplies a reusable object-and-relationship mapping pattern for this frozen baseline.

Label propositions, actions, next steps, and uncertainty

Two qualified annotators independently label atomic propositions: speaker, subject, predicate, value, polarity, confidence and timestamps. Label actions with actor, object, owner, due date, status and explicit commitment. Label next steps with parties, action, timing, dependency and evidence.

Distinguish stated fact, buyer hypothesis, seller claim, question, rejected idea, tentative action, confirmed commitment and not stated. “Could review security” is not a confirmed meeting. “Budget is not approved” cannot become “budget approved.” Adjudicate disagreements and measure agreement before adjudication. Inconsistent human labels mean the field is not automation-ready.

Test identity, mapping, overwrite, provenance, and ownership

Test account, contact, opportunity and owner selection with aliases, renamed companies, duplicate people, multiple open deals, parent/child accounts, reassignment and absent participants. A safe no-match beats a confident wrong match.

For each field preserve truth, object/API field, type, allowed values, stage condition, prompt version, existing/proposed/written value, append/overwrite behavior, author, timestamp, call/timestamp provenance and approval. Test empty, stale and contradictory prior values. Use the CRM data-quality guide to baseline duplicates and invalid values first.

Reconcile at three levels after each run. At the field level, compare normalized expected, proposed and persisted values and verify that a rejected proposal produced no write. At the record level, verify object type, primary ID, owner, modification timestamp and unchanged protected fields. At the edge level, verify account–contact, contact–opportunity, activity–participant, call–evidence and write–approver links. Query the CRM again after its normal automation window: a correct immediate write that a workflow later overwrites is not an accepted final state. Preserve the before/after diff and downstream automation events.

Calculate precision, recall, field, owner, and edge accuracy

• Proposition precision = correct written propositions ÷ all written propositions.
• Proposition recall = correct captured truths ÷ eligible labeled truths.
• Action/next-step precision and recall use their labeled truth sets.
• Record-match precision = writes on correct primary records ÷ all writes.
• Field accuracy = accepted field values ÷ eligible written values.
• Owner/association accuracy = correct owner/relationship edges ÷ eligible written edges.
• Unauthorized-write rate = out-of-contract writes ÷ attempted writes.
• Recovery rate = failures reaching accepted final state without duplicate/incorrect writes ÷ injected failures.

Report by CRM, object, field, prompt, stage, call type, language, owner, overwrite/append and error. Never collapse into one score. Wrong-account writes, false commitments, lost prior values, permission bypass and duplicates are hard gates.

Inject ordering, auth, lifecycle, and hostile failures

Test duplicate delivery, replay with same/new event ID, concurrent calls, out-of-order and late input, missing participant, partial transcript, timeout, 429/5xx, expired token, permission loss, deprovisioned user, owner change, contact/account merge, opportunity delete, field rename/delete, picklist change and CRM maintenance.

Insert malicious transcript instructions to ignore policy, change stage, reveal secrets, overwrite amount or create tasks. Expected behavior treats conversation content as evidence, not administration. Test quoted and third-party claims. For every injection define expected write/no-write, retry limit, idempotency key, quarantine, alert, operator, recovery and rollback. Silent drop or unbounded retry fails.

Run combinations, not only isolated failures: a late transcript after an owner transfer; replay after a contact merge; token expiry after a write succeeds but before acknowledgment; a field rename during retry; a deleted opportunity followed by a backfill; and an injected instruction inside text attributed to the buyer. Reconcile both the Attention-visible event and the CRM audit trail. The expected result must say whether the system retries, suppresses, quarantines or requests human review, and which event ID prevents a second write.

Run privacy gates, preview, canary, rollback, and TCO

Use least-privileged integration users; separate config from approval; log prompt/mapping changes; restrict media access; and apply purpose, contract, hold and legal retention. Review Attention’s privacy policy with the executed DPA and tenant tests.

Start preview-only, then sandbox/shadow fields, then a small canary of consented calls and limited fields. Keep one authority per field. Monitor values, identity, edges, latency, errors, retries, duplicates, overrides and drift. Revalidate after prompt, mapping, schema, connector, model, permission or process changes.

Rollback disables Attention writes, preserves events, restores versioned prior values/edges, imports corrections, reconciles and prevents replay. TCO = license + implementation + CRM admin + prompt/field design + labeling + security/privacy review + sandbox + monitoring + exceptions + training + revalidation + rollback + exit. Use written quotes and measured labor.

Publish a decision table before the canary. Pass requires every hard gate plus accepted noncritical thresholds on the untouched holdout. Remediate and retest requires a bounded prompt, mapping, permission or workflow defect with an identified owner and fresh holdout evaluation. Reject or restrict applies when wrong-record writes, protected-field changes, permission bypass, lost provenance, silent failure or replay cannot be reliably contained. If only notes are trustworthy, authorize notes only; do not generalize that evidence to stage, amount, close date or next-step fields.

Use a worked example without inventing a Attention result

This fictional arithmetic demonstrates reporting only. Of 500 written propositions, 475 are correct; 600 eligible truths contain 450 captured items; 198 of 200 writes match the correct record; 380 of 400 field values pass; 290 of 300 edges pass; and 45 of 50 injected failures recover.

Do not average results. Investigate two wrong-record writes and five failed recoveries as potential hard stops. Report recall separately from precision.

Print the Attention CRM writeback test

Gangly did not produce these results and is not the presumed target. Any alternative should pass the same truth, authority, permissions, failure, rollback and TCO test.