By Siddharth GangalA Gong transcript accuracy test should compare Gong output with a blinded human reference on the calls, languages, audio conditions, speakers, summaries, action items, and CRM objects your team actually uses. Measure word errors, speaker attribution, fact precision and recall, critical errors, record association, duplicate writes, retry behavior, and correction effort. Set the pass gates before anyone sees results.
Direct answer. Do not accept a demo transcript or a vendor-wide accuracy percentage as your decision. Build a consented, stratified corpus; create an independently annotated reference; run the contracted Gong configuration; score every layer separately; and reject the pilot if a privacy, association, destructive-write, or critical-fact gate fails.
This guide is a buyer-controlled validation protocol, not a Gong review, a Gangly-versus-Gong comparison, or a claim about Gong’s observed accuracy. We reviewed current public documentation on August 8, 2026, but did not test a Gong instance or customer recordings. All numbers in the worked example are illustrative. Your result will depend on edition, region, recording path, language, audio, CRM, permissions, mappings, and configuration.
Define what the Gong accuracy test owns
The test owns measurement, not product positioning. The Gong review owns broad capabilities and fit. The Gangly versus Gong comparison owns a head-to-head buying decision. The conversation-intelligence software guide owns category selection, and the AI call-analysis guide owns the general workflow. This page answers one narrower question: can the configured system reproduce and move decision-critical call evidence accurately enough for your controlled use?
Define the evaluated pipeline before collecting a call: capture → transcript → speaker attribution → generated summary and action items → CRM association → field or activity write → correction and audit. Record the exact Gong edition, enabled features, model or processing date when visible, transcription-language setting, conferencing or telephony source, CRM connector, object mappings, integration-user permissions, and any manual intervention. A change to one of those conditions creates a new test version.
Gong’s language documentation says support differs across transcription, analysis, translation, language, region, and dialect. Its spoken-language guidance also documents automatic detection, a selected language, and reprocessing. Therefore, test automatic detection and a correctly forced language as separate conditions. A translated summary cannot validate the source-language transcript.
Build a stratified, consented call corpus
Sample the production distribution and its risky edges. Start with a minimum corpus your annotators can review twice without rushing, then expand until each material stratum has enough observations to expose recurring errors. Do not let twenty clean English Zoom calls stand in for a multilingual, mobile, telephony-heavy operation.
| Label | Include | Why it matters |
|---|---|---|
| Language condition | Language, dialect or locale when operationally relevant, code-switching, automatic versus selected language | Separates coverage and detection failures from recognition failures. |
| Speaker condition | Two or more speakers, similar voices, interruptions, late joiners, internal/external roles | Tests who said what, not only which words appeared. |
| Audio condition | Headset, laptop, mobile, PSTN, room mic, noise, echo, cross-talk, low bandwidth | Represents actual capture paths and degradation. |
| Commercial content | Names, companies, product terms, amounts, dates, negation, uncertainty, commitments | Weights business-critical evidence explicitly. |
| CRM topology | One and multiple contacts, accounts and opportunities; duplicates, subsidiaries, merged records | Exposes association and overwrite risk. |
| Consent path | Approved notice/consent, opt-out, late joiner, recording stopped, excluded call | Confirms that an accurate output was lawfully and intentionally captured. |
Never infer accent from a person’s name, appearance, or protected identity. Where accent is material to the operating test, use a self-described or responsibly assigned acoustic label with appropriate notice, minimize the data, and report performance by the operational condition without ranking people. Keep a separate “unlabeled” group rather than inventing a category.
Accuracy begins with capture governance. Gong documents consent profiles that can combine a pre-call email, consent page, and audio prompt. It also notes that organizations configure these options for their policies and applicable requirements. Have privacy or legal owners approve the corpus, retention, access, transfer, and deletion plan. This guide is not legal advice. A consent failure is a hard stop, not an accuracy footnote.
Create a blinded human reference
Annotators should not see Gong output while creating the reference. Give two trained reviewers the same source audio, annotation codebook, and CRM ground truth. Ask them independently to produce a normalized transcript, speaker turns, summary propositions, action items, decision-critical facts, and intended CRM associations. Hide Gong output, vendor identity, seller performance labels, and the other annotator’s work.
Define normalization before scoring: casing, punctuation, filled pauses, numbers, abbreviations, partial words, overlapping speech, inaudible spans, names, and product terms. Preserve both a readable transcript and the normalized scoring text. Otherwise, one evaluator may count “twenty thousand” and “20,000” as equal while another counts two substitutions.
Calibrate on a small set before the main corpus. Measure raw agreement for categorical labels and an appropriate chance-adjusted statistic, such as Cohen’s kappa, where its assumptions fit. Review disagreements, refine definitions, then lock the codebook. Adjudicate the production reference without silently rewriting the rules. Report pre-adjudication agreement, unresolved audio, and adjudicated changes so the “human truth” is not presented as infallible.
Create a claim register for critical facts: exact wording, normalized value, speaker, timestamp, evidence span, confidence, and CRM target. Examples include price, quantity, renewal date, legal entity, security constraint, buyer commitment, seller promise, decision owner, and next meeting. Negation and uncertainty must remain attached: “not approved” cannot become “approved,” and “may renew” cannot become “will renew.”
Measure transcript and speaker accuracy
Use word error rate for transcript comparability, then add business-aware measures. NIST’s OpenASR evaluation uses word error rate (WER) as a primary speech-recognition metric. Calculate WER = (substitutions + deletions + insertions) / reference words after applying the locked normalization. Publish the numerator, denominator, and distribution by stratum—not only one blended average.
WER treats every reference word similarly. That makes it useful but insufficient for revenue workflows. Also score critical-token accuracy for names, domains, product terms, amounts, currencies, dates, negation, and commitments. Report unscorable audio separately. A system should not receive a perfect score for guessing an inaudible amount, nor be punished as if it clearly misheard one.
For speaker attribution, score each reference utterance as correctly attributed, incorrectly attributed, unknown, or split/merged beyond the codebook rule. Report attributed-utterance accuracy and critical speaker reversals. If the seller’s offer is assigned to the buyer, the commercial meaning can invert even when every word is correct. Teams using a standard diarization error rate may add it, but should keep the business-facing attribution table.
Break results down by language, selected-language condition, audio path, noise, overlap, call length, number of speakers, and critical vocabulary. Suppress or combine tiny groups where reporting could identify people. Do not compare groups with different call difficulty without stating the imbalance. The purpose is to find operational limits, not to make claims about speakers.
Score summaries, action items, and CRM fields
Score generated outputs as propositions, not polished paragraphs. Split the human reference and Gong summary into atomic propositions: who did what, to whom, when, with which status or uncertainty. Match each generated proposition to reference evidence. Summary precision is supported generated propositions divided by generated propositions. Summary recall is required reference propositions recovered divided by required reference propositions. Preserve “unsupported,” “contradicted,” “wrong speaker,” and “wrong certainty” as separate error labels.
Use the same method for action items, adding owner, action, due date, status, source span, and whether the commitment was explicit or merely suggested. An elegant action item with the wrong owner or invented date is not a partial success when it could trigger a workflow. Count any fabricated commitment, reversed negation, wrong material amount/date, or buyer/seller reversal as a critical error according to the frozen codebook.
For CRM fields, compare each proposed or written value with the reference CRM state and mapping contract. Field precision equals correct written values divided by all written values. Field recall equals required correct values written divided by required reference values. Also report omission, unsupported write, stale overwrite, formatting failure, permission failure, and correct value on the wrong object.
Gong documents that CRM output depends on the CRM and configuration. Its CRM guide describes Salesforce tasks or events and a Gong Conversation object with the relevant app, while its Salesforce connection guide describes integration-user authorization and import/export. Validate the exact contracted objects, fields, editions, and permissions in a sandbox. Do not generalize a Salesforce result to another CRM.
Test association, idempotency, and retries
An accurate sentence attached to the wrong deal is a system failure. Seed ambiguous and adverse cases in the CRM sandbox: duplicate contacts, shared email domains, parent and subsidiary accounts, simultaneous opportunities, a merged record, changed owner, missing required field, invalid picklist, revoked token, rate limit, timeout before write, timeout after write, partial batch failure, late event, replayed webhook, and concurrent human edit.
For each case, log source call ID, event ID, attempt number, timestamp, target object and record, proposed change, prior value, result, error, retry, final value, and audit entry. Association accuracy is correct target associations divided by scored associations. Report a separate wrong-record rate because one destructive misassociation can matter more than many harmless omissions.
Test idempotency by delivering the same accepted event twice and after a simulated timeout. The final state should contain one intended activity or value, not duplicates. Test retry boundaries with retryable and non-retryable errors, exponential delays where documented, out-of-order delivery, and expired credentials. Verify that a partial failure is visible and recoverable rather than silently treated as complete.
Start read-only. Then allow writes only to sandbox objects with least privilege, approval, field-level allowlists, conflict behavior, audit, rollback, and a kill switch. Follow the CRM integration best-practices guide for the broader control model. Production access comes after the write test, not before it.
Freeze acceptance gates before the pilot
Thresholds must come from workflow risk, not the observed score. Before processing the corpus, owners from revenue operations, sales, privacy, security, data, and CRM administration sign the gates. Keep hard gates pass/fail; do not let a high average compensate for a privacy incident or wrong opportunity update.
| Gate | Buyer must define | Evidence |
|---|---|---|
| Capture and consent | Approved recording paths, exclusions, retention, access, deletion; zero tolerated prohibited captures | Configuration, call log, consent evidence, deletion test |
| Transcript | Maximum WER and critical-token error by required stratum | Locked corpus, SCTK or reproducible scoring, denominators |
| Speaker attribution | Minimum attributed-utterance accuracy and maximum critical reversals | Turn-level reference and error log |
| Generated facts | Minimum precision/recall; maximum critical errors and unsupported commitments | Proposition register with source spans |
| CRM integrity | Minimum field and association accuracy; maximum destructive writes | Sandbox before/after states and audit trail |
| Reliability | Required duplicate prevention, retry recovery, alerting, rollback, and incident response | Injected failures and replay logs |
Define three decisions: accept for a bounded use, remediate and retest, or reject. A system might pass searchable transcript use while failing automatic CRM field writes. That is a valid bounded decision. Document exclusions, required human review, correction SLA, override authority, monitoring cadence, and the event that forces revalidation, such as a model, language, connector, mapping, or permission change.
Use the worked scoring example
The following arithmetic teaches the method; it is not a Gong result. Suppose an illustrative normalized reference contains 10 words. The candidate transcript has one substitution, one deletion, and no insertion. WER is (1 + 1 + 0) / 10 = 20%. Whether that passes depends on the threshold frozen for that stratum and whether either error is commercially critical.
Now suppose the reference contains eight required CRM facts. The system writes seven values and six are correct. Illustrative field precision is 6 / 7 = 85.7%; recall is 6 / 8 = 75%. If the wrong value is a harmless formatting variant under the codebook, it may be acceptable. If it changes the opportunity amount or writes to the wrong account, it may trigger the critical-error gate regardless of the averages.
For a corpus of 40 illustrative calls, if two calls contain at least one codebook-defined critical error, the call-level critical-error rate is 2 / 40 = 5%. Keep both call-level and error-level counts. One call can contain several errors, and reporting only one denominator can hide concentration. Never tune the gate after seeing these illustrative calculations applied to real output.
Printable result row: call ID · consent state · language/dialect label · audio path · reference words · S/D/I · WER · critical-token errors · attributed utterances/correct · summary TP/FP/FN · action-item errors · CRM field TP/FP/FN · target association · duplicate/retry result · critical error · reviewer · adjudication · decision.
Run the pilot and calculate full TCO
Run baseline, controlled processing, failure injection, and monitored use as separate phases. In phase one, measure current human transcription, note, and CRM effort without Gong. In phase two, process the frozen corpus with no production writes. In phase three, run sandbox association, permissions, duplicate, retry, rollback, deletion, and incident tests. In phase four, allow a bounded user group and approved use case with mandatory review. Keep the same definitions throughout.
Track WER and critical-token error by stratum, speaker attribution, summary and action-item precision/recall, CRM field and association accuracy, critical-error rate, unscorable rate, correction minutes, review minutes, latency, duplicate writes, failed retries, incidents, overrides, opt-outs, and adoption of the bounded workflow. Report confidence intervals or at least sample counts; a perfect result on three calls is not stable evidence.
Full three-year TCO equals Gong licenses and required editions + CRM or conferencing dependencies + implementation + security/privacy/legal review + corpus creation + double annotation and adjudication + integration and mapping + sandbox and test automation + human review + correction + monitoring + incident response + administration + training + support + retention/export/deletion + parallel systems and exit − tools and labor demonstrably retired. Use contracted quotes and observed pilot labor, not public list-price guesses.
Renew only if the bounded use remains inside its gates and total correction, governance, and incident work fits the case. Re-run the stratified sample after material configuration or model changes and on a fixed production-monitoring cadence. The useful conclusion is not “Gong is accurate.” It is: “This documented configuration passed these predeclared tests for these calls and uses, with these limits.”