How SynthID Text watermark detection works

The short answer

SynthID Text is a generative statistical watermark. During generation, keyed pseudorandom functions score possible next tokens. Tournament Sampling gently favors candidates with stronger watermark values. During detection, the text is tokenized again and scored with the matching tokenizer, algorithm configuration, and secret keys. The detector does not need the original language model.

Watermark Tidy exactly reproduces one pinned Google DeepMind reference implementation on synthetic token fixtures. The live checker now tokenizes pasted text locally and runs that engine under an explicit Watermark Tidy test profile. This produces real reference statistics, not a Claude verdict: Anthropic confirms Claude uses a version of SynthID Text and offers detection in private preview, but has not published the compatible tokenizer, keys, algorithm profile, thresholds, or test vectors.

The method in four steps

What is a g-value?

A g-value is a keyed pseudorandom score for a candidate token in a particular recent context. In the paper's binary example, each function returns 0 or 1. Tournament Sampling repeatedly compares sampled token candidates and favors the candidate with the higher value at each layer. This creates a correlation between chosen tokens and the secret watermark functions without inserting hidden Unicode characters.

What does the detector score?

The basic Mean Score averages the g-values across usable token positions and watermark layers. The published method expects watermarked text to score higher on average under the correct key. A score alone is not a verdict: a production detector needs calibrated thresholds for the exact profile, text length, and operating error rates.

The paper's simplified score is shown below. When repeated-context masking is enabled, masked positions are omitted and the mean is calculated over the remaining evidence.

Mean Score = (1 / mT) · Σ_t Σ_ℓ g_ℓ(x_t, r_t)

What the Nature paper found

The 2024 paper, Scalable watermarking for identifying large language model outputs, presents SynthID Text as a production-oriented system designed around three constraints: detection strength, text quality, and low computational overhead. It modifies token sampling rather than model training, and detection does not require access to the underlying LLM.

• Production scale: the authors evaluated feedback from nearly 20 million watermarked and unwatermarked Gemini responses. The observed differences in thumbs-up and thumbs-down rates were statistically insignificant.
• Quality: automated benchmarks and a separate human comparison of 3,000 paired responses did not provide sufficient evidence of a quality difference for the non-distortionary configuration studied.
• Detection: the experiments reported stronger detectability than the compared generative watermark baselines, while performance varied with text length, model entropy, sampling settings, and watermark strength.
• Editing: deletion and paraphrasing weakened detection. Longer text retained more evidence, while strong paraphrasing remained a substantial attack.
• Abstention: the supplement describes a selective prediction region between positive and negative thresholds when short or low-entropy text cannot meet the required error rates reliably.

These are empirical results for the paper's Google configurations and evaluation data. They do not establish that another provider uses the same tokenizer, keys, sampling depth, hash variant, or calibration.

Why the exact profile matters

The paper uses a recent-context window of H = 4 and commonly reports m = 30 tournament layers for its non-distortionary experiments. Those are paper values, not known Claude settings. Public reference keys are likewise test data, not Anthropic keys.

The official Hugging Face Transformers implementation uses a seeded sampling table and a different hash initialization. We track it under a separate algorithm ID and do not silently treat it as equivalent to the DeepMind 0.2.1 variant.

What Watermark Tidy has implemented

We built a small native JavaScript detector foundation for the pinned google-deepmind/synthid-text package 0.2.1 at commit addb4a158143c7c6851a1308f78b89fceed59683. Its algorithm ID is google-deepmind/synthid-text@0.2.1:sha256-lcg-v1.

• Exact integer behavior: signed 64-bit overflow and hashing match the pinned reference.
• Token contexts: the engine rebuilds n-gram g-values and the repeated-context mask.
• Scores: Mean Score and the reference Weighted Mean Score are reproduced.
• Differential oracle: 7 named fixtures and 64 deterministically seeded token sequences match official reference output.
• Controls: correct-key, wrong-key, unmarked, repeated-context, short, long, and edited-token cases remain distinct.
• Fail-closed profile: anthropic/claude-pending contains no keys or detector and always returns unavailable.

In the committed oracle fixtures, the correct-key marked sequence has a Mean Score of 0.6989, while the same tokens under the wrong key score 0.4810 and the unmarked control scores 0.5017. That ordering is a deterministic implementation check only, not a detection threshold or real-world accuracy estimate.

Three separate decision paths

The same word—detector—can hide very different questions. Watermark Tidy keeps the verified reference calculation, an experimental keyless indicator, and real Claude verification separate so that one signal cannot silently become another claim.

The keyless experiment is not exposed in the browser checker. It uses checked-in token fixtures only, never user text, and reports no single-text verdict. Its null result is useful: it shows that a plausible statistical footprint does not automatically become a reliable detector once the secret profile is removed.

How to read a detector result correctly

• A raw or mean score is not probability. It is an aggregate statistic under one configured key and algorithm.
• A p-value is not the probability that Claude wrote the text. It describes compatibility with a statistical null model under stated assumptions.
• No mark found would not prove human authorship. The text may be unsupported, too short, low-entropy, edited, generated without watermarking, or checked with the wrong profile.
• A positive signal would not provide complete provenance. It would support the presence of a compatible mark, not explain every later edit or establish sole authorship.
• Abstention is a valid result. A reliable detector should say when the available evidence cannot satisfy its error targets.

For that reason, Watermark Tidy labels the displayed Mean and Weighted Mean as reference statistics rather than a confidence meter. It deliberately implements no Claude threshold, p-value, or Bayesian classifier without provider-compatible ground truth.

What this means for Claude

Anthropic now confirms that Claude will use a version of the SynthID Text approach. Its explanation matches the central decision-path idea: a secret key and recent text shape the randomness used to choose among plausible continuations, and evidence builds as those keyed choices accumulate. This confirms the method family, not compatibility with Watermark Tidy's exact DeepMind reference profile.

Activation requires a compatible Anthropic tokenizer or tokenization contract, detector algorithm, keys or verification interface, official positive and negative test vectors, and calibration across length, language, entropy, edits, and repetition. Until those gates are reproducible, the reference engine remains verified and the Claude profile remains pending.

A future Anthropic release would be tested first against official positive and negative controls. It would then become a new, explicitly versioned profile; neither today's DeepMind reference score nor the keyless experiment would be relabelled as Claude evidence.