How SynthID Text watermark detection works

The short answer

SynthID Text is a generative statistical watermark. During generation, keyed pseudorandom functions score possible next tokens. Tournament Sampling gently favors candidates with stronger watermark values. During detection, the text is tokenized again and scored with the matching tokenizer, algorithm configuration, and secret keys. The detector does not need the original language model.

Watermark Tidy now contains a research-only engine that exactly reproduces one pinned Google DeepMind reference implementation on synthetic token fixtures. It does not make the live website a SynthID or Claude detector: Anthropic has not published the compatible Claude tokenizer, keys, algorithm profile, thresholds, or test vectors.

The method in four steps

What is a g-value?

A g-value is a keyed pseudorandom score for a candidate token in a particular recent context. In the paper's binary example, each function returns 0 or 1. Tournament Sampling repeatedly compares sampled token candidates and favors the candidate with the higher value at each layer. This creates a correlation between chosen tokens and the secret watermark functions without inserting hidden Unicode characters.

What does the detector score?

The basic Mean Score averages the g-values across usable token positions and watermark layers. The published method expects watermarked text to score higher on average under the correct key. A score alone is not a verdict: a production detector needs calibrated thresholds for the exact profile, text length, and operating error rates.

The paper's simplified score is shown below. When repeated-context masking is enabled, masked positions are omitted and the mean is calculated over the remaining evidence.

Mean Score = (1 / mT) · Σ_t Σ_ℓ g_ℓ(x_t, r_t)

What the Nature paper found

The 2024 paper, Scalable watermarking for identifying large language model outputs, presents SynthID Text as a production-oriented system designed around three constraints: detection strength, text quality, and low computational overhead. It modifies token sampling rather than model training, and detection does not require access to the underlying LLM.

• Production scale: the authors evaluated feedback from nearly 20 million watermarked and unwatermarked Gemini responses. The observed differences in thumbs-up and thumbs-down rates were statistically insignificant.
• Quality: automated benchmarks and a separate human comparison of 3,000 paired responses did not provide sufficient evidence of a quality difference for the non-distortionary configuration studied.
• Detection: the experiments reported stronger detectability than the compared generative watermark baselines, while performance varied with text length, model entropy, sampling settings, and watermark strength.
• Editing: deletion and paraphrasing weakened detection. Longer text retained more evidence, while strong paraphrasing remained a substantial attack.
• Abstention: the supplement describes a selective prediction region between positive and negative thresholds when short or low-entropy text cannot meet the required error rates reliably.

These are empirical results for the paper's Google configurations and evaluation data. They do not establish that another provider uses the same tokenizer, keys, sampling depth, hash variant, or calibration.

Why the exact profile matters

The paper uses a recent-context window of H = 4 and commonly reports m = 30 tournament layers for its non-distortionary experiments. Those are paper values, not known Claude settings. Public reference keys are likewise test data, not Anthropic keys.

The official Hugging Face Transformers implementation uses a seeded sampling table and a different hash initialization. We track it under a separate algorithm ID and do not silently treat it as equivalent to the DeepMind 0.2.1 variant.

What Watermark Tidy has implemented

We built a small native JavaScript detector foundation for the pinned google-deepmind/synthid-text package 0.2.1 at commit addb4a158143c7c6851a1308f78b89fceed59683. Its algorithm ID is google-deepmind/synthid-text@0.2.1:sha256-lcg-v1.

• Exact integer behavior: signed 64-bit overflow and hashing match the pinned reference.
• Token contexts: the engine rebuilds n-gram g-values and the repeated-context mask.
• Scores: Mean Score and the reference Weighted Mean Score are reproduced.
• Differential oracle: 7 named fixtures and 64 deterministically seeded token sequences match official reference output.
• Controls: correct-key, wrong-key, unmarked, repeated-context, short, long, and edited-token cases remain distinct.
• Fail-closed profile: anthropic/claude-pending contains no keys or detector and always returns unavailable.

In the committed oracle fixtures, the correct-key marked sequence has a Mean Score of 0.6989, while the same tokens under the wrong key score 0.4810 and the unmarked control scores 0.5017. That ordering is a deterministic implementation check only, not a detection threshold or real-world accuracy estimate.

How to read a detector result correctly

• A raw or mean score is not probability. It is an aggregate statistic under one configured key and algorithm.
• A p-value is not the probability that Claude wrote the text. It describes compatibility with a statistical null model under stated assumptions.
• No mark found would not prove human authorship. The text may be unsupported, too short, low-entropy, edited, generated without watermarking, or checked with the wrong profile.
• A positive signal would not provide complete provenance. It would support the presence of a compatible mark, not explain every later edit or establish sole authorship.
• Abstention is a valid result. A reliable detector should say when the available evidence cannot satisfy its error targets.

For that reason, Watermark Tidy does not expose its synthetic fixture scores as a public confidence meter and deliberately implements no Claude threshold, p-value, or Bayesian classifier without provider-compatible ground truth.

What this means for Claude

Anthropic says supported Claude models embed an imperceptible mark during generation and that detection details will follow in technical documentation. Anthropic has not publicly confirmed that its method is compatible with the implemented DeepMind variant. The product-owner hypothesis that Claude may use a SynthID-compatible method is therefore a research hypothesis, not an Anthropic fact.

Activation requires a compatible Anthropic tokenizer or tokenization contract, detector algorithm, keys or verification interface, official positive and negative test vectors, and calibration across length, language, entropy, edits, and repetition. Until those gates are reproducible, Watermark Tidy keeps Claude at Not tested.