DeTeCtive: Rethinking AI-Generated Text Detection via Multi-Level Style Attribution

TL;DR

Existing AI text detectors often break when facing a new model like GPT-4 or Llama-3 because they treat "AI-generated" as a single, static category. DeTeCtive flips the script: it treats every LLM as a unique author with its own fingerprint. By using a multi-level contrastive learning framework and a training-free adaptation strategy, it achieves state-of-the-art accuracy and shows unprecedented resilience to out-of-distribution (OOD) domains and paraphrasing attacks.

The Motivating Insight: LLMs as Authors

Modern AI detection is a cat-and-mouse game. When a new LLM is released, old detectors (trained on GPT-2 or GPT-3) often fail because the "statistical signature" of the text has shifted. The authors of DeTeCtive argue that the problem lies in the Binary Classification Trap.

Instead of asking, "Is this human or AI?", they ask, "What is the stylistic kinship of this text?". By mapping writing styles into a high-dimensional feature space, they can identify the "familial" similarities between models from the same company (e.g., the Llama family) and the stark differences between human prose and machine output.

Methodology: Contrastive Kinship

The core of DeTeCtive is its Multi-task Auxiliary Multi-level Contrastive Learning.

1. Hierarchical Relationships: The model is trained to recognize four layers of distribution:
   • $P_1$: Text from a specific LLM.
   • $P_2$: Text from the same model series/company.
   • $P_3$: Text from any LLM.
   • $P_4$: Human-written text.
2. Multi-Level Contrastive Loss: Unlike standard contrastive learning that treats all "negatives" the same, DeTeCtive uses a weighted loss that forces the encoder to understand that a Llama-7B text is "closer" to a Llama-65B text than it is to a GPT-3 text, and much further from human text.
3. Dense Retrieval Inference: Instead of a simple Softmax layer, DeTeCtive uses a KNN-based retrieval pipeline. It pre-encodes the training set into a feature database. During inference, it searches for the nearest neighbors of the query text to determine its source.

Disruptive Feature: Training-Free Incremental Adaptation (TFIA)

Perhaps the most practical contribution is TFIA. In traditional systems, to detect a new model (e.g., Llama-3), you would need to fine-tune the entire detector. With DeTeCtive, you simply:

1. Generate/Collect a small sample of the new model's text.
2. Run them through the frozen DeTeCtive encoder.
3. Add these vectors to your retrieval database.

The system now "knows" what the new model looks like without a single gradient update.

Experimental Results: Setting New Benchmarks

The authors tested DeTeCtive against baselines like RoBERTa, DetectGPT, and Longformer across Deepfake, M4, and TuringBench datasets.

• Generalization: On "Unseen Domains" (OOD), DeTeCtive outperformed existing SOTA by 14.2% in Average Recall.
• Attack Robustness: Even under heavy paraphrasing attacks (DIPPER/OUTFOX), where most detectors' performance collapses, DeTeCtive maintained near-peak accuracy thanks to its fault-tolerant KNN mechanism.

Visualizing the Latent Space: UMAP projections show that while pre-trained models see a messy cloud of text, DeTeCtive creates clear, clustered islands for different model families and human sources.

Critical Analysis & Conclusion

Takeaway: DeTeCtive proves that style is a more stable signal than statistical probability when it comes to LLM detection. Its ability to adapt via TFIA makes it one of the few frameworks truly ready for the rapid-fire release cycle of modern LLMs.

Limitations:

• Database Scaling: As the feature database grows to millions of entries, retrieval latency might become an issue (though tools like Faiss mitigate this).
• Interpretability: While the clusters are clear, what specific stylistic tokens (syntax, vocabulary choices) the model is latching onto remains a "black box" for future research.

DeTeCtive represents a significant step forward in ensuring LLM compliance and information security in an era where the line between human and machine is increasingly blurred.