GAN-RFW: Using Generative Power to Detect the Machine

TL;DR

As LLMs become increasingly adept at mimicking human nuance, the "Turing Test" has moved to the silicon level. This paper proposes a novel framework called GAN-RFW, which repurposes a Generative Adversarial Network's discriminator to extract deep structural features from text and pairs it with a weighted Random Forest. The result? A staggering 99.60% accuracy in distinguishing human tweets from AI-generated ones.

Problem & Motivation: The Vanishing Boundary

The rapid diffusion of generative AI has created a double-edged sword: while helpful for productivity, it facilitates the mass production of misinformation. Current detection methods—ranging from simple statistical checks to BERT-based classifiers—often struggle with "short-form" content like tweets, where context is limited.

The author's core insight is that generative models themselves possess an inherent understanding of "artificial" patterns. By training a GAN on structured text representations, we can force the model to learn the mathematical signatures that separate human randomness from machine-calculated probability.

Methodology: The GAN-Based Feature Extractor

The methodology is structured into a four-stage pipeline that emphasizes structural representation over simple word-counting.

1. Multi-Dimensional Text Representation

Instead of simple embeddings, each tweet is converted into a square co-occurrence matrix consisting of:

• Upper Triangle: Pearson correlation (word dependencies).
• Lower Triangle: Raw co-occurrence counts.
• Main Diagonal: TF-IDF weights (word importance).

2. The Discriminator as a Feature Engine

The heart of the system is a GAN based on the UNet architecture. While typical GANs focus on generating new data, this work prioritizes the Discriminator. By training it to distinguish real matrices from noise-reconstructed ones, the model learns abstract, high-level features that traditional NLP methods miss.

Figure 1: The four-step workflow: Preprocessing, Matrix Representation, GAN Feature Extraction, and RF Detection.

3. Weighted Random Forest (RFW)

The extracted features (a weight vector of $11imes5imes128$) are fed into a Random Forest. Unlike standard RFs, this version uses a weighting mechanism where more accurate decision trees are given higher influence in the final vote, significantly reducing False Positives.

Experiments & Results: SOTA Performance

The model was tested against a massive dataset of 1.7 million tweets, including outputs from Gemini, GPT-4, Claude, and LLaMA.

• Accuracy: 99.60% (1.5% higher than IDEATE).
• Recall: 0.9978 (indicating it captures almost all AI content).
• Robustness: The model's accuracy increases as tweet length grows, but even at 50 characters, it remains highly competitive.

Figure 2: Average accuracy comparison showing the proposed GAN+RFW outperforming SeqXGPT, BERT, and IDEATE.

The confusion matrix analysis reveals that the model is equally effective at identifying human-generated text as it is at catching AI, a critical requirement for avoiding "shadow-banning" legitimate users on social platforms.

Critical Analysis & Conclusion

Why it works

The success of this approach lies in the Synergistic Integration. The GAN captures the "manifold" of machine-generated text, while the Weighted Random Forest provides the ensemble stability needed for high-stakes classification.

Limitations

• Computational Cost: Training GANs and optimizing ensemble weights is resource-intensive compared to lighter linear classifiers.
• Evolving LLMs: As LLMs are trained to specifically bypass detectors (Adversarial Training), this model will need continuous iterative updates.

Future Outlook

This work sets a precedent for adaptive detection. By utilizing generative architectures to police other generative models, researchers are building a self-regulating ecosystem that could finally bring accountability back to digital discourse.