Intercepting LLM to transform every other token reveals surprising robustness

4 months ago 5

A real-time LLM stream interceptor for token-level interaction research

Every Other Token is a research tool that intercepts OpenAI's streaming API responses and applies transformations to alternating tokens in real-time. Instead of waiting for complete responses, it intervenes at the token level, creating a new paradigm for LLM interaction and analysis.

This tool opens up novel research possibilities:

** Token Dependency Analysis**: Study how LLMs handle disrupted token sequences
** Interpretability Research**: Understand token-level dependencies and causality
** Creative AI Interaction**: Build co-creative systems with human-AI token collaboration
** Real-time LLM Steering**: Develop new prompt engineering techniques
** Stream Manipulation**: Explore how semantic meaning degrades with token alterations

# Clone the repository git clone https://github.com/Mattbusel/Every-Other-Token.git cd every-other-token # Install dependencies pip install openai # Set your OpenAI API key export OPENAI_API_KEY='your-api-key-here'

# Simple example python every_other_token.py "Tell me a story about a robot" # With specific transformation python every_other_token.py "Explain quantum physics" uppercase # With different model python every_other_token.py "Write a haiku" mock gpt-4

The script intercepts the OpenAI streaming API response and applies transformations based on token position:

Even tokens (0, 2, 4, 6...): Passed through unchanged
Odd tokens (1, 3, 5, 7...): Transformed using the selected method

Original: "The quick brown fox jumps over the lazy dog" With reverse transform: "The kciuq brown xof jumps revo the yzal dog"

Available Transformations

Transform Description Example

reverse	Reverses odd tokens	"hello" → "olleh"
uppercase	Converts odd tokens to uppercase	"hello" → "HELLO"
mock	Creates alternating case (mocking text)	"hello" → "hElLo"
noise	Adds random characters to odd tokens	"hello" → "hello*"

1. Token Dependency Studies

# Study how meaning degrades with token corruption python every_other_token.py "Solve this math problem: 2+2=" reverse

2. Semantic Robustness Testing

# Test how well LLMs maintain coherence under disruption python every_other_token.py "Continue this story logically..." noise

3. Creative Collaboration

# Use transformations to create unexpected creative outputs python every_other_token.py "Write a poem about nature" mock

python every_other_token.py [PROMPT] [TRANSFORM] [MODEL]

PROMPT: Your input prompt (required)
TRANSFORM: Transformation type (default: reverse)
MODEL: OpenAI model (default: gpt-3.5-turbo)

# Basic usage python every_other_token.py "Hello world" # Specific transform python every_other_token.py "Explain AI" uppercase # Different model python every_other_token.py "Creative writing" mock gpt-4 # All parameters python every_other_token.py "Technical explanation" noise gpt-4-turbo

The tool provides detailed statistics:

EVERY OTHER TOKEN INTERCEPTOR Transform: reverse Model: gpt-3.5-turbo Prompt: Tell me about AI ================================================== Response (with transformations): AI si a daorb field fo computer ecneics that... ================================================== Complete! Processed 156 tokens. Transform applied to 78 tokens.

Causality Testing: How does corrupting early tokens affect later generation?
Semantic Drift: At what corruption level does meaning break down?
Model Comparison: How do different models handle token disruption?
Domain Analysis: Which topics are most/least robust to token corruption?

Recursive Mutation: Feed transformed output back as input
Multi-Model Chains: Use tokens from different models alternately
Human-in-the-Loop: Replace odd tokens with human input
Bidirectional Analysis: Compare forward vs backward token importance

The tool includes comprehensive error handling:

API Key Validation: Checks for valid OpenAI API key
Network Error Recovery: Handles connection issues gracefully
Invalid Transform Detection: Validates transformation types
Model Availability: Checks if requested model exists

We welcome contributions! Here are ways to get involved:

New Transformations: Add creative token transformation functions
Analysis Tools: Build utilities for analyzing output patterns
Visualization: Create tools to visualize token-level changes
Documentation: Improve examples and research applications

# Fork and clone git clone https://github.com/yourusername/every-other-token.git cd every-other-token # Create virtual environment python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate # Install development dependencies pip install -r requirements-dev.txt # Run tests python -m pytest tests/

Research Papers & Citations

If you use this tool in academic research, please cite:

@software{every_other_token, title={Every Other Token: Real-time LLM Stream Interceptor}, author={Your Name}, year={2025}, url={https://github.com/yourusername/every-other-token} }

Web Interface: Browser-based tool for easier experimentation
Batch Processing: Process multiple prompts simultaneously
Export Functionality: Save results in various formats (JSON, CSV)
Visualization Dashboard: Real-time charts and analysis
Custom Transformations: User-defined transformation functions
Multi-API Support: Extend to other LLM providers (Anthropic, Cohere)
Collaborative Mode: Multiple users contributing tokens
Research Templates: Pre-built experiments for common research patterns