Ask HN: Has anyone verified that 'offline' AI tools are offline?

# Testing "Offline" AI Claims: Unexpected Network Behavior in llama.cpp

## Summary While testing various "offline" AI solutions under strict network isolation, I discovered unexpected network activity that contradicts documentation claims. These findings are reproducible and suggest many "local AI" tools maintain hidden network dependencies.

## Key Findings

### 1. Hidden Network Requests During Model Loading *Observation*: llama.cpp generates unexplained outbound network requests even after removing network tools and disabling DNS resolution.

*Test Setup*: - Removed wget, curl, ping, netcat from container - Disabled apt and package managers - Set /etc/resolv.conf to 0.0.0.0 - Monitored with tcpdump/wireshark

*Result*: Quantized GGUF models still attempted external network access during initialization phase.

### 2. Environment-Dependent "Offline" Behavior *Observation*: Same model files behave differently across environments when truly isolated.

*Test Results*: - Ubuntu Docker base: Multiple hidden dependencies, frequent network attempts - Arch Linux (minimal): Cleaner behavior but still required manual intervention for full isolation - Model loading failed under strict airgap conditions despite valid file checksums

### 3. LM Studio vs Raw llama.cpp Discrepancy *Observation*: LM Studio succeeds where raw llama.cpp fails under identical isolation conditions.

*Analysis*: LM Studio appears to include hidden fallback mechanisms or patches that mask dependency failures, suggesting the underlying llama.cpp behavior requires network access for stable operation.

### 4. CUDA Library Network Dependencies *Observation*: CUDA-enabled builds show additional network activity through dynamically linked libraries.

*Technical Details*: - Network requests originate from CUDA runtime modules - Occurs even with CUDA toolkit supposedly "offline" - Activity persists despite stripping network utilities from system

## Reproducible Test Steps

### Basic Network Isolation Test ```bash # 1. Create isolated container docker run --network none -it ubuntu:20.04

# 2. Install llama.cpp dependencies offline # (transfer files via mounted volumes)

# 3. Monitor network attempts tcpdump -i any -n

# 4. Run llama.cpp with model ./main -m model.gguf -p "test prompt"

# 5. Observe unexpected network activity ```

### Advanced Isolation Test ```bash # 1. Disable all network interfaces ip link set dev eth0 down

# 2. Clear DNS resolution echo "0.0.0.0" > /etc/resolv.conf

# 3. Remove network tools rm /usr/bin/wget /usr/bin/curl

# 4. Test model loading with packet capture ./main -m model.gguf -p "hello" 2>&1 | tee output.log & tcpdump -w capture.pcap ```

## Documentation vs Reality

*Official Claims*: "llama.cpp enables LLM inference with minimal setup... locally"

*Observed Behavior*: - Models fail to load under true network isolation - Hidden dependencies on external resources - Environment-specific network requirements not documented

## Call for Verification

These findings need independent verification. If you work with local AI:

- Try running your setup under strict network isolation - Monitor network traffic during model loading - Share your packet captures and observations

*Tools for testing*: - `tcpdump -i any -n` (Linux) - Wireshark (cross-platform) - `netstat -tuln` (check listening ports)

## Technical Impact

For users seeking true privacy: - "Offline" AI may not be offline as advertised - Network isolation testing reveals hidden dependencies - Additional hardening required for air-gapped deployments

---

These are technical observations from testing popular AI tools under network isolation. Results are reproducible with standard network monitoring tools. Looking for community input and verification.