TikTok detects duplicates better than you think
Here's how to beat the system 🧵
– File hash: every file has a digital fingerprint. If you upload the exact same file (no edits), TikTok can detect it instantly by comparing the hash. – Visual/audio fingerprinting: TikTok uses AI to analyze frames and audio. Even if you cut 1 second or add text, it can still recognize the content. – Metadata: things like creation date, device name, or codec can give clues. These are easy to change though, so they're not the main method. – Audio matching: if your video uses a soundtrack or voice already on the platform, TikTok can detect it through audio fingerprinting.
How to avoid detection (aka not get flagged as a duplicate):
– Slightly crop or zoom the frame (e.g. zoom to 103%) – Add overlay text (even a single invisible or tiny word) – Use filters or tweak color settings (contrast, saturation, etc.) – Change playback speed just a bit (e.g. 0.95x or 1.05x) – Add a small watermark or dot in a corner
– Shift the audio pitch up/down slightly – Add background noise (ambient sounds, white noise, etc.) – Use TikTok's voiceover feature for a few seconds – Cut or offset the audio by 0.2s to break the pattern
– Re-export the video using CapCut, Premiere, etc. – Edit the metadata using tools like HandBrake or InShot
To avoid detection: modify at least 2-3 layers (visual, audio, metadata). Don't just trim a second or flip the video, go deeper.
This repository contains a Python script (video_processor.py) and a Streamlit-based Graphical User Interface (video_gui.py) designed to process video files (e.g., downloaded from platforms like Instagram or TikTok) to prepare them for re-uploading to these or other platforms. The primary goal is to make these videos technically unique, which can help reduce the risk of being flagged or "shadowbanned" for being simple re-uploads, regardless of the source or destination platform (e.g., Instagram to TikTok, TikTok to Instagram, TikTok to TikTok, Instagram to Instagram).
The video_processor.py script handles the core FFmpeg processing, while video_gui.py provides an easy-to-use interface for it.
The video_processor.py script uses FFmpeg (a powerful open-source multimedia framework) to perform a series of transformations on input video files:
- Input Source: Processes .mp4 video files located in a videos/ subfolder.
- Specific File Processing: Can process all .mp4 files in the videos/ folder or a single specified video file using the -f or --file command-line argument.
- Output Destination: Saves the processed videos into a treated/ subfolder, prefixing each filename with tt_.
- Output Folder Cleaning: Automatically clears all contents of the treated/ folder before each processing run to ensure a fresh set of output files.
- Metadata Removal: Strips all existing metadata (e.g., Exif data, original creation timestamps, software tags) from the input videos using the -map_metadata -1 FFmpeg option. This helps remove traces of the video's origin.
- Resizing & Aspect Ratio:
- Resizes videos to a standard 1080x1920 resolution (9:16 vertical aspect ratio), which is optimal for TikTok.
- Uses a combination of scale and pad filters in FFmpeg to ensure the video fits these dimensions, adding black bars if the original aspect ratio doesn't match (without cropping content).
- Sets the Sample Aspect Ratio (SAR) to 1:1 for compatibility.
- Video Trimming: Trims videos to a maximum duration of 29 seconds (-t 29), adhering to common short-form video limits and further altering the file from its original.
- Visual Adjustments:
- Applies a very slight brightness and contrast adjustment (eq=brightness=0.005:contrast=1.005) to subtly change the video's visual data.
- Performs a subtle 3 % centre-zoom (crop) to shift pixel positions enough to change TikTok's visual hash while remaining imperceptible to viewers.
- Adds a virtually invisible 2 × 2 px white dot in the top-left corner of every frame to further alter the bitmap without affecting user experience.
- Applies a random hue shift of up to ±5 ° and injects light film-grain noise (strength 4-8) to perturb colour histograms and texture uniformly across the clip.
- Applies a subtle lens distortion (k1≈0.01) to introduce barrel-style warping that is imperceptible on phone screens yet lowers frame similarity.
- Horizontal Flip (Optional): Can horizontally flip the video using the --hflip command-line argument. This is another common technique to make content appear different to detection algorithms.
- Video Encoding:
- Uses the libx264 codec (H.264) for video encoding, which is widely compatible.
- Employs CRF (Constant Rate Factor) encoding (randomly between 21-25) instead of a fixed bitrate. This adjusts bitrate dynamically to maintain consistent visual quality, which also helps in making the video file signature more unique compared to fixed-bitrate encodes.
- Audio Re-encoding & Manipulation:
- Re-encodes the audio stream to the AAC (Advanced Audio Coding) codec (-c:a aac).
- Sets an audio bitrate of 192k (-b:a 192k) for good quality stereo audio.
- Applies a slight pitch shift (~3 %) via asetrate/aresample to alter the audio fingerprint without changing tempo.
- Offsets the audio track by 200 ms (adelay) to further break direct alignment with original material.
- Automatic Background Noise: If a file named background_noise.mp3 exists in the sounds/ directory, it is automatically mixed in as very low-volume background noise. This adds another layer of audio uniqueness. The noise audio is looped and its volume is significantly reduced. If the file is not found, the script automatically generates a low-volume white noise track that serves the same purpose, ensuring all videos have this audio uniqueness feature applied.
- Cross-Platform Compatibility: The script is designed to be compatible with both macOS and Windows, provided Python 3 and FFmpeg are correctly installed and accessible. It includes logic to try and find the FFmpeg executable.
- Graphical User Interface (GUI): A video_gui.py script using Streamlit provides a user-friendly way to interact with the video processor. Features include:
- Drag-and-drop uploading of up to 10 .mp4 video files at a time (default was 5, updated to 10 as per current GUI code).
- A global checkbox to enable/disable horizontal video flipping for all processed videos in a batch.
- Per-Video Text Overlay Customization: For each uploaded video, you can individually:
- Enable or disable text overlay.
- Set the text content.
- Choose text position (Top Center, Middle Center, Bottom Center).
- Define font size.
- Specify text color.
- Set a background color for the text (can be semi-transparent or "none").
- Choose text style: Bold and/or Italic. (Requires providing Roboto static font files in the fonts/Roboto/static/ directory for reliable styling; see Font Handling section).
- Per-Video Rotation: For each uploaded video, you can set a rotation angle (in degrees, from -45 to +45, default is 0). Positive values rotate clockwise. Even a small rotation (0.5 to 2 degrees) can significantly lower SSIM scores by altering pixel structure, further reducing the chance of being flagged as duplicate content. Rotated areas are filled with black.
- Per-Video Playback Speed: Each clip now has its own speed slider (0.5×–1.5×). The script adjusts video PTS and chains atempo filters so audio pitch stays natural, avoiding the historical grey-screen bug.
- Progress bar during processing.
- Download a .zip file containing all processed videos.
Social media platforms like TikTok use algorithms to detect and sometimes deprioritize content that appears to be a direct, unaltered re-upload from other platforms or sources. By systematically modifying various technical aspects of the video, this script helps to:
- Create a Unique File Signature: Changing metadata, resolution, bitrate, encoding parameters for both video and audio, duration, and optionally adding subtle background noise results in a file that is technically different from the original.
- Reduce Automated Detection: These alterations can make it less likely for automated systems to flag the video as a simple duplicate.
- Optimize for TikTok: Resizing to the preferred 9:16 aspect ratio ensures the video looks good on the platform.
While no script can guarantee that a video won't be subject to platform algorithms, these steps significantly enhance the "uniqueness" of the video file itself, which is a good practice when repurposing content.
-
Prerequisites:
- Python 3: Ensure Python 3 is installed on your system.
- FFmpeg: Ensure FFmpeg is installed and accessible in your system's PATH, or the script will attempt to guide you if it can't find it in common locations.
- Dependencies: Install required Python packages:
pip install -r requirements.txt
- Optional Background Noise File: To enable custom background noise mixing, create a sounds/ directory in the same location as the script, and place an audio file named background_noise.mp3 inside it. If no custom noise file is provided, the script will automatically generate a subtle white noise track.
-
Setup:
- Place the .mp4 video files you want to process into a folder named videos/ in the same directory as the video_processor.py script (this is mainly for the command-line version, the GUI uses direct uploads).
-
Running the Application:
There are two ways to use the video processor:
A) Using the Graphical User Interface (Recommended for ease of use):
- Open your terminal or command prompt.
- Navigate to the project directory.
- Font Handling for Text Overlays (Important!): For best results with text overlays, especially for Bold and Italic styles using the default Roboto font, ensure you have the following static .ttf (TrueType Font) files in the fonts/Roboto/static/ subfolder of your project:
- Roboto-Regular.ttf (for regular text)
- Roboto-Bold.ttf (for bold text)
- Roboto-Italic.ttf (for italic text)
- Roboto-BoldItalic.ttf (for bold and italic text) These files should be obtained from a font provider like Google Fonts (download the "static" versions, not variable fonts for this purpose). If these specific files are not found, the script will attempt to use system default fonts (like Helvetica on macOS or Arial on Windows), which may not support all styles or look as intended. The script will provide warnings in the console if it cannot find the requested styled Roboto fonts.
- Run the GUI:
python3 -m streamlit run video_gui.py(If python3 is not found, try python -m streamlit run video_gui.py)
- The GUI will open in your web browser. Drag and drop your videos, select options, and click "Process Videos".
- After processing, a download button will appear to get a .zip file of the treated videos.
B) Using the Command-Line Script (video_processor.py):
- Open your terminal or command prompt.
- Navigate to the directory where video_processor.py is located.
- To process all videos in the videos/ folder:
python3 video_processor.py
- To process a specific video file (e.g., my_video.mp4) located in the videos/ folder:
python3 video_processor.py -f my_video.mp4
- To process a specific video and flip it horizontally:
python3 video_processor.py -f my_video.mp4 --hflip
- To process all videos and flip them horizontally:
python3 video_processor.py --hflip
- To process a specific video and mix in background noise from sounds/background_noise.mp3:
python3 video_processor.py -f my_video.mp4 --noise_file sounds/background_noise.mp3
- To process all videos and mix in background noise:
python3 video_processor.py --noise_file sounds/background_noise.mp3
-
Output:
- The processed videos will be saved in a folder named treated/, with each filename prefixed by tt_.
This script was developed iteratively, adding features based on common requirements for video processing, particularly for social media:
- Initial version focused on basic metadata removal, resizing, and trimming.
- Enhanced to include slight visual adjustments (brightness/contrast) and explicit video bitrate.
- Further improved by adding audio re-encoding instead of just copying the audio stream.
- Made more user-friendly by adding command-line options for single-file processing and auto-clearing of the output directory.
- Added optional background noise mixing (automatically detected from sounds/background_noise.mp3) for further audio differentiation.
- Addressed and fixed bugs, such as an initial gray screen issue caused by a previous speed adjustment filter (which has since been removed).
- Added a Streamlit-based GUI (video_gui.py):
- Allows drag-and-drop of up to 10 video files (updated from 5).
- Provides an option for horizontal flipping.
- Introduced per-video customizable text overlays:
- Toggle overlay on/off per video.
- Set custom text, position, font size, color, and background color.
- Options for Bold and Italic text styles (reliant on user-provided font files for best results).
- Added per-video configurable rotation to further decrease SSIM and enhance uniqueness.
- Displays processing progress.
- Enables downloading processed files as a .zip archive.
When you process videos via video_gui.py a SSIM (%) column appears for each file:
- SSIM (Structural Similarity Index) compares every frame of the original input against the processed output, then averages the results.
- The GUI rescales both videos to 1080 × 1920 first, so we always compare like-for-like frames.
- The value is reported as a percentage:
- ≈ 100 % → virtually identical (barely altered).
- 90 – 95 % → minor visual changes only (consider adding rotation or text overlay if aiming lower).
- 80 – 90 % → moderate changes (good starting point; using rotation of 0.5-2 degrees often lands here or lower).
- < 80 % → significant visual difference (lowest chance of duplicate-content flags; achieved with zoom + rotation).
TikTok's exact thresholds are unknown, but anecdotal evidence suggests:
- ≥ 95 %: Too high – platform likely treats it as the same clip.
- ≈ 80–90 %: Usually acceptable. To increase uniqueness, consider adding a slight rotation (0.5-2 degrees via the GUI) or enabling text overlay if not already used.
- < 80 %: Safe zone for most duplicate-detection systems. With the default Ken Burns zoom and a small rotation, scores often fall into this range or lower.
Remember that TikTok also fingerprints audio and metadata. The script already tweaks those layers (pitch-shift, background noise, metadata wipe), so even an 80 % visual SSIM can be fine in practice, especially if rotation or text overlays are also applied.
In short: An 80 % SSIM means the processed video still shares 80 % of its pixel structure with the source – it's changed by about 20 %. That is generally OK, but lower scores provide a wider safety margin. Using the rotation feature is a good way to push scores lower.
Quick guideline ▶︎ Aim for an average SSIM of ≤ 80 % (≈ 0.80) before uploading repurposed videos to TikTok. Scores in the 70–80 % range are generally safe while preserving perceived quality.
The script always applies a baseline set of safety tweaks (metadata wipe, resize/pad, slight brightness/contrast, random hue shift ±5 °, light film-grain noise, subtle lens distortion, micro Ken-Burns zoom 1.00 → 1.10, and a 2 × 2 px dot).
With only these defaults you can expect SSIM in the 90–95 % range—enough for light uniqueness without visible degradation.
To push SSIM lower (≈ 60-80 %) enable one or more of the following in the GUI:
• Increase the End zoom scale slider (e.g. 1.15 – 1.25).
• Add a small rotation (0.5–2 ° is usually plenty).
• Horizontally flip the video.
• Overlay text/watermark or change playback speed.
Combining two or three of these options typically lands well below the 80 % threshold while keeping quality high. Feel free to experiment; the SSIM column updates per file so you can see the effect immediately.
This table summarizes the features suggested in the specs document and their current status in this project:
General | File Hash Modification | ✅ Implemented | Achieved via re-encoding and multiple subtle/optional visual and audio changes. |
Visual/Audio Fingerprinting | ✅ Implemented | Addressed by various visual (zoom, rotate, color, grain, lens) & audio (pitch, noise, offset) modifications. | |
Metadata Removal | ✅ Implemented | -map_metadata -1 strips all metadata. | |
Re-export Video | ✅ Implemented | Videos are fully re-encoded using libx264 (CRF 21-25) and aac. | |
Visual Changes | Slightly crop or zoom the frame | ✅ Implemented | Default Ken Burns (1.0 → 1.10 zoom). GUI slider for end-zoom up to 2.0 (capped at 1.18 in ssim_score_improvement.md plan). |
Add overlay text | ✅ Implemented | GUI: Per-video text, position, size, color, background, bold/italic. | |
Use filters or tweak color settings | ✅ Implemented | Automatic: Brightness/contrast (+0.005), Hue Shift (±5°). | |
Flip the video horizontally | ✅ Implemented | Optional via CLI (--hflip) or global GUI checkbox. | |
Change playback speed just a bit | ✅ Implemented | GUI: Per-video speed slider (0.5x – 1.5x), affects video and audio (pitch preserved). | |
Add a small watermark or dot in a corner | ✅ Implemented | Automatic: 2x2px white dot (top-left). Text overlay can serve as watermark. | |
(Implicit) Resize to 1080x1920 | ✅ Implemented | Automatic: All videos scaled/padded to 1080x1920. | |
(Implicit) Trim video duration | ✅ Implemented | Automatic: Videos trimmed to 29 seconds. | |
(Internal) Lens Distortion | ✅ Implemented | Automatic: Subtle lens correction (k1=k2 ≈ 0.008-0.02). | |
(Internal) Film Grain | ✅ Implemented | Automatic: Light film grain noise (strength 4-8). | |
Audio Changes | Shift the audio pitch up/down slightly | ✅ Implemented | Automatic: Pitch shifted by ~3% (asetrate). |
Add background noise | ✅ Implemented | Automatic: Mixes sounds/background_noise.mp3 or generated white noise. | |
Use TikTok's voiceover feature | ❌ Not Implemented | Requires TikTok app integration or separate audio track generation/mixing, which is outside the current script's scope. | |
Cut or offset the audio by 0.2s | ✅ Implemented | Automatic: Audio delayed by 200ms (adelay). | |
From SSIM Plan | Randomised X/Y Pan Path (Zoompan) | 🟡 Partially Done | GUI-driven zoom uses random X/Y offsets, but default zoom is center-panned. Plan suggested more dynamic pan path. |
"Uniqueness Strength" Knob (GUI) | ❌ Not Implemented | A single slider to scale multiple effects (zoom, rotation, hue, noise) as per ssim_score_improvement.md MVP. | |
Stronger Brightness/Contrast Options | ❌ Not Implemented | Plan mentioned eq ±2–3% as high-leverage; current is fixed at +0.5%. Not a GUI option yet. | |
Specific GUI Rotation Limits & Default | ❌ Not Implemented | Plan suggested GUI rotation slider limited to ±1° (default 0.7°); current is ±45° (default 0°). | |
Temporal Jitter (Frame Drop/Duplication) | ❌ Not Implemented | Stretch Idea: minterpolate, tblend to drop/duplicate 1-2% of frames. | |
Per-Frame Random Grain Mask | ❌ Not Implemented | Stretch Idea: Procedurally generate 2-frame-periodic random noise. |
Features listed under "Status in Project" as "Automatic" are applied to all videos by default without needing user input. Others are available via GUI or CLI options.