.png)
1 month ago
5
Where technology speaks, and you feel the echo
A comprehensive keyword tracking platform that monitors technology trends across Hacker News, Reddit, and YouTube. Analyze keyword popularity, track trends, and discover insights across multiple tech communities.
Click the image above to watch the full demo
- Multi-platform tracking: Hacker News, Reddit, YouTube
- Trend visualization: Monthly data trends with interactive charts
- Cross-platform insights: Compare keyword performance across platforms
- Top contributors: Identify key voices in each community
- Hacker News: 2-year historical dataset with stories and comments
- Reddit: All-time search across subreddits
- YouTube: Video metadata, views, and engagement metrics
- Discord: Community data (limited to accessible groups)
- Total mentions and unique authors
- Monthly trend analysis
- Interaction counts (upvotes, comments, views)
- Platform-specific insights
- Python 3.8+
- API keys for Reddit and YouTube (optional for demo)
# Clone the repository
git clone https://github.com/witch-Judy/BuzzScope.git
cd BuzzScope
# Install dependencies
pip install -r requirements.txt
# Configure environment (optional)
cp env.example .env
# Edit .env with your API keys
# Launch the main application
streamlit run app_new_keyword_test.py --server.port 8502
Visit http://localhost:8502 to access the application.
The application comes with pre-analyzed data for:
- ai - Artificial Intelligence discussions
- iot - Internet of Things topics
- mqtt - MQTT protocol discussions
- unified_namespace - Industrial automation concepts
- Enter keywords in the sidebar (comma-separated)
- Click "Collect Missing Data" for new keywords
- View real-time analysis and trends
- Explore cross-platform insights
- Trend Analysis: Monthly mention trends with raw data tables
- Platform Comparison: Side-by-side keyword performance
- Top Contributors: Most active users per platform
- Interactive Charts: Zoom, filter, and explore data
- Uses 2-year historical parquet dataset
- Analyzes stories, comments, and user interactions
- Covers 7.6M+ records from 2022-2024
- Searches across all subreddits
- Collects posts, comments, and metadata
- Uses Reddit's public JSON API
- Searches video titles and descriptions
- Collects view counts, likes, and comments
- Uses YouTube Data API v3
BuzzScope/
├── app_new_keyword_test.py # Main Streamlit application
├── src/
│ ├── collectors/ # Data collection modules
│ ├── analyzers/ # Analysis engines
│ └── visualization/ # Chart generation
├── data/
│ ├── cache/ # Processed data cache
│ └── Hackernews_raw/ # Hacker News historical data
└── requirements.txt # Dependencies
- Visit Reddit App Preferences
- Create a new app (script type)
- Add credentials to .env:
REDDIT_CLIENT_ID=your_client_id
REDDIT_CLIENT_SECRET=your_client_secret
REDDIT_USER_AGENT=your_app_name/1.0
- Go to Google Cloud Console
- Enable YouTube Data API v3
- Create API key
- Add to .env:
YOUTUBE_API_KEY=your_api_key
The application requires a 2-year Hacker News dataset for full functionality:
Option 1: Download from Google
- Search for "Hacker News 2 years dataset" on Google
- Download the parquet file to ./data/Hackernews_raw/
Option 2: Contact the Author
- The author can provide the dataset upon request
- Contact via GitHub issues or email
Option 3: Use API Only
- The application will work with real-time API data only
- Limited to recent posts and comments
- Caching: Pre-processed metrics for fast loading
- Charts: Pre-generated HTML charts for instant display
- Storage: JSON-based cache with Parquet for large datasets
- Performance: Optimized for real-time analysis
- Hacker News: Official API + 2-year historical dataset (7.6M+ records)
- Reddit: Public JSON API (no authentication required)
- YouTube: Data API v3 with quota management
- Discord: Community data (requires access to specific groups)
- Fork the repository
- Create a feature branch
- Make your changes
- Submit a pull request
MIT License - see LICENSE file for details.
For questions or issues:
- Create an issue on GitHub
- Check the troubleshooting section
- Review the documentation
Start tracking technology trends today
