Scrape Twitter / X with AI

# Scrape Twitter / X — SoloStack Skill File
> Drop this file into your SoloStack repo and Claude Code will use it as
> context to run this workflow automatically.
---
## Overview
Extract tweets, user profiles, follower data, and trending topics from Twitter/X using Apify and Claude Code.
**Apify Actor:** `apidojo/tweet-scraper`
**Pricing:** ~$0.60 per 1,000 tweets
## Capabilities
1. **Keyword & Hashtag Search** — Scrape tweets matching any keyword, phrase, or hashtag with date range filters. Find every public conversation about your brand, product, or industry.
2. **Account Timeline Scraping** — Extract the full tweet history of any public account — original tweets, replies, retweets, and quote tweets with complete engagement data.
3. **Thread & Reply Extraction** — Capture full conversation threads including all replies and nested discussions. Essential for understanding context and sentiment depth.
4. **Engagement Metrics** — Pull likes, retweets, quote tweets, reply counts, bookmark counts, and impression estimates for every tweet.
5. **Author Profile Data** — Extract tweet author profiles — follower count, bio, location, verification status, and account creation date for credibility assessment.
6. **Media & Link Extraction** — Capture embedded images, videos, GIFs, and external links shared in tweets for content analysis and competitive research.
## Pipeline
1. **Define queries** — Set up keyword searches, @mention tracking, hashtag monitoring, or specific account scraping with date ranges and filters
2. **Scrape tweets** — Apify extracts tweet text, author profiles, engagement metrics, media, reply threads, and quote tweets
3. **Analyze sentiment** — Run sentiment analysis on scraped tweets to gauge brand perception, identify complaints, and spot emerging narratives
4. **Alert & act** — Surface high-priority mentions for response, feed insights into marketing strategy, and archive data for trend analysis
## Use Cases
- **Sentiment analysis & PR monitoring:** Track every mention of your brand, product, or CEO in real-time. Detect PR crises early, measure campaign reception, and identify your most vocal advocates and critics.
- **Audience research:** Understand what your target audience talks about, who they follow, and what content resonates. Use tweet data to build audience personas grounded in actual behavior, not surveys.
- **Trending topic discovery:** Monitor emerging conversations in your industry before they hit mainstream. Identify rising hashtags, viral threads, and shifting narratives to inform timely content creation.
- **Competitive intelligence:** Track competitor announcements, product launches, and customer feedback in real-time. Analyze their content strategy, posting frequency, and audience engagement patterns.
## Warnings
- Twitter/X frequently changes their anti-scraping measures. Expect occasional 24-48 hour periods where scraping throughput is reduced while Apify updates their actors.
- Tweet impressions data is only available for recent tweets (last 30 days). Older tweets will have likes, retweets, and replies but not impression counts.
- Be cautious with large keyword searches — common terms like 'AI' can return millions of results. Always use date ranges and engagement minimums to filter results to a manageable volume.
## Sample Output
```json
{
  "text": "Just shipped our AI agent pipeline and it's wild. 100 cold emails drafted in 2 minutes. Reviewed and sent in 5. This is 10x leverage, not 10% improvement.",
  "author": {
    "username": "techfounder",
    "displayName": "Alex Rivera",
    "followers": 45200,
    "isVerified": true,
    "bio": "Building the future of sales automation. CEO @streamlineai. YC W24.",
    "location": "San Francisco, CA"
  },
  "metrics": {
    "likes": 1247,
    "retweets": 234,
    "quoteTweets": 56,
    "replies": 89,
    "bookmarks": 312,
    "impressions": 187000
  },
  "timestamp": "2026-03-04T16:45:00Z",
  "media": [],
  "urls": ["https://streamlineai.com/blog/ai-agents"],
  "hashtags": ["AIagents", "salesautomation"],
  "inReplyTo": null,
  "isRetweet": false,
  "isQuoteTweet": false,
  "tweetUrl": "https://x.com/techfounder/status/1234567890"
}
```
Download the complete skill file with full implementation details,
agent prompts, and ready-to-run scripts.