Twint is an advanced Twitter scraping tool written in Python that allows for scraping Tweets from Twitter profiles without using Twitter's API. Twint utilizes Twitter's search operators to let you scrape Tweets from specific users, scrape Tweets relating to certain topics, hashtags & trends, or sort out sensitive information from Tweets like e-mail and phone numbers. I find this very useful, and you can get really creative with it too.
You can install Twint in three ways. Which one to use is up to your needs, if you don't know which one to use, running
pip3 install twint is suggested.
git clone https://github.com/twintproject/twint.git
pip3 install -r requirements.txt
pip3 install twint
twint -u username- Scrape all the Tweets from
twint -u username -s pineapple- Scrape all Tweets from the
user's timeline containing pineapple.
twint -s pineapple- Collect every Tweet containing
pineapplefrom everyone's Tweets.
twint -u username --year 2019- Collect Tweets that were tweeted
twint -u username --since 2018-12-20- Collect Tweets that were tweeted since 2018-12-20.
twint -u username -o file.txt- Scrape Tweets and save to file.txt.
twint -u username -o file.csv --csv- Scrape Tweets and save as a csv file.
twint -u username --email --phone- Show Tweets that might have phone numbers or email addresses.
twint -s "John Cohen" --verified- Display Tweets by verified users that Tweeted about John Cohen.
twint -g="48.880048,2.385939,1km" -o file.csv --csv- Scrape Tweets from a radiusof 1km around a place in Paris and export them to a csv file.
twint -u username -es localhost:9200- Output Tweets to Elasticsearch
twint -u username -o file.json --json- Scrape Tweets and save as a json file.
twint -u username --database tweets.db- Save Tweets to a SQLite database.
twint -u username --followers- Scrape a Twitter user's followers.
twint -u username --following- Scrape who a Twitter user follows.
twint -u username --favorites- Collect all the Tweets a user has favorited.
twint -u username --following --user-full- Collect full user information a person follows
twint -u username --profile-full- Use a slow, but effective method to gather Tweets from a user's profile (Gathers ~3200 Tweets, Including Retweets).
twint -u username --retweets- Use a quick method to gather the last 900 Tweets (that includes retweets) from a user's profile.
twint -u username --resume 10940389583058- Resume a search starting from the specified Tweet ID.
More detail about the commands and options are located in the wiki