Retrieve Twitter posts and comments using Python
By: Queency in Python Tutorials on 2023-03-17
Here's some sample Python code that uses the Twitter API to collect tweets and replies and saves them in a dataset:
import tweepy import pandas as pd # set up the Twitter API credentials consumer_key = 'your_consumer_key' consumer_secret = 'your_consumer_secret' access_token = 'your_access_token' access_secret = 'your_access_secret' # authenticate with the Twitter API auth = tweepy.OAuthHandler(consumer_key, consumer_secret) auth.set_access_token(access_token, access_secret) api = tweepy.API(auth) # search for tweets containing a particular keyword or hashtag query = 'python' tweets = tweepy.Cursor(api.search_tweets, q=query, tweet_mode='extended').items(1000) # create a list to store the tweet and reply data data = [] # loop through each tweet and get its replies for tweet in tweets: tweet_data = { 'text': tweet.full_text, 'user': tweet.user.screen_name, 'created_at': tweet.created_at } replies = tweepy.Cursor(api.search_tweets, q='to:'+tweet.user.screen_name, since_id=tweet.id, tweet_mode='extended').items() for reply in replies: reply_data = { 'text': reply.full_text, 'user': reply.user.screen_name, 'created_at': reply.created_at, 'in_reply_to': tweet.id } data.append(reply_data) data.append(tweet_data) # convert the list of tweet and reply data into a Pandas DataFrame df = pd.DataFrame(data) # save the DataFrame as a CSV file df.to_csv('twitter_data.csv', index=False)
In this code, we first set up the Twitter API credentials using the values for consumer_key, consumer_secret, access_token, and access_secret. We then use Tweepy to authenticate with the Twitter API and search for tweets containing a particular keyword or hashtag (query).
We loop through each tweet that we find and use Tweepy to get its replies by searching for tweets that are directed to the same user and have a higher ID than the current tweet. We store the tweet and reply data in a list called data, which we then convert into a Pandas DataFrame and save as a CSV file.
The resulting CSV file will contain columns for the text of the tweet and reply, the user who posted the tweet and reply, the timestamp for when the tweet and reply were created, and the ID of the tweet that the reply is in response to.
Add Comment
This policy contains information about your privacy. By posting, you are declaring that you understand this policy:
- Your name, rating, website address, town, country, state and comment will be publicly displayed if entered.
- Aside from the data entered into these form fields, other stored data about your comment will include:
- Your IP address (not displayed)
- The time/date of your submission (displayed)
- Your email address will not be shared. It is collected for only two reasons:
- Administrative purposes, should a need to contact you arise.
- To inform you of new comments, should you subscribe to receive notifications.
- A cookie may be set on your computer. This is used to remember your inputs. It will expire by itself.
This policy is subject to change at any time and without notice.
These terms and conditions contain rules about posting comments. By submitting a comment, you are declaring that you agree with these rules:
- Although the administrator will attempt to moderate comments, it is impossible for every comment to have been moderated at any given time.
- You acknowledge that all comments express the views and opinions of the original author and not those of the administrator.
- You agree not to post any material which is knowingly false, obscene, hateful, threatening, harassing or invasive of a person's privacy.
- The administrator has the right to edit, move or remove any comment for any reason and without notice.
Failure to comply with these rules may result in being banned from submitting further comments.
These terms and conditions are subject to change at any time and without notice.
- Data Science
- Android
- React Native
- AJAX
- ASP.net
- C
- C++
- C#
- Cocoa
- Cloud Computing
- HTML5
- Java
- Javascript
- JSF
- JSP
- J2ME
- Java Beans
- EJB
- JDBC
- Linux
- Mac OS X
- iPhone
- MySQL
- Office 365
- Perl
- PHP
- Python
- Ruby
- VB.net
- Hibernate
- Struts
- SAP
- Trends
- Tech Reviews
- WebServices
- XML
- Certification
- Interview
categories
Related Tutorials
Python program to get location meta data from an image
Retrieve Twitter posts and comments using Python
How to install Jupyter in Ubuntu and make it accessible through Apache Reverse Proxy
Python Basics - Setting up your Python Development Environment
Schwartzian Transform in python
Multidimensional list (array) in python
Perl's chomp() equivalent for removing trailing newlines from strings in python
Comments