EzDevInfo.com

tweepy documentation [closed]

I just began working on a little twitter-app using tweepy.

is there any kind of useful (and complete) documentation for tweepy?

I googled like hell but didn't find anything.

How can I consume tweets from Twitter's streaming api and store them in mongodb

I need to develop an app that lets me track tweets and save them in a mongodb for a research project (as you might gather, I am a noob, so please bear with me). I have found this piece of code that sends tweets streaming through my terminal window:

import sys
import tweepy

consumer_key=""
consumer_secret=""
access_key = ""
access_secret = "" 


auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)

class CustomStreamListener(tweepy.StreamListener):
    def on_status(self, status):
        print status.text

    def on_error(self, status_code):
        print >> sys.stderr, 'Encountered error with status code:', status_code
        return True # Don't kill the stream

    def on_timeout(self):
        print >> sys.stderr, 'Timeout...'
        return True # Don't kill the stream

sapi = tweepy.streaming.Stream(auth, CustomStreamListener())
sapi.filter(track=['Gandolfini'])

Is there a way I can modify this piece of code so that instead of having tweets streaming over my screen, they are sent to my mongodb database?

Thanks

Traceback when updating status on twitter via Tweepy

I've been trying to post the readings from my Rpi on Twitter using tweepy, but first I wanted to check if tweepy was working properly, but it's not.

I installed the packages properly, but when I'm trying to run a simple code to post something, I got an error (Yes, I already created an app and have the 4 credentials).

The code I'm trying to run:

import tweepy
consumer_key = ''
consumer_secret = ''
access_token = ''
access_token_secret = ''

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
single_tweet = 'hello world'
api.update_status(single_tweet)
print "successfully Updated"

I got this:

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
  File "build/bdist.linux-armv6l/egg/tweepy/api.py", line 193, in update_status
  File "build/bdist.linux-armv6l/egg/tweepy/binder.py", line 239, in _call
  File "build/bdist.linux-armv6l/egg/tweepy/binder.py", line 223, in execute
tweepy.error.TweepError: [{u'message': u'media_ids parameter is invalid.', u'code': 44}]

I'm runing the python code that is in the tweepy folder "oauth.py" (adding it my credentials)

$ sudo python oauth.py
RapiCARA
Traceback (most recent call last):
  File "oauth.py", line 34, in <module>
    api.update_status(' Hello world ')
  File "build/bdist.linux-armv6l/egg/tweepy/api.py", line 193, in update_status
  File "build/bdist.linux-armv6l/egg/tweepy/binder.py", line 239, in _call
  File "build/bdist.linux-armv6l/egg/tweepy/binder.py", line 223, in execute
tweepy.error.TweepError: [{u'message': u'media_ids parameter is invalid.', u'code': 44}]

The code in that file, you can find it in its origial source: Tweepy in GITHUB

Any help/advice please?

No module named pip.req

I am installing tweepy, but I am running into an error about pip.req. I have pip installed, but for some reason pip.req still can't be found. I did a bunch of research online and the most I could find was some issue about incompatibilities between zapo (?) and python 2.7 causing the same error for some other user. The discussion was unclear about how to solve the problem, though. Thanks!

$ python2 setup.py install
Traceback (most recent call last):
  File "setup.py", line 5, in <module>
    from pip.req import parse_requirements
ImportError: No module named pip.req

How to add a location filter to tweepy module

I have found the following piece of code that works pretty well for letting me view in Python Shell the standard 1% of the twitter firehose:

import sys
import tweepy

consumer_key=""
consumer_secret=""
access_key = ""
access_secret = "" 


auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)


class CustomStreamListener(tweepy.StreamListener):
    def on_status(self, status):
        print status.text

    def on_error(self, status_code):
        print >> sys.stderr, 'Encountered error with status code:', status_code
        return True # Don't kill the stream

    def on_timeout(self):
        print >> sys.stderr, 'Timeout...'
        return True # Don't kill the stream

sapi = tweepy.streaming.Stream(auth, CustomStreamListener())
sapi.filter(track=['manchester united'])

How do I add a filter to only parse tweets from a certain location? Ive seen people adding GPS to other twitter related Python code but I cant find anything specific to sapi within the Tweepy module.

Any ideas?

Thanks

return actual tweets in tweepy?

I was writing a twitter program using tweepy. When I run this code, it prints the Python ... values for them, like

<tweepy.models.Status object at 0x95ff8cc>

Which is not good. How do I get the actual tweet?

import tweepy, tweepy.api
key = XXXXX
sec = XXXXX

tok  = XXXXX
tsec = XXXXX

auth = tweepy.OAuthHandler(key, sec)
auth.set_access_token(tok, tsec)
api = tweepy.API(auth)

pub = api.home_timeline()
for i in pub:
        print str(i)

How to "see" the structure of an object in python

I am wokring in python 2.7. I have been experimenting with the tweepy package. There is an object called the tweepy.models.status object, whose function is defined here: https://github.com/tweepy/tweepy/blob/master/tweepy/models.py.

I have a function that looks like this:

    def on_status(self, status):
        try:
            print status
            return True
        except Exception, e:
            print >> sys.stderr, 'Encountered Exception:', e
            pass

The object I am referring to is the one returned from the on_status function, called status. When the print status line executes i get this printed on the screen;

tweepy.models.Status object at 0x85d32ec>

My question is actually pretty generic. I want to know how to visually print out the contents of this status object? I want to know what information is available inside this object.

I tried the for i, v in status : approach, but it says this objects is not iterable. Also not all of the object attributes are described in the function definition.

Thanks a lot!

Tweepy public stream filter by a changing variable

I was working with the Tweepy library for python to access the public twitter stream and ran into a problem where one the stream is running, it doesn't stop. Now, that makes sence for what it does, but I wanted it to start filtering with an empty list of user IDs and after a while, userIDs are added to the list after someone tweets a particulate track word, so that once they tweet a word, the tracker starts tracking all their tweets. The problem is that once the stream is started with the initial filter options, changing the variables doesn't affect the filter; it just keeps on using the initial arguments.

userIDs = []

trackWords = ["#Obama"]

def stream():

    s = Stream(auth, StreamListener())

    s.filter(follow = userIDs, track = trackWords)

I was able to get around this earlier by recalling the stream definition again after a new keyword is added, but I have multiple streams searching and I put them in separate threads so they can all run simultaneously. I can't figure out how to refresh the threads, so trying to refresh the filter without recalling the definition seems easier.

I'm fairly new to programming, so maybe this is a fundamental concept I don't know yet, but hopefully there's an easy trick to get it to refresh.

Here's all my relevant code if that helps anyone. The above was just a quick thing to help show what I'm talking about:

userIDs = []
userNames = []

account = ['@DMS_423']

publicKeyWords = ['the','be','to','of','and','are','is','were','was']

class AStreamListener(StreamListener):
    def on_status(self, status):
        if status.author.screen_name not in userNames:
            userNames.append(str(status.author.screen_name))
            userIDs.append(str(api.get_user(str(status.author.screen_name)).id))
            print status.author.screen_name, "has joined the game."

def uStream():
    s = Stream(auth, StreamListener())
    s.filter(follow = userIDs)

def pStream():
    ps = PStream(pAuth, PStreamListener())
    ps.filter(track = publicKeyWords)

def aStream():
    adds = Stream(auth, AStreamListener())
    adds.filter(track = account)

t1 = Thread(target = aStream)
t2 = Thread(target = uStream)
t3 = Thread(target = pStream)

def run():
    t1.start()
    t2.start()
    t3.start()

run()

how to properly close a tweepy stream

I'm trying to figure out how to properly close an asynchronous tweepy stream.

The tweepy streaming module can be found here.

I start the stream like this:

stream = Stream(auth, listener)
stream.filter(track=['keyword'], async=True)

When closing the application, I try to close the stream as simple as:

stream.disconnect()

This method seems to work as intended but it seems to have one problem: the stream thread is still in the middle of the loop (waiting/handling tweets) and is not killed until the next loop, so when the stream receives a tweet even after the app has closed, it still tries to call the listener object (this can be seen with a simple print syntax on the listener object). I'm not sure if this is a bad thing or if it can simply be ignored.

I have 2 questions:

1. Is this the best way to close the stream or should I take a different approach?
2. Shouldn't the async thread be created as a daemon thread?

tweepy/ twitter api error type

I am using tweepy to make a twitter application. When users tweet/update profile, etc, they will get some errors. I want to classify error and give user more information.

try:
    tweet/update profile/ follow....

except tweepy.TweepError, e:

    if tweepy.TweepError is "Account update failed: Description is too long (maximum is 160 characters)"
        Do something
    if tweepy.TweepError is "Failed to send request: Invalid request URL: http://api.twitter.com/1/account/update_profile.json?location=%E5%85%B5%E5%BA%A"
        Do something
    if tweepy.TweepError is "[{u'message': u'Over capacity', u'code': 130}]" 
        Do something

Is the only way to classify error is to compare e with string, for example, Account update failed: Description is too long (maximum is 160 characters)?

Remaining limits error

I am trying to find a way, to create a remain limits in tweepy, in order to handle remaining limits error that twitter API sends. I am using the following code:

limits = myapi.rate_limit_status()
remain_search_limits = limits['resources']['search']['/search/tweets']['remaining']
print remain_search_limits
print limits

Actually the above code prints:

179
{u'rate_limit_context': {u'access_token': u'179927437-vtHojLnksgUD3FfnueTWVag62Kx5mvAA3uhX7qd7'}, u'resources': {u'account': {u'/account/verify_credentials': {u'reset': 1399886205, u'limit': 15, u'remaining': 0}, u'/account/settings': {u'reset': 1399886857, u'limit': 15, u'remaining': 15}}, u'blocks': {u'/blocks/list': {u'reset': 1399886857, u'limit': 15, u'remaining': 15}, u'/blocks/ids': {u'reset': 1399886857, u'limit': 15, u'remaining': 15}}, u'users': {u'/users/contributors': {u'reset': 1399886857, u'limit': 15, u'remaining': 15}, u'/users/lookup': {u'reset': 

The weird issue, is that every time that I print remain_search_limits I am getting 179 as answer. What should I have to use in order to waint until I ll be able again to fetched data from twitter?

My final code is the following:

limits = api.rate_limit_status()
remain_follower_limits = limits['resources']['followers']['/followers/ids']  ['remaining']

with open("ids.txt") as f:
    content = f.readlines()

ids = []
for page in tweepy.Cursor(api.followers_ids, user_id="dankanter").pages():
while remain_follower_limits >4:
    limits = api.rate_limit_status()
    remain_follower_limits = limits['resources']['followers']['/followers/ids']['remaining']
    ids.extend(page)
else:
    time.sleep(60)

print ids
print len(ids)

However sometimes it stucks in the line limits = api.rate_limit_status() inside while loop. In case that a user has millions of users it returns only a subset which actually is very small.

Avoid twitter api limitation with Tweepy

I saw in some thread in stack exchange that the limitation can be in function of the number of requests per 15 minutes and depends also on the complexity of the alghorithm, except that this is not a complex one .

So I use this code :

import tweepy
import sqlite3
import time

db = sqlite3.connect('data/MyDB.db')
# Get a cursor object
cursor = db.cursor()
cursor.execute('''CREATE TABLE IF NOT EXISTS MyTable(id INTEGER PRIMARY KEY, name TEXT, geo TEXT, image TEXT, source TEXT, timestamp TEXT, text TEXT, rt INTEGER)''')
db.commit()

consumer_key = ""
consumer_secret = ""
key = ""
secret = ""

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(key, secret)

api = tweepy.API(auth)

search = "#MyHashtag"

for tweet in tweepy.Cursor(api.search,
                           q=search,
                           include_entities=True).items():
    while True: 
        try:
            cursor.execute('''INSERT INTO MyTable(name, geo, image, source, timestamp, text, rt) VALUES(?,?,?,?,?,?,?)''',(tweet.user.screen_name, str(tweet.geo), tweet.user.profile_image_url, tweet.source, tweet.created_at, tweet.text, tweet.retweet_count))
        except tweepy.TweepError:           
                time.sleep(60 * 15)
                continue        
        break       
db.commit() 
db.close()

I am always having the twitter limitation error :

Traceback (most recent call last):
  File "stream.py", line 25, in <module>
    include_entities=True).items():
  File "/usr/local/lib/python2.7/dist-packages/tweepy/cursor.py", line 153, in next
    self.current_page = self.page_iterator.next()
  File "/usr/local/lib/python2.7/dist-packages/tweepy/cursor.py", line 98, in next
    data = self.method(max_id = max_id, *self.args, **self.kargs)
  File "/usr/local/lib/python2.7/dist-packages/tweepy/binder.py", line 200, in _call
    return method.execute()
  File "/usr/local/lib/python2.7/dist-packages/tweepy/binder.py", line 176, in execute
    raise TweepError(error_msg, resp)
tweepy.error.TweepError: [{'message': 'Rate limit exceeded', 'code': 88}]

Tweepy (Twitter API) Not Returning all Search Results

I'm using the search feature with Tweepy for Twitter and for some reason the search results are limited to 15. Here is my code

results=api.search(q="Football",rpp=1000)

for result in results:
    print "%s" %(clNormalizeString(result.text))

print len(results)

and only 15 results are returned. Does it have something to do with different pages of results or something?

How can I retrieve all Tweets and attributes for a given user using Python?

I am attempting to retrieve data from Twitter, using Tweepy for a username typed at the command line. I'm wanting to extract quite a bit of data about the status and user,so have come up with the following:

Note that I am importing all the required modules ok and have oauth + keys (just not included it here) and filename is correct, just been changed:

# define user to get tweets for. accepts input from user
user = tweepy.api.get_user(input("Please enter the twitter username: "))

# Display basic details for twitter user name
print (" ")
print ("Basic information for", user.name)
print ("Screen Name:", user.screen_name)
print ("Name: ", user.name)
print ("Twitter Unique ID: ", user.id)
print ("Account created at: ", user.created_at)

timeline = api.user_timeline(screen_name=user, include_rts=True, count=100)
    for tweet in timeline:
        print ("ID:", tweet.id)
        print ("User ID:", tweet.user.id)
        print ("Text:", tweet.text)
        print ("Created:", tweet.created_at)
        print ("Geo:", tweet.geo)
        print ("Contributors:", tweet.contributors)
        print ("Coordinates:", tweet.coordinates) 
        print ("Favorited:", tweet.favorited)
        print ("In reply to screen name:", tweet.in_reply_to_screen_name)
        print ("In reply to status ID:", tweet.in_reply_to_status_id)
        print ("In reply to status ID str:", tweet.in_reply_to_status_id_str)
        print ("In reply to user ID:", tweet.in_reply_to_user_id)
        print ("In reply to user ID str:", tweet.in_reply_to_user_id_str)
        print ("Place:", tweet.place)
        print ("Retweeted:", tweet.retweeted)
        print ("Retweet count:", tweet.retweet_count)
        print ("Source:", tweet.source)
        print ("Truncated:", tweet.truncated)

I would like this eventually to iterate through all of a user's tweets (up to the 3200 limit). First things first though. So far though I have two problems, I get the following error message regarding retweets:

Please enter the twitter username: barackobamaTraceback (most recent call last):
  File " usertimeline.py", line 64, in <module>
    timeline = api.user_timeline(screen_name=user, count=100, page=1)
  File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
    raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 401
Traceback (most recent call last):
  File "usertimeline.py", line 42, in <module>
    user = tweepy.api.get_user(input("Please enter the twitter username: "))
  File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
    raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 404

Passing the username as a variable seems to be a problem also:

Traceback (most recent call last):
  File " usertimleline.py", line 64, in <module>
    timeline = api.user_timeline(screen_name=user, count=100, page=1)
  File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
    raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 401

I've isolated both these errors, i.e. they aren't working together.

Forgive my ignorance, I am not too hot with Twitter APIs but am learning pretty rapidly. Tweepy documentation really does suck and I've done loads of reading round on the net, just can't seem to get this fixed. If I can get this sorted, i'll be posting up some documentation.

I know how to transfer the data into an MySQL db once extracted (it will do that, rather than print to screen) and manipulate it so that I can do stuff with it, it is just getting it out that I am having the problems with. Does anyone have any ideas or is there another method I should be considering?

Any help really appreciated. Cheers

EDIT:

Following on from @Eric Olson's suggestion this morning; I did the following.

1) Created a completely brand new set of Oauth credentials to test. 2) Copied code across to a new script as follows:

Oauth

consumer_key = "(removed)"
consumer_secret = "(removed)"
access_key="88394805-(removed)"
access_secret="(removed)"
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api=tweepy.API(auth)



# confirm account being used for OAuth
print ("API NAME IS: ", api.me().name)
api.update_status("Using Tweepy from the command line")

The first time i run the script, it works fine and updates my status and returns the API name as follows:

>>> 
API NAME IS:  Chris Howden

Then from that point on I get this:

Traceback (most recent call last):
  File "C:/Users/Chris/Dropbox/Uni_2012-3/6CC995 - Independent Studies/Scripts/get Api name and update status.py", line 19, in <module>
    api.update_status("Using Tweepy frm the command line")
  File "C:\Python32\lib\site-packages\tweepy-1.4-py3.2.egg\tweepy\binder.py", line 153, in _call
    raise TweepError(error_msg)
tweepy.error.TweepError: Twitter error response: status code = 403

The only reason I can see for it doing something like this is that it is rejecting the generated access token. I shouldn't need to renew the access token should I?

urllib.request in Python 2.7

I can use urllib.request module with Python 3.1. But when I execute the same program using Python 2.7, an error comes along the lines of;

AttributeError: 'module' object has no attribute 'request'.

I believe this error is because theres no request module in urllib for Python 2.7. Because I need to use tweepy i'll have to stick with Python 2.7 since tweepy does not support Python 3.

So how I can use urllib.request module in Python 2.7?