EzDevInfo.com

get URL of any submission for subreddits

I am trying to use PRAW to get new posts from subreddits on Reddit. The following code snippet shows how I get new items on a particular subreddit.

Is there a way to also get the URL of the particular submission?

submissions = r.get_subreddit('todayilearned')
submission = submissions.get_new(limit=1)
sub = [str(x) for x in submission]
print sub

Reddit search API not giving all results

import praw

def get_data_reddit(search):
    username=""
    password=""
    r = praw.Reddit(user_agent='')
    r.login(username,password,disable_warning=True)
    posts=r.search(search, subreddit=None,sort=None, syntax=None,period=None,limit=None)
    title=[]
    for post in posts:
        title.append(post.title)
    print len(title)


search="stackoverflow"
get_data_reddit(search)

Ouput=953

Why the limitation?

1. Documentation mentions

We can at most get 1000 results from every listing, this is an upstream limitation by reddit. There is nothing we can do to go past this limit. But we may be able to get the results we want with the search() method instead.

Any workaround? I hoping someway to overcome in API, I wrote an scrapper for twitter data and find it to be not the most efficient solution.

Same Question:https://github.com/praw-dev/praw/issues/430 Please refer the aformentioned link for related discussion too.

Python code that works fine on my PC doesn't work on my Raspberry Pi

I recently bought a raspberry pi for many projects, one of which is reddit bots. I'm using PRAW to make the bot and it works perfectly on my desktop PC(Windows 8.1) but on my raspberry by (Raspbian) it doesn't seem to work. I've narrowed it down to these few lines in question.
Brief explanation of what I'm trying to do: Get the selftext of the post and split it by spaces into words then check if certain words are in the post.

text = submission.selftext.replace(","," ").encode("utf-8").lower().split()
for i in range (0, len(players)):
    player = players[i].lower()
    if player in text:
        print(player)

On my PC this works fine but on the Pi the if statement was never triggered even though I'm using the EXACT SAME POST for each of these tests.

If you are not familiar with PRAW, I'm splitting a string and looking for words inside it, where text is the array of words to look through and players is the dictionary.

Edit: The code does not produce any errors, it just doesn't trigger the if statement like it does on my desktop.

Edit #2: Seems that it works fine with hard-coded variable text and player. I left text hard-coded to include an entry from what should be in players but it would not find it which leads me to believe the error is in how I create the list players Below is the method that I use to create it.

def getPlayers():
    players = []
    with open("PlayerIDs.txt") as f:
        for line in f:
            players.append(line)
    return players

SOLUTION between Python 2 and 3 there were changes to how unicode and encoding in general worked and therefor the code would not run on python 2.x on my pi. I ended up installing python3 and using virtualenv to setup python 3 and it all works fine now :)

recursion max error when when using futures.ProcessPoolExecutor but not futures.ThreadPoolExecutor with PRAW wrapper

I am using this code to scrape an API:

submissions = get_submissions(1)
with futures.ProcessPoolExecutor(max_workers=4) as executor:
#or using this: with futures.ThreadPoolExecutor(max_workers=4) as executor:
    for s in executor.map(map_func, submissions):
        collection_front.update({"time_recorded":time_recorded}, {'$push':{"thread_list":s}}, upsert=True)

It works great/fast with threads but when I try to use processes I get a full queue and this error:

  File "/usr/local/lib/python3.4/dist-packages/praw/objects.py", line 82, in __getattr__
    if not self.has_fetched:
RuntimeError: maximum recursion depth exceeded
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.4/threading.py", line 868, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.4/concurrent/futures/process.py", line 251, in _queue_management_worker
    shutdown_worker()
  File "/usr/lib/python3.4/concurrent/futures/process.py", line 209, in shutdown_worker
    call_queue.put_nowait(None)
  File "/usr/lib/python3.4/multiprocessing/queues.py", line 131, in put_nowait
    return self.put(obj, False)
  File "/usr/lib/python3.4/multiprocessing/queues.py", line 82, in put
    raise Full
queue.Full

Traceback (most recent call last):
  File "reddit_proceses.py", line 64, in <module>
    for s in executor.map(map_func, submissions):
  File "/usr/lib/python3.4/concurrent/futures/_base.py", line 549, in result_iterator
    yield future.result()
  File "/usr/lib/python3.4/concurrent/futures/_base.py", line 402, in result
    return self.__get_result()
  File "/usr/lib/python3.4/concurrent/futures/_base.py", line 354, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

Note that originally the processes worked great and very fast for small data retrievals, but now they're not working at all. Is this a bug or what's going on that the PRAW object would cause a recursion error with Processes but not with Threads?

PRAW: How to get a reddit comment object with just the comment ID?

I'm working on a bot where I only have the comment IDs, e.g., t1_asdasd. I don't have access to the parent thread or anything. Can I pull the corresponding comment object with just the comment ID?

Using praw, how can I iterate over all of a user's comments? [closed]

I'm just wondering if there's a simple enough way to iterate over all comments made by a particular user (I want to check for a particular phrase). Any help is appreciated :)

PRAW: get_submission works inconsistently

The script gets the information from a specific comment on Reddit. It will work or not depending on the permalink given to it.

import praw
from pprint import pprint
clipboard = ['permalink']
com = r.get_submission(clipboard).comments[0]
pprint(vars(com))

This permalink works : http://www.reddit.com/r/redditgetsdrawn/comments/1tvjmr/i_got_a_new_job_recently_and_would_love_to_have/cec9wh4

This one doesn't : http://www.reddit.com/r/redditgetsdrawn/comments/1ts9hi/surprise_me_thanks_in_advance/cec0897?context=3

I have the feeling that the permalinks ending with ?context=3 or other unusual ends have particular characteristics that screw up the .get_submission.

Here's the error report :

Traceback (most recent call last):
  File "C:\Users\Aymeric\Python33\Scripts\brdg\brdg.py", line 14, in <module>
    com = r.get_submission(clipboard).comments[0]
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\__init__.py", line 875, in get_submission
    comment_sort=comment_sort)
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\decorators.py", line 320, in wrapped
    return function(cls, *args, **kwargs)
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\objects.py", line 837, in from_url
    s_info, c_info = reddit_session.request_json(url, params=params)
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\decorators.py", line 158, in wrapped
    return_value = function(reddit_session, *args, **kwargs)
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\__init__.py", line 476, in request_json
    response = self._request(url, params, data)
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\__init__.py", line 338, in _request
    cache_key = (normalize_url(request.url), tuple(key_items))
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\helpers.py", line 140, in normalize_url
    if url.endswith('.json'):
AttributeError: 'list' object has no attribute 'endswith'

Edit: Here's what i get when i change the list to a string :

Traceback (most recent call last):
  File "C:\Users\Aymeric\Python33\lib\json\decoder.py", line 368, in raw_decode
    obj, end = self.scan_once(s, idx)
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Aymeric\Python33\Scripts\brdg\brdg.py", line 14, in <module>
    com = r.get_submission(clipboard).comments[0]
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\__init__.py", line 875, in get_submission
    comment_sort=comment_sort)
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\decorators.py", line 320, in wrapped
    return function(cls, *args, **kwargs)
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\objects.py", line 837, in from_url
    s_info, c_info = reddit_session.request_json(url, params=params)
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\decorators.py", line 158, in wrapped
    return_value = function(reddit_session, *args, **kwargs)
  File "C:\Users\Aymeric\Python33\lib\site-packages\praw\__init__.py", line 483, in request_json
    data = json.loads(response, object_hook=hook)
  File "C:\Users\Aymeric\Python33\lib\json\__init__.py", line 332, in loads
    return cls(**kw).decode(s)
  File "C:\Users\Aymeric\Python33\lib\json\decoder.py", line 352, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\Aymeric\Python33\lib\json\decoder.py", line 370, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Edit2: It's now working if i remove ?context=3 and change the list to a string. Apparently, it never worked as a list anyway (my bad memory got in there !), and ?context=3 seems to be the problem.

Praw AttributeError: can't set attribute

I coded a reddit bot that will search through a subreddit's comments, and if a comment says feels it will reply to it. It then writes the comment id into a text file so that it never comments on the same comment twice.

Here is the code:

import praw
import time
import sys
import random

r = praw.Reddit('Posts Feels gif in response to someone saying feels'
                'by: Mjone77')
r.login('Feels_Bot', 'nottherealpassword')
file = open('Commentids.txt', 'r')
already_done = file.read()
file.close()
times = 0
feels = 0


while True:
   # try: 
        subreddit = r.get_subreddit('frozen')
        subreddit_comments = subreddit.get_comments()
        times+=1
        for comment in subreddit_comments:
            commentSays = comment.body
            commentSays = commentSays.lower()
            #print(commentSays)
            #print('\n')
            #if 'stop feels_bot' in commentSays:
                #sys.exit("Commanded to stop.")
            if comment.id not in already_done and 'feels' in commentSays:
                gif = random.randrange(0,3)
                if gif == 0:
                    comment.reply('[Relevant](http://i.imgur.com/pXBrf.gif)\n\n___\n\n^I ^am ^a ^bot ^not ^a ^real ^redditor \n\n ^Please ^contact ^/u/Mjone77 ^with ^any ^problems')
                if gif == 1:
                    comment.reply('[Relevant](http://gfycat.com/BraveSerpentineAzurevase)\n\n___\n\n^I ^am ^a ^bot ^not ^a ^real ^redditor \n\n ^Please ^contact ^/u/Mjone77 ^with ^any ^problems')
                if gif == 2:
                    comment.reply('[Relevant](http://www.gfycat.com/PlushMeanCowrie)\n\n___\n\n^I ^am ^a ^bot ^not ^a ^real ^redditor \n\n ^Please ^contact ^/u/Mjone77 ^with ^any ^problems')
                already_done = already_done+' '+comment.id
                file = open('Commentids.txt', 'w')
                file.write(already_done)
                file.close()
                #print('Commented~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~')
                time.sleep(5)
                feels+=1
        print('\n\n\n\n\n\n\n')        
        print('Feels:')
        print(feels)
        print('Times Ran:')
        print(times)
        time.sleep(60)
    #except AttriuteError:
        #pass

I ran it and left it running and came back to an HTTP Error 504: Gateway Time-out. Does anyone know how to keep that from happening or make it wait 60 seconds then try again?

Also, when I ran it again, without changing any code from the working program, it decided to give me this:

Traceback (most recent call last):
  File "C:\Users\Me\Desktop\My Programs\Feels Bot\Feels Bot\FeelsBot.py", line 20, in <module>
    for comment in subreddit_comments:
  File "C:\Python34\lib\site-packages\praw\__init__.py", line 471, in get_content
    page_data = self.request_json(url, params=params)
  File "C:\Python34\lib\site-packages\praw\decorators.py", line 161, in wrapped
    return_value = function(reddit_session, *args, **kwargs)
  File "C:\Python34\lib\site-packages\praw\__init__.py", line 516, in request_json
    data = json.loads(response, object_hook=hook)
  File "C:\Python34\lib\json\__init__.py", line 331, in loads
    return cls(**kw).decode(s)
  File "C:\Python34\lib\json\decoder.py", line 343, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Python34\lib\json\decoder.py", line 359, in raw_decode
    obj, end = self.scan_once(s, idx)
  File "C:\Python34\lib\site-packages\praw\__init__.py", line 403, in _json_reddit_objecter
    return object_class.from_api_response(self, json_data['data'])
  File "C:\Python34\lib\site-packages\praw\objects.py", line 58, in from_api_response
    return cls(reddit_session, json_dict=json_dict)
  File "C:\Python34\lib\site-packages\praw\objects.py", line 514, in __init__
    underscore_names=['replies'])
  File "C:\Python34\lib\site-packages\praw\objects.py", line 72, in __init__
    self.has_fetched = self._populate(json_dict, fetch)
  File "C:\Python34\lib\site-packages\praw\objects.py", line 141, in _populate
    setattr(self, name, value)
  File "C:\Python34\lib\site-packages\praw\objects.py", line 102, in __setattr__
    object.__setattr__(self, name, value)
AttributeError: can't set attribute
sys:1: ResourceWarning: unclosed <socket object at 0x0342D930>
C:\Python34\lib\importlib\_bootstrap.py:2150: ImportWarning: sys.meta_path is empty

It no longer runs at all, anyone know how to fix this error?

Using regular expressions to match a word in Python

I am using PRAW to make a reddit bot that takes the comment author of someone who says "alot" and stores their username into a list. I am having troubles with the regular expression and how to get the string to work. Here is my code.

#importing praw for reddit api and time to make intervals

import praw
import time
import re


username = "LewisTheRobot"
password = 



r = praw.Reddit(user_agent = "Counts people who say alot")

word_to_match = ['\balot\b']

storage = []

r.login(username, password)

def run_bot():
    subreddit = r.get_subreddit("test")
    print("Grabbing subreddit")
    comments = subreddit.get_comments(limit=200)
    print("Grabbing comments")
    for comment in comments:
        comment_text = comment.body.lower()
        isMatch = any(string in comment_text for string in word_to_match)
        if comment.id not in storage and isMatch:
            print("Match found! Storing username: " + str(comment.author) + " into list.")
            storage.append(comment.author)


    print("There are currently: " + str(len(storage)) + " people who use 'alot' instead of ' a lot'.")


while True:
    run_bot()
    time.sleep(5)

so the regular expression I am using looks for the word alot instead of alot as part of a string. Example zealot. Whenever I run this, it will not find a comment that I have made. Any suggestions?

Praw: API Login Fails with Client Error

I couldn't log into my own Reddit account using the code below.

Error message:

raise HTTPError(http_error_msg, response=self) HTTPError: 403 Client Error: Forbidden

Is there a way to get around this error?

class PrawTest(webapp2.RequestHandler):
  def get(self):
    self.response.headers['Content-Type'] = 'text/plain'
    self.response.out.write('Yo, imma redit bot!')

    get_login= ConfigParser.ConfigParser()
    get_login.read("logins.ini")

    r = praw.Reddit(user_agent='Captain Reddit!')

    r.login(get_login.get("login1", "username"),get_login.get("login1","password"))


app = webapp2.WSGIApplication([('/hype_shit_up', PrawTest)], debug=True)

Installing praw on a mac

I have already installed homebrew and pip. When I run $pip install praw the following comes up:

dhcp-215-185:~ my_name$ pip install praw
Downloading/unpacking praw
  Downloading praw-2.1.16-py2.py3-none-any.whl (70kB): 70kB downloaded
Downloading/unpacking update-checker>=0.10 (from praw)
  Downloading update_checker-0.10-py2.py3-none-any.whl
Downloading/unpacking requests>=1.2.0 (from praw)
  Downloading requests-2.2.1-py2.py3-none-any.whl (625kB): 625kB downloaded
Downloading/unpacking six>=1.4 (from praw)
  Downloading six-1.6.1-py2.py3-none-any.whl
Installing collected packages: praw, update-checker, requests, six
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.5.5-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.5.5-py2.7.egg/pip/commands/install.py", line 283, in run
requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.5-py2.7.egg/pip/req.py", line 1435, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/Library/Python/2.7/site-packages/pip-1.5.5-py2.7.egg/pip/req.py", line 671, in install
self.move_wheel_files(self.source_dir, root=root)
  File "/Library/Python/2.7/site-packages/pip-1.5.5-py2.7.egg/pip/req.py", line 901, in move_wheel_files
pycompile=self.pycompile,
  File "/Library/Python/2.7/site-packages/pip-1.5.5-py2.7.egg/pip/wheel.py", line 215, in move_wheel_files
clobber(source, lib_dir, True)
  File "/Library/Python/2.7/site-packages/pip-1.5.5-py2.7.egg/pip/wheel.py", line 205, in clobber
os.makedirs(destdir)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/praw'

Storing debug log for failure in /Users/my_name/Library/Logs/pip.log
dhcp-215-185:~ my_name$ 

Does anyone have a fix? Installing this has been a real headache. Thanks a lot!

How to do a try-except for a lazy object in Python using PRAW?

I was wondering if anyone could assist me with checking to see if "get_redditor" returns an error or not. I have used the "fetch=True" argument and it still returns. However, if you go to the user "Alaska88" page then it does not exist. The error happens when the program reaches the "for comment in comments" line and I am assuming the try-except doesn't work due to it being a lazy object. Thank you in advance for any time or help.

import praw
import urllib2
r = praw.Reddit('testing scraper')
r.login()
account = r.get_redditor('Alaska88',fetch=True)
comments = account.get_comments(sort ='new',time ='all')        
print 'before comment loop'
try:
        for comment in comments:
                print 'in comment loop' 
                print(comment.body.encode('utf-8'))
        print('/////////////////////////')
except urllib2.HTTPError:
        print 'In Except'       
        time.sleep(60)
        pass 

The error begins here =>

File "reddit_bot.py", line 9, in for comment in comments: File "/usr/local/lib/python2.7/dist-packages/praw-

The error then ends here =>

raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found

How to return a word in a string if it starts with a certain character? (Python)

I'm building a reddit bot for practice that converts US dollars into other commonly used currencies, and I've managed to get the conversion part working fine, but now I'm a bit stuck trying to pass the characters that directly follow a dollar sign to the converter.

This is sort of how I want it to work:

def run_bot():
    subreddit = r.get_subreddit("randomsubreddit")
    comments = subreddit.get_comments(limit=25)
    for comment in comments:
        comment_text = comment.body
        #If comment contains a string that starts with '$'
            # Pass the rest of the 'word' to a variable

So for example, if it were going over a comment like this:

"I bought a boat for $5000 and it's awesome"

It would assign '5000' to a variable that I would then put through my converter

What would be the best way to do this?

(Hopefully that's enough information to go off, but if people are confused I'll add more)

How to get comments from last week using PRAW?

I'm reading the doc for get_comments and I can't figure out what the correct input for the time parameter is. I know the default is "all", but do I input specific dates, or does it have to be words like "day", "week", etc?

Say I want to get comments a user made within the week, would it be something like:

r = praw.Reddit(user_agent='blah')
user = r.get_redditor(username)
comments = user.get_comments(sort='new', time='week', limit=None)

possible to get subscription information for subreddits/users of Reddit?

I would like to see either a list of subscribers for a given subreddit or a list of subreddits to which an individual user subscribes to. Is either of these bits of information available? In the PRAW docs I see my_reddits() but that seems to mean you can only retrieve the subscription list for yourself as a logged in user.

I also see this in the Reddit API docs: https://www.reddit.com/dev/api#GET_subreddits_mine_subscriber This also seems to assume you're looking at your own/logged-in account?

Sorry to post but I have not been able to find this in the Reddit API docs or the PRAW docs. I'd appreciate any relevant links or advice.