EzDevInfo.com

Using the clint progress bar to show the status of urllib.urlretrieve()

The clint progress bar is based on an iterator.
urllib.urlretrieve() has a callback that reports the completion of a chunk downloading.
Can I set the iterator to be in a certain position everytime the callback is called?

Py2exe requests, Clint, LXML

My setup.py throws the following error when trying to make my py2exe setup script

The following modules appear to be missing
['OpenSSL.SSL', '_scproxy', 'backports.ssl_match_hostname', 'builtins', 'certifi', 'clint.textui.puts', 'http', 'http.client', 'http.cookies', 'ndg.httpsclient.ssl_peer_verification', 'ndg.httpsclient.subj_alt_
name', 'packages.ssl_match_hostname.CertificateError', 'packages.ssl_match_hostname.match_hostname', 'packages.urllib3.Retry', 'packages.urllib3.util.Timeout', 'packages.urllib3.util.parse_url', 'pyasn1.codec.d

What am i missing from my setup.py so it will work ?

from setuptools import setup
from sys import version

try:
    import py2exe
except ImportError:
    print "Warning: py2exe is not installed."
    print "(Though it may not be available on your platform.)"

requires = ['requests', 'clint', 'lxml']
if version < '2.6.0':
    requires.append("simplejson")

setup(
    name='CFSRESD',
    version='0.1',
    author='Shane Rees',
    url='http://github.com/Shaggs/cfsresd',
    options=dict(py2exe=dict(includes=['scraper'])),

    requires= requires,
    license='GPL3',
    console=['daemon.py']
    )

and here is the top my daemon.py script

from scraper import Scraper
import sys
import requests
from clint.textui import puts, colored

I thought that my setup.py was correct but appears not.

Any Ideas ?

Printing columns of an equal custom set spacing while not letting long strings flow over other columns. (using clint perhaps?)

Suppose I have these two lists:

column1 = ["Attribute:","Virtual machine:","Troll:"]
column2 = ["A value associated with an object which is referenced by name using dotted expressions. For example, if an object o has an attribute a it would be referenced as o.a.", 
           "A computer defined entirely in software. Python's virtual machine executes the bytecode emitted by the bytecode compiler.",
           "Someone who posts inflammatory, extraneous, or off-topic messages in an online community, such as a forum, chat room, or blog, with the primary intent of provoking readers into an emotional response or of otherwise disrupting normal on-topic discussion."]

This code:

for c1, c2 in zip(column1, column2):
    print "%-16s %s" % (c1, c2)

Ouputs this text:

Attribute:       A value associated with an object which is referenced by name u
sing dotted expressions. For example, if an object o has an attribute a it would
be referenced as o.a.
Virtual machine: A computer defined entirely in software. Python's virtual machi
ne executes the bytecode emitted by the bytecode compiler.
Troll:           Someone who posts inflammatory, extraneous, or off-topic messag
es in an online community, such as a forum, chat room, or blog, with the primary
 intent of provoking readers into an emotional response or of otherwise disrupti
ng normal on-topic discussion.

While I would like this:

Attribute:       A value associated with an object which is referenced by name 
                 using dotted expressions. For example, if an object o has an 
                 attribute a it would be referenced as o.a.
Virtual machine: A computer defined entirely in software. Python's virtual 
                 machine executes the bytecode emitted by the bytecode compiler.
Troll:           Someone who posts inflammatory, extraneous, or off-topic 
                 messages in an online community, such as a forum, chat room, or 
                 blog, with the primary intent of provoking readers into an 
                 emotional response or of otherwise disrupting normal on-topic
                 discussion.

How do I get this result for any terminal size? (Someone advised me that Clint might be able to achieve this quite easily, has anyone done this yet?)

Note: The requirement that a word is not chopped in pieces by the end of a line is only secondary for me. The most important requirement is letting each string element of the column2 list continue with the same horizontal spacing as where the string of each element in that list started.

Python internal error Handling

I'm having issues with my program just closing at random stages and am not sure why.

At first, I thought it was because it was getting an error but I added an error handle. still for some reason it just closes after say a few days of running and no error is displayed. code below

import requests
import lxml.html as lh
import sys
import time
from clint.textui import puts, colored

API_URL = "http://urgmsg.net/livenosaas/ajax/update.php"

class Scraper (object):
    id_stamp = 0

    def __init__(self, timeout, recent_messages=True):
        self.timeout = timeout
        self.handlers = []
        self.recent_messages = recent_messages

    def register_handler(self, handler):
        self.handlers.append(handler)
        return handler

    def scrape(self):
        try:
            resp = requests.get(API_URL, params={'f': self.id_stamp}).json()
        except requests.exceptions.ConnectionError as e:
            puts("Error encountered when connecting to urgmsg: ", newline=False)
            puts(colored.red(e.__class__.__name__), newline=False)
            puts(" " + e.message)
            return

        if not resp['updated']:
            return

        old_id_stamp = self.id_stamp
        self.id_stamp = resp['IDstamp']
        # if old_id_stamp is 0, this is the first scrape
        # which will return a whole bunch of recent past messages
        if not self.recent_messages and old_id_stamp == 0: return

        # Pager messages are returned newest to oldest, we want to
        # process them oldest to newest
        frags = lh.fragments_fromstring(resp['data'])[::-1]
        for frag in frags:
            msg = PagerMessage(frag)
            for handler in self.handlers:
                handler(msg)

    def run(self):
        while True:
            self.scrape()
            time.sleep(self.timeout)

class PagerMessage:
    def __init__(self, fragment):
        children = fragment.getchildren()
        self.datetime = children[0].text
        self.text = children[1].text
        # channel starts with `- `
        self.channel = children[1].getchildren()[0].text[2:]
        self.response = 'CFSRES' in self.text
    def __str__(self):
        return "{} [{}]: {}".format(self.channel, self.datetime, self.text)

if __name__ == "__main__":
    scraper = Scraper(5)
    @scraper.register_handler
    def handler(msg):
        puts(colored.yellow(msg.channel), newline=False)
        puts(" [", newline=False)
        puts(colored.green(msg.datetime), newline=False)
        puts("] ", newline=False)
        if msg.response:
            puts(colored.red(msg.text))
        else:
            puts(msg.text)
    scraper.run()

Have I set this part out wrong ?

except requests.exceptions.ConnectionError as e:
                puts("Error encountered when connecting to urgmsg: ", newline=False)
                puts(colored.red(e.__class__.__name__), newline=False)
                puts(" " + e.message)
                return

How to monitor progress of a HTTP PUT upload using Python Requests and Clint

I am writing a simple commandline application - transfer.py - to allow for uploading and downloading files from the transfer.sh service as a learning exercise, using the 'requests' library for HTTP. Thanks to some answers on here, I was able to implement a progress bar using python-clint and python-requests for monitoring the file download - said functionality being seen here.

Anyway, I got very, very lost when trying to implement the same kind of progress bar to monitor the upload - which uses HTTP PUT. I understand conceptually it should be very similar, but cannot for some reason figure it out, and would be very thankful if someone could point me in the right direction on this. I tried a few methods using multipart encoders and suchlike, but those lead to the file being mangled on the way up (the service accepts raw PUT requests, and multipart encoding messes it up seemingly).

The end goal is to write a script to AES encrypt the file to be uploaded with a random key, upload it to the service, and print a link + encryption key that can be used by a friend to download/decrypt the file, mostly for fun and to fill in some knowledge-gaps in my python.

Download PDF by URL giving 0 byte of corrupted file in c#

byte[] arrayofbyte = new System.Net.WebClient().DownloadData("http://pdfurl");
MemoryStream ms = new MemoryStream(arrayofbyte);

System.Web.HttpContext.Current.Response.ClearContent();
System.Web.HttpContext.Current.Response.ContentEncoding = System.Text.Encoding.UTF8;
System.Web.HttpContext.Current.Response.AddHeader("ContentType", "application/pdf; charset=utf-8");
System.Web.HttpContext.Current.Response.AddHeader("Content-Disposition", "attachment; filename=" + DateTime.Now.Ticks.ToString() + ".pdf" + ";");
System.Web.HttpContext.Current.Response.BinaryWrite(ms.ToArray());
System.Web.HttpContext.Current.Response.Flush();
System.Web.HttpContext.Current.Response.Close();