HTTPretty
HTTP client mocking tool for Python, it's like ruby's FakeWeb for python
HTTPretty by gabrielfalcao
As Httpretty works on the Python socket layer, even Twisted web requests should be mocked out. But i am seeing some weird behavior on using httpretty. It tries to connect to localhost somehow. Below example shows the difference:
import httpretty
from twisted.internet import reactor
from twisted.internet.defer import Deferred
from twisted.web.client import Agent
from twisted.web.http_headers import Headers
import requests
@httpretty.activate
def main():
httpretty.register_uri(
httpretty.GET, "http://example.com",
body='[{"title": "Test Deal"}]',
content_type="application/json")
agent = Agent(reactor)
d = agent.request(
'GET',
'http://example.com',
Headers({'User-Agent': ['Twisted Web Client Example']}),
None)
def cbError(message):
print 'Async Failed : %s' % message
d.addErrback(cbError)
def cbShutdown(ignored): reactor.stop()
d.addBoth(cbShutdown)
reactor.run()
print 'Response received from Sync: %s' % \
requests.get('http://example.com').status_code
main()
And the response is :
Async Failed : [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionRefusedError'>: Connection was refused by other side: 111: Connection refused.
]
Response received from Sync: 200
How can i use httpretty with Twisted web client?
Source: (StackOverflow)
TL;DR
What libraries/calls are available to handle query strings containing semi-colons differently than parse_qs?
>>> urlparse.parse_qs("tagged=python;ruby")
>>> {'tagged': ['python']}
Full Background
I'm working with the StackExchange API to search for tagged questions.
Search is laid out like so, with tags separated by semi-colons:
/2.1/search?order=desc&sort=activity&tagged=python;ruby&site=stackoverflow
Interacting with the API is just fine. The problem comes in when I want to test the calls, particularly when using httpretty to mock HTTP.
Under the hood, httpretty is using urlparse.parse_qs from the python standard libraries to parse the querystring.
>>> urlparse.parse_qs("tagged=python;ruby")
{'tagged': ['python']}
Clearly that doesn't work well. That's the small example, here's a snippet of httpretty (outside of testing context).
import requests
import httpretty
httpretty.enable()
httpretty.register_uri(httpretty.GET, "https://api.stackexchange.com/2.1/search", body='{"items":[]}')
resp = requests.get("https://api.stackexchange.com/2.1/search", params={"tagged":"python;ruby"})
httpretty_request = httpretty.last_request()
print(httpretty_request.querystring)
httpretty.disable()
httpretty.reset()
I want to use the machinery from httpretty, but need a workaround for parse_qs. I can monkey patch httpretty for now, but would love to see what else can be done.
Source: (StackOverflow)
I'm currently experimenting with using the Python version of Selenium WebDriver along with the Pytest testing frameworkto do automation testing of web applications. I came across a problem when trying to do HTTP request mocking within my Selenium code. I wrote a module named "Selenium_WebDriver_Mocking_test.py" where I navigate to the official Python website and fill in a search term in the search box at the top of the page and then press Enter to move to the results page. The code works perfectly when I don't try to mock anything. Assuming you have both Selenium and Pytest installed, you can run it from the command line interface by typing the following:
py.test full/path/to/module/Selenium_WebDriver_Mocking_test.py
or by typing the following:
python -m pytest full/path/to/module/Selenium_WebDriver_Mocking_test.py
The code is shown below.
# contents of Selenium_WebDriver_Mocking_test.py
import selenium.webdriver
import selenium.webdriver.common.keys as common
import selenium.webdriver.support.ui as user_interface
import selenium.webdriver.support.expected_conditions as expected_conditions
import selenium.webdriver.common.by as locate
import re
import pytest
browsers = {
"firefox_driver": selenium.webdriver.Firefox(),
"chrome_driver": selenium.webdriver.Chrome()
}
website_homepage_url = "http://www.python.org"
title_1 = "Welcome to Python.org"
# Test set-up and tear down functions
@pytest.fixture(scope = 'session', params = browsers.keys())
def browser(request):
driver = browsers[request.param]
driver.maximize_window()
def close_browser():
driver.quit()
request.addfinalizer(close_browser)
return driver
@pytest.mark.parametrize(
"search_term",
[
("pycon"),
("tkinter"),
("django"),
]
)
def test_mocking_get_request_works_properly(search_term, browser):
browser.get(website_homepage_url)
assert title_1 in browser.title # Check you are on the right page
search_text_box = browser.find_element_by_css_selector("#id-search-field")
search_text_box.send_keys(search_term)
search_text_box.send_keys(common.Keys.RETURN)
search_page_title = user_interface.WebDriverWait(browser, 10).until(
expected_conditions.visibility_of_element_located(
(locate.By.XPATH, "//h2[contains(., 'Search')]")))
assert search_page_title.is_displayed() # Check you are on the right page
I then wrote some HTML for a fake results page and tried to use the HTTPretty tool written by Gabriel Falcao to return that page as a response that appears in the browser instead of the real Python.org results page. The modified code is shown below.
# contents of Selenium_WebDriver_Mocking_test.py
import selenium.webdriver
import selenium.webdriver.common.keys as common
import selenium.webdriver.support.ui as user_interface
import selenium.webdriver.support.expected_conditions as expected_conditions
import selenium.webdriver.common.by as locate
import time
import httpretty
import re
import pytest
browsers = {
"firefox_driver": selenium.webdriver.Firefox(),
"chrome_driver": selenium.webdriver.Chrome()
}
website_homepage_url = "http://www.python.org"
title_1 = "Welcome to Python.org"
request_url_pattern = re.compile('https://www.python.org/search/.*')
response_html_lines = ["<!DOCTYPE html>",
"<html>",
"<head>",
"<title>Welcome to my fun page</title>",
"<meta charset=\"UTF-8\">"
"</head>",
"<body>",
"<h2>Search Python.org</h2>",
"<h2>So far so good (^_^)</h2>",
"<img src = \"http://i.imgur.com/EjcKmEj.gif\">",
"</body>",
"</html>"]
fake_response_html = "\n".join(response_html_lines)
# Test set-up and tear down functions
@pytest.fixture(scope = 'session', params = browsers.keys())
def browser(request):
driver = browsers[request.param]
driver.maximize_window()
def close_browser():
driver.quit()
request.addfinalizer(close_browser)
return driver
@pytest.mark.parametrize(
"search_term",
[
("pycon"),
("tkinter"),
("django"),
]
)
def test_mocking_get_request_works_properly(search_term, browser):
httpretty.enable()
httpretty.register_uri(httpretty.GET,
request_url_pattern,
body = fake_response_html)
browser.get(website_homepage_url)
assert title_1 in browser.title # Check you are on the right page
search_text_box = browser.find_element_by_css_selector("#id-search-field")
search_text_box.send_keys(search_term)
search_text_box.send_keys(common.Keys.RETURN)
search_page_title = user_interface.WebDriverWait(browser, 10).until(
expected_conditions.visibility_of_element_located(
(locate.By.XPATH, "//h2[contains(., 'Search')]")))
assert search_page_title.is_displayed() # Check you are on the right page
time.sleep(10)
httpretty.disable()
httpretty.reset()
That approach did not work and I got the following message in the log of one of the test iterations:
C:\Python27\python.exe "C:\Program Files (x86)\JetBrains\PyCharm
Community Edition 4.0.4\helpers\pycharm\pytestrunner.py" -p pytest_teamcity C:/Users/johnsmith/Computer_Code/Python/Automation_Testing/Selenium_WebDriver_Mocking_test.py
Testing started at 10:10 ...
============================= test session starts =============================
platform win32 -- Python 2.7.6 -- py-1.4.26 -- pytest-2.6.4
plugins: xdist
collected 6 items
../../../../../../Users/johnsmith/Computer_Code/Python/Automation_Testing/Selenium_WebDriver_Mocking_test.py F
search_term = 'pycon'
browser = <selenium.webdriver.firefox.webdriver.WebDriver object at 0x0000000002E67EB8>
@pytest.mark.parametrize(
"search_term",
[
("pycon"),
("tkinter"),
("django"),
]
)
def test_mocking_get_request_works_properly(search_term, browser):
httpretty.enable()
httpretty.register_uri(httpretty.GET,
request_url_pattern,
body = fake_response_html)
> browser.get(website_homepage_url)
C:\Users\johnsmith\Computer_Code\Python\Automation_Testing\Selenium_WebDriver_Mocking_test.py:70:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py:187: in get
self.execute(Command.GET, {'url': url})
C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py:173: in execute
response = self.command_executor.execute(driver_command, params)
C:\Python27\lib\site-packages\selenium\webdriver\remote\remote_connection.py:349: in execute
return self._request(command_info[0], url, body=data)
C:\Python27\lib\site-packages\selenium\webdriver\remote\remote_connection.py:379: in _request
self._conn.request(method, parsed_url.path, body, headers)
C:\Python27\lib\httplib.py:973: in request
self._send_request(method, url, body, headers)
C:\Python27\lib\httplib.py:1007: in _send_request
self.endheaders(body)
C:\Python27\lib\httplib.py:969: in endheaders
self._send_output(message_body)
C:\Python27\lib\httplib.py:829: in _send_output
self.send(msg)
C:\Python27\lib\httplib.py:791: in send
self.connect()
C:\Python27\lib\httplib.py:772: in connect
self.timeout, self.source_address)
C:\Python27\lib\site-packages\httpretty\core.py:477: in create_fake_connection
s.connect(address)
C:\Python27\lib\site-packages\httpretty\core.py:311: in connect
self.truesock.connect(self._address)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
name = 'connect', self = <socket._socketobject object at 0x000000000385F9A0>
args = (('127.0.0.1', '49952'),)
def meth(name,self,*args):
> return getattr(self._sock,name)(*args)
E TypeError: an integer is required
C:\Python27\lib\socket.py:224: TypeError
I also tried using the responses tool written by David Cramer. While that did not return any errors, it also did not do anything at all and the test proceeded as if no mocking has happened. My question is: is there a way in Python to mock requests sent by the Selenium WebDriver and have the fake response bodies displayed in the browser instance driven by the driver instead ?
Any help is appreciated.
Source: (StackOverflow)
I have a project in which I am accessing a server that returns an object stream. Concurrently I am working on enhancing my TDD-fu. I'm not very familiar with mocking, as with Python's httpretty library. Is it possible via a mock to only return some N objects from the stream when initially testing general connectivity?
Source: (StackOverflow)
How does one go about asserting that a target URL was never requested when using HTTPretty to intercept HTTP requests in Python unit tests?
Source: (StackOverflow)
While making python-intercom Python 3 compatible, I ran into an issue on Travis.
The nosetests command doesn't appear to exit when run on Python 3.4 (it behaves as expected on Python 2.7). I narrowed it down to tests that use HTTPretty and created a small project to highlight the issue, and to publish the results of a failed build:
$ nosetests
..
----------------------------------------------------------------------
Ran 2 tests in 0.061s
OK
No output has been received in the last 10 minutes, this potentially
indicates a stalled build or something wrong with the build itself.
The build has been terminated
Right now I don't know how to proceed.
Source: (StackOverflow)
Whenever I enable HTTPretty, I'm unable to make a connection with PyMongo. I know that HTTPretty alters the core socket module; is there any way around this?
Code Example:
import pymongo
import httpretty
import time
httpretty.enable()
try:
client = pymongo.MongoClient()
except pymongo.errors.AutoReconnect:
print("AutoReconnect")
time.sleep(2)
Raises exception:
Traceback (most recent call last):
File "C:\Python33\lib\site-packages\pymongo\mongo_client.py", line 363, in __init__
self._ensure_connected(True)
File "C:\Python33\lib\site-packages\pymongo\mongo_client.py", line 924, in _ensure_connected
self.__ensure_member()
File "C:\Python33\lib\site-packages\pymongo\mongo_client.py", line 797, in __ensure_member
member, nodes = self.__find_node()
File "C:\Python33\lib\site-packages\pymongo\mongo_client.py", line 888, in __find_node
raise AutoReconnect(', '.join(errors))
pymongo.errors.AutoReconnect: [WinError 10035] A non-blocking socket operation could not be completed immediately
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "tmp.py", line 7, in
client = pymongo.MongoClient()
File "C:\Python33\lib\site-packages\pymongo\mongo_client.py", line 366, in __init__
raise ConnectionFailure(str(e))
pymongo.errors.ConnectionFailure: [WinError 10035] A non-blocking socket operation could not be completed immediately
I am on Windows 8.1 using Python 3.3.
Can anyone explain this behavior and how to resolve it? Thanks!
Source: (StackOverflow)
Using the HTTPretty library for Python, I can create mock HTTP responses of choice and then pick them up i.e. with the requests library like so:
import httpretty
import requests
# set up a mock
httpretty.enable()
httpretty.register_uri(
method=httpretty.GET,
uri='http://www.fakeurl.com',
status=200,
body='My Response Body'
)
response = requests.get('http://www.fakeurl.com')
# clean up
httpretty.disable()
httpretty.reset()
print(response)
Out: <Response [200]>
Is there also the possibility to register an uri which cannot be reached (e.g. connection timed out, connection refused, ...) such that no response is received at all (which is not the same as an established connection which gives an HTTP error code like 404)?
I want to use this behaviour in unit testing to ensure that my error handling works as expected (which does different things in case of 'no connection established' and 'connection established, bad bad HTTP status code'). As a workaround, I could try to connect to an invalid server like http://192.0.2.0 which would time out in any case. However, I would prefer to do all my unit testing without using any real network connections.
Source: (StackOverflow)
My Travis build is timing out for some reason.
Here is the link to the logs: https://travis-ci.org/madedotcom/atomicpuppy/builds/70202335
And my current travis yml https://github.com/madedotcom/atomicpuppy/blob/master/.travis.yml
I have tried running it with a normal script command
script: run-contexts --verbose
As well as creating a bash script that ran the tests and exited with $?
script: ./run-tests.sh
# run-tests.sh
run-contexts --verbose ./tests 2>/dev/null
exit $?
Even this didn't help.
My best guess is that there is some task running in the background of my Travis build due to depending on asynchronous python libs, but how can I debug that on Travis? Is there any way to increase verbosity?
Worth adding that locally tests are very fast to run, and exit with 0 without any problems, using the same commands.
Source: (StackOverflow)
I have written a decorator that is working correctly but i stumbled with the correct solution by trial and error and my litle knowledge about decorators tells me that something is not well defined.
The case is i'm mocking a Rest Api to do some TDD, and this Rest is behind a token security. So before making any request i first must get my user token. I'm using httpretty for mocking the API.
So far i had to register_uri in every test case, one to mock the /token resource and another to test any other resource. But i found that very cumbersome, so a came with a solution to write a simple decorator that would mock the /token and then only had to mock the tested resource.
This is my currently working decorator...
def activate_security(func):
def test_case(test_case):
httpretty.enable()
uri = 'http://{}:{}/token'.format(HOST, PORT)
httpretty.register_uri(httpretty.GET, uri,
body=dumps({'token': 'dummy_token'}),
content_type='application/json')
test_case()
httpretty.disable()
return test_case
And this is how is called.
@activate_security
@httpretty.activate
def test_case_one(self):
#Test case here
I had to pass the test_case parameter to the inner function 'cause without it it wouldn't work, and that test_case is test_case_one method, which i thought it would be passed in the func argument, but func in the outer scope holds the object at memory of test_case.
Should't be func the returned value of a decorator? If i do that, the decorator doesn't work. When the inner function is passed that parameter?
Source: (StackOverflow)
The following pytest-test uses httpretty, to mock a request. It writes the fetched data to a file:
import requests
import httpretty
import json
from os import listdir
from os.path import join
@httpretty.activate
def test_write_file_from_datasource():
tmpdir = './mytestdir'
# mock the connection
concert_url = 'http://apis.is/concerts'
httpretty.register_uri(httpretty.GET, concert_url,
body = json.dumps({'results': []}),
content_type='application/json')
# fetch data
concerts = requests.get(concert_url).json()
# write data
with open(join(tmpdir, 'concerts.json'), 'w') as json_data:
json.dump(concerts, json_data, indent=2)
assert len(listdir(tmpdir)) == 1
What I would like to do now, is making use of the pytest tmpdir feature. To reach this, I wrote a test like this (imports same as above):
@httpretty.activate
def test_write_file_from_datasource_failing(tmpdir):
tmpdir = str(tmpdir)
# mock the connection
concert_url = 'http://apis.is/concerts'
httpretty.register_uri(httpretty.GET, concert_url,
body = json.dumps({'results': []}),
content_type='application/json')
# fetch data
concerts = requests.get(concert_url).json()
# write data
with open(join(tmpdir, 'concerts.json'), 'w') as json_data:
json.dump(concerts, json_data, indent=2)
assert len(listdir(tmpdir)) == 1
It fails, because the httpretty decorator seems to have problems with the additional parameter:
TypeError: test_write_file_from_datasource_failing() takes exactly 1 argument (0 given)
Any ideas, how to fix this?
Source: (StackOverflow)
I'm new to python tests so don't hesitate to provide any obvious information.
Basically I want to do some RESTful tests using python, and found the httpretty and sure libraries which look really nice.
I have a python file containing:
#!/usr/bin/python
from sure import expect
import requests, httpretty
@httpretty.activate
def RestTest():
httpretty.register_uri(httpretty.GET, "http://localhost:8090/test.json",
body='{"status": "ok"}',
content_type="application/json")
response = requests.get("http://localhost:8090/test.json")
expect(response.json()).to.equal({"status": "ok"}
Which is basically the same as the example code provided at https://github.com/gabrielfalcao/HTTPretty
My question is; how do I simply run this test to see it either passing or failing? I tried just executing it using ./pythonFile but that doesn't work.
Source: (StackOverflow)