Requests Guard
requests-guard is a small library that allows you to impose size and time limits on your HTTP requests.
Installation
Just use pip
to install it:
pip install requests-guard
and you're done.
Usage
import requests
from requests_guard import guard
r = requests.get("https://www.google.com/", stream=True, timeout=30)
content = guard(r, max_size=3000, timeout=10)
requests-guard
will raise a ValueError
if it receives more than max_size
data in total, or if the entire
request takes more than timeout
seconds to be completed. That means that the call will always return after (roughly,
see below for details) timeout
seconds.
Note: You must call requests in the manner above, with stream=True
and timeout
. stream=True
allows the size
and time limits to be set, and the timeout
parameter to requests
instructs it to close the connection if no data
has been received for that number of seconds.
Another note: requests-guard
works by looking at the data as it receives it, so the timeout
parameter to
requests-guard
will apply to the entire connection, not just the latest chunk. The timeout
parameter to
requests
means "quit if we haven't received any data for that long", which means that a response may take an
arbitrary amount of time, as long as it always returns something every timeout
seconds. This means that a request
may potentially take up to the sum of the timeout specified to requests
and to requests-guard
, if the server stops
replying completely just before the timeout in requests-guard
elapses.