Python (requests · httpx) — Docs

requests

The proxies argument accepts a dict keyed by scheme. Use the URL-embedded credential form:

example.pypython

import requests

proxies = {
    "http":  "http://USER:[email protected]:8080",
    "https": "http://USER:[email protected]:8080",
}

r = requests.get("https://api.ipify.org", proxies=proxies, timeout=20)
print(r.status_code, r.text)

Note the https key value is still http://... — that's the proxy connection scheme, not the target scheme. CONNECT tunnelling for HTTPS targets is handled automatically.

Session-level proxies

Set proxies on a Session once, reuse for many calls. The connection pool keeps things fast.

python

import requests

s = requests.Session()
s.proxies.update({
    "http":  "http://USER:[email protected]:8080",
    "https": "http://USER:[email protected]:8080",
})

# A handful of GETs reuse the same TCP pool
for u in ["https://example.com/a", "https://example.com/b"]:
    print(s.get(u, timeout=20).status_code)

httpx (sync and async)

httpx takes a proxy= keyword (or the legacy proxies= dict). Async support is a one-line change.

sync httpxpython

import httpx

with httpx.Client(
    proxy="http://USER:[email protected]:8080",
    timeout=20,
) as client:
    r = client.get("https://api.ipify.org")
    print(r.status_code, r.text)

async httpxpython

import asyncio
import httpx

async def main():
    async with httpx.AsyncClient(
        proxy="http://USER:[email protected]:8080",
        timeout=20,
    ) as client:
        r = await client.get("https://api.ipify.org")
        print(r.status_code, r.text)

asyncio.run(main())

Sticky sessions

Append -session-TOKEN to the username to keep the same exit IP across calls. Useful for multi-step flows (login → dashboard → checkout).

python

import secrets, requests

session_token = secrets.token_hex(8)  # opaque, your choice
proxy = f"http://USER-session-{session_token}:[email protected]:8080"

s = requests.Session()
s.proxies = {"http": proxy, "https": proxy}

s.post("https://target.com/login", data={"u": "...", "p": "..."})
s.get("https://target.com/dashboard")  # same IP as the login above

Retry / backoff

Most failures are upstream target errors (5xx, timeouts) — retry with backoff. tenacity is the cleanest stand-alone helper:

python

from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type
import requests

@retry(
    reraise=True,
    stop=stop_after_attempt(4),
    wait=wait_exponential(multiplier=0.6, max=8),
    retry=retry_if_exception_type((requests.RequestException,)),
)
def fetch(url, sess):
    r = sess.get(url, timeout=20)
    r.raise_for_status()
    return r

# 4 attempts: ~0.6s, ~1.2s, ~2.4s, ~4.8s waits between failures.
fetch("https://target.com/x", sess)

On rotating products each retry pulls a different exit IP, so a transient block on one IP usually clears within the first retry. See the error reference for which codes are worth retrying.

TLS verification

We don't MITM TLS — the proxy is purely an L4 tunnel for HTTPS targets. Leave verify=True (the default) and your trust store applies as normal.

Don't disable TLS verification

If you find yourself reaching for verify=False, the problem is almost certainly on the target — likely a self-signed cert. We have nothing to do with that path.