Virteche
Guideschevron_rightProxies

Proxies · 7 min read · Updated

Using a Proxy in Python, Playwright, Puppeteer and Selenium

Every library takes the same four values (address, port, username, password), but not in the same place. Python's requests and httpx take a proxy URL, Playwright takes a proxy object with the username and password as separate fields, and Puppeteer takes the address on launch and the credentials through page.authenticate. Selenium with Chrome cannot pass proxy credentials at all, so it goes through a small local forwarder. Browsers need the HTTP port: Chrome does not support SOCKS5 with a password.

The Values You Need

In your dashboard, each proxy shows its address, an HTTP port, a SOCKS5 port, a username and a password. The examples below use these placeholders:

IP     198.51.100.42
PORT   the HTTP port
SPORT  the SOCKS5 port
USER   your username
PASS   your password

Every example fetches https://api.ipify.org, which replies with the address it saw. If it prints the proxy's address rather than your own, the proxy is working.

Python: requests

import requests

proxy = "http://USER:[email protected]:PORT"
r = requests.get("https://api.ipify.org", proxies={"http": proxy, "https": proxy}, timeout=30)
print(r.text)

For SOCKS5, install the extra and use socks5h://, so the proxy also looks up the site's address:

pip install "requests[socks]"

proxy = "socks5h://USER:[email protected]:SPORT"

To reuse connections across many requests, set it once on a session:

s = requests.Session()
s.proxies = {"http": proxy, "https": proxy}

Python: httpx

import httpx

with httpx.Client(proxy="http://USER:[email protected]:PORT", timeout=30) as client:
    print(client.get("https://api.ipify.org").text)

Recent httpx versions take proxy=; the older proxies= argument has been removed. For SOCKS5, pip install "httpx[socks]" and use a socks5:// URL. httpx.AsyncClient takes the same argument.

Playwright

Python:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(proxy={
        "server": "http://198.51.100.42:PORT",
        "username": "USER",
        "password": "PASS",
    })
    page = browser.new_page()
    page.goto("https://api.ipify.org")
    print(page.inner_text("body"))
    browser.close()

Node.js takes the same object:

import { chromium } from "playwright";

const browser = await chromium.launch({
  proxy: { server: "http://198.51.100.42:PORT", username: "USER", password: "PASS" },
});
const page = await browser.newPage();
await page.goto("https://api.ipify.org");
console.log(await page.innerText("body"));
await browser.close();

To use different proxies in one browser, pass the same proxy object to browser.new_context() (newContext() in Node.js) instead, one context per proxy.

Puppeteer

import puppeteer from "puppeteer";

const browser = await puppeteer.launch({ args: ["--proxy-server=http://198.51.100.42:PORT"] });
const page = await browser.newPage();
await page.authenticate({ username: "USER", password: "PASS" });
await page.goto("https://api.ipify.org");
console.log(await page.evaluate(() => document.body.innerText));
await browser.close();

Call page.authenticate on every new page before it loads anything. A page without it gets a 407 from the proxy.

Selenium

Chrome takes a proxy address on its command line, but not a username and password, and Selenium has no simple way to answer the proxy's login prompt. The dependable fix is a forwarder on your own machine that adds the credentials. With gost, a single program:

gost -L http://127.0.0.1:8080 -F "http://USER:[email protected]:PORT"

Then point Chrome at the forwarder, with no credentials needed:

from selenium import webdriver

opts = webdriver.ChromeOptions()
opts.add_argument("--proxy-server=http://127.0.0.1:8080")
driver = webdriver.Chrome(options=opts)
driver.get("https://api.ipify.org")
print(driver.find_element("tag name", "body").text)
driver.quit()

Keep the forwarder on 127.0.0.1. Listening on all addresses would turn it into an open proxy that anyone could use with your account. Run one forwarder per proxy, on different local ports, to use several.

Keep Credentials out of Code

Read the proxy URL from an environment variable or a secrets file, not from the source. Then a changed password is a config change, not a commit, and the credentials never end up in your git history:

import os
proxy = os.environ["PROXY_URL"]

For curl, browsers and system-wide settings, see How to set up a proxy. Proxies are covered by our terms of service, like everything else you send through them.

Common Questions

  • Why does my browser script ignore the username and password?

    Chrome and Chromium do not take credentials as part of the --proxy-server address, and do not support SOCKS5 with a username and password at all. Use the HTTP port, and pass the credentials the way each tool expects: Playwright's proxy option, Puppeteer's page.authenticate, or a local forwarder for Selenium.

  • My password has special characters. Do I need to change it?

    No, but in a proxy URL characters such as @, : and / must be percent-encoded, or the URL is read wrongly. In Python, urllib.parse.quote(password, safe="") does it. Options that take the username and password as separate fields, like Playwright's, need no encoding.

  • How do I use several proxies?

    Give each worker, browser context or session its own address from your list and keep it for the whole task, so a site sees one consistent visitor per address. With static ISP proxies, each address stays the same until you change it.

  • Why do I get 407 Proxy Authentication Required?

    The proxy was reached but the credentials were not accepted. Check for a stray space when copying, that the URL is encoded, and that you are using the HTTP port with an HTTP proxy URL and the SOCKS5 port with a socks5 URL.

Static US ISP proxies with an HTTP and a SOCKS5 port on every address, and credentials in your dashboard as soon as payment clears.

See proxy pricing

Keep Reading

All guides