Proxy Server Rotation Script in Python

Rotating Proxy Servers in Python

Recently, I have used ProxyMesh proxy server for a project. ProxyMesh offers 15 proxy servers, each denoting a specific location (such as us-dc) with 10 IP addresses rotating twice per day.

However, the free trial allows you to have one proxy server at a time. This means that if you are working with a server or CDN that strictly throttles IPs, you have to change the proxy server manually to keep rotating between the 15 proxy servers.

Fortunately, ProxyMesh provides an API that can be used to add and delete the proxy servers in user dashboard. Once a proxy server is assigned to the user, it can be fetched and used. All we need is a script that rotates between the proxy servers, deleting and adding them as required.

Based on this, I have written a script that would rotate through ProxyMesh proxy servers using their API.

Prerequisites

An account with ProxyMesh, either free trial or paid. Set the user name and password in rotatingproxy.py

self.user = ""
self.password = ""

Usage

Setting the Proxy Server

from rotatingproxy import RotatingProxy

rproxy = RotatingProxy()

The proxy server can either be set randomly or selected from an available list of proxy servers.

The active proxy server is saved in a text file which can be accessed as required.

rproxy.set_proxy(israndom="r")  # select a random proxy server

rproxy.set_proxy(proxy_num=1)   # select proxy server with index=1 from the list of proxy servers.

Accessing the Proxy Server

def get_proxy_from_file():
    # fetches proxy from proxy.txt
    with open("proxy.txt", "r") as f:
        return loads(f.read())

proxy = get_proxy_from_file()

The proxy can now be used with requests:

import requests
response = requests.get("url-to-fetch", proxies=proxy)

Blog post: http://www.kashifaziz.me/proxy-server-rotation-python.html/

GitHub: https://github.com/kashaziz/rotating-proxy-python

The Practical Developer责编内容来自:The Practical Developer (源链) | 更多关于

阅读提示:酷辣虫无法对本内容的真实性提供任何保证,请自行验证并承担相关的风险与后果!
本站遵循[CC BY-NC-SA 4.0]。如您有版权、意见投诉等问题,请通过eMail联系我们处理。
酷辣虫 » 综合编程 » Proxy Server Rotation Script in Python

喜欢 (0)or分享给?

专业 x 专注 x 聚合 x 分享 CC BY-NC-SA 4.0

使用声明 | 英豪名录