Requests请求封装以及API

使用代理

import requests

proxies = {
    "http": "http://xxx:xxx&@xxx.xxx.xxx.cn",
    "https": "http://xxx:xxx&@xxx.xxx.xxx.cn",
}

res = requests.get("http://127.0.0.1:8110", proxies=proxies)
print(res.text)

重试机制

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

session = requests.session()

retry = Retry(
            total=5,
            read=5,
            connect=5,
            backoff_factor=0.3,
            status_forcelist=(500, 502, 504),
        )
adapter = HTTPAdapter(max_retries=retry)
session.mount("http://", adapter)
session.mount("https://", adapter)

添加Cookie

常规添加

headers

import requests

headers = {"Cookie":"test=111;test1=2222"}
res = requests.get("https://xxxx-xxx.xxxx/get",headers=headers)
print(res.text)

cookies

import requests


cookies = {"test":"111","test1":"222"}
res = requests.get("https://xxxx-xxx.xxxx/get",cookies=cookies)

print(res.text)

session添加

1. 自动设置
2. 直接赋值（一般情况不好用 不推荐）

   session.cookies["test"]="111"
   session.cookies["test2"]="222"

   or

   session.cookies.set("test","111",path="/",domain="xxx.xxx.cn")

3. add_dict_to_cookiejar (推荐)

import requests

session = requests.session()
cookies = {"test":"111","test1":"222"}
requests.utils.add_dict_to_cookiejar(session.cookies,cookie_dict=cookie_dict)

res = session.get("https://xxx-xxx.xxx/get")

print(session.cookies)

4. RequestsCookieJar 对象设置

import requests

session = requests.session()

test = requests.sessions.RequestsCookieJar()
test.set('test', '111', path='/', domain='xxx-xxx.cn')
session.cookies.update(test)

res = session.get("https://xxx-xxx.cn/get")
print(session.cookies)

5. cookiejar_from_dict

import requests

session = requests.session()
cookie_dict = {"test":"111","test2":"222"}
test = requests.utils.cookiejar_from_dict(cookie_dict=cookie_dict)
session.cookies.update(test)

res = session.get("https://xxx-xxx.cn/get")
print(session.cookies)

忽略SSL证书以及错误警告

报错SSL

import requests
requests.get("https://xxx.xxx.com" , verify=False)

此时会报错警告

InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)

全局添加以下代码

requests.packages.urllib3.disable_warnings()

重定向

import requests

requests.get("https://xxx.xxx.com" , allow_redirects=False)

allow_redirects

为False ： 禁止重定向
为True： 允许重定向

一般返回的response.url 为正确URL

其他

剩下的params | data | cookies | files | auth | timeout | hooks | stream | cert | json
要不就是太简单 要不就是很少用。这里不讲了。