How to set up Accept-Encoding to gzip in Python pyppeteer and print pyppeteer headers?

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

how to set headers for pyppeteer for example:Accept-Encoding: gzip

how to print pyppeteer headers in python. i know java

const response = await page.goto('https://example.org/')
console.log(response.headers)
result
{ date: 'Sun, 29 Oct 2017 13:35:59 GMT',
'content-encoding': 'gzip',
'last-modified': 'Fri, 09 Aug 2013 23:54:35 GMT',
server: 'ECS (lga/1318)',
etag: '"359670651+gzip"',
vary: 'Accept-Encoding',
'x-cache': 'HIT',
'content-type': 'text/html',
status: '200',
'cache-control': 'max-age=604800',
'content-length': '606',
expires: 'Sun, 05 Nov 2017 13:35:59 GMT' }
from pyppeteer.network_manager import Request, Response
async def req_intercept(req: Request):
    print(f'Original header: {req.headers}')
    req.headers.update({'Accept-Encoding': 'gzip'})
    await req.continue_(overrides={'headers': req.headers})
async def resp_intercept(resp: Response):
    print(f"New header: {resp.request.headers}")
async def test():
    browser = await pyppeteer.launch()
    page = await browser.newPage()
    await page.setRequestInterception(True)
    page.on('request', req_intercept)
    page.on('response', resp_intercept)
    resp = await page.goto('https://example.org/')
    print(resp.headers)
asyncio.get_event_loop().run_until_complete(test())
result:
Original header: {'upgrade-insecure-requests': '1', 'user-agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3494.0 Safari/537.36', 'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8'}
New header:      {'upgrade-insecure-requests': '1', 'user-agent':'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3494.0 Safari/537.36', 'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8', 'Accept-Encoding': 'gzip'}
{'status': '200', 'content-encoding': 'gzip', 'accept-ranges': 'bytes', 'cache-control': 'max-age=604800', 'content-type': 'text/html; charset=UTF-8', 'date': 'Sat, 13 Apr 2019 03:07:49 GMT', 'etag': '"1541025663"', 'expires': 'Sat, 20 Apr 2019 03:07:49 GMT', 'last-modified': 'Fri, 09 Aug 2013 23:54:35 GMT', 'server': 'ECS (dcb/7F84)', 'vary': 'Accept-Encoding', 'x-cache': 'HIT', 'content-length': '606'}
        Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.