mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-08-31 14:07:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

gh-91539: improve performance of get_proxies_environment (#91566)

Browse source 
* improve performance of get_proxies_environment when there are many environment variables

* 📜🤖 Added by blurb_it.

* fix case of short env name

* fix formatting

* fix whitespace

* whitespace

* Update Lib/urllib/request.py

Co-authored-by: Carl Meyer <carl@oddbird.net>

* Update Lib/urllib/request.py

Co-authored-by: Carl Meyer <carl@oddbird.net>

* Update Lib/urllib/request.py

Co-authored-by: Carl Meyer <carl@oddbird.net>

* Update Lib/urllib/request.py

Co-authored-by: Carl Meyer <carl@oddbird.net>

* whitespace

* Update Misc/NEWS.d/next/Library/2022-04-15-11-29-38.gh-issue-91539.7WgVuA.rst

Co-authored-by: Carl Meyer <carl@oddbird.net>

* Update Lib/urllib/request.py

Co-authored-by: Carl Meyer <carl@oddbird.net>

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
This commit is contained in:
Pieter Eendebak 2022-10-05 19:57:52 +02:00 • committed by GitHub
parent 9442105ce7
commit aeb28f5130
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 17 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

26
Lib/urllib/request.py
View file

@ -2508,28 +2508,34 @@ def getproxies_environment():
this seems to be the standard convention. If you need a this seems to be the standard convention. If you need a
different way, you can pass a proxies dictionary to the different way, you can pass a proxies dictionary to the
[Fancy]URLopener constructor. [Fancy]URLopener constructor.
""" """
proxies = {}
# in order to prefer lowercase variables, process environment in # in order to prefer lowercase variables, process environment in
# two passes: first matches any, second pass matches lowercase only # two passes: first matches any, second pass matches lowercase only
for name, value in os.environ.items():
name = name.lower() # select only environment variables which end in (after making lowercase) _proxy
if value and name[-6:] == '_proxy': proxies = {}
proxies[name[:-6]] = value environment = []
for name in os.environ.keys():
# fast screen underscore position before more expensive case-folding
if len(name) > 5 and name[-6] == "_" and name[-5:].lower() == "proxy":
value = os.environ[name]
proxy_name = name[:-6].lower()
environment.append((name, value, proxy_name))
if value:
proxies[proxy_name] = value
# CVE-2016-1000110 - If we are running as CGI script, forget HTTP_PROXY # CVE-2016-1000110 - If we are running as CGI script, forget HTTP_PROXY
# (non-all-lowercase) as it may be set from the web server by a "Proxy:" # (non-all-lowercase) as it may be set from the web server by a "Proxy:"
# header from the client # header from the client
# If "proxy" is lowercase, it will still be used thanks to the next block # If "proxy" is lowercase, it will still be used thanks to the next block
if 'REQUEST_METHOD' in os.environ: if 'REQUEST_METHOD' in os.environ:
proxies.pop('http', None) proxies.pop('http', None)
for name, value in os.environ.items(): for name, value, proxy_name in environment:
# not case-folded, checking here for lower-case env vars only
if name[-6:] == '_proxy': if name[-6:] == '_proxy':
name = name.lower()
if value: if value:
proxies[name[:-6]] = value proxies[proxy_name] = value
else: else:
proxies.pop(name[:-6], None) proxies.pop(proxy_name, None)
return proxies return proxies
def proxy_bypass_environment(host, proxies=None): def proxy_bypass_environment(host, proxies=None):

1
Misc/NEWS.d/next/Library/2022-04-15-11-29-38.gh-issue-91539.7WgVuA.rst Normal file
View file

@ -0,0 +1 @@
Improve performance of ``urllib.request.getproxies_environment`` when there are many environment variables