Commit graph

297 commits

Author SHA1 Message Date
Boštjan Mejak
158695817d closes bpo-35309: cpath should be capath (GH-10699) 2018-11-25 12:32:50 -06:00
Cheryl Sabella
0250de4819 bpo-27485: Rename and deprecate undocumented functions in urllib.parse (GH-2205) 2018-04-25 16:51:54 -07:00
Serhiy Storchaka
3f2e6f15d6
Revert unneccessary changes made in bpo-30296 and apply other improvements. (GH-2624) 2018-02-26 16:50:11 +02:00
INADA Naoki
579e0b80b9
urllib.request: Remove unused import (GH-5268) 2018-01-22 16:45:31 +09:00
Jon Dufresne
3972628de3 bpo-30296 Remove unnecessary tuples, lists, sets, and dicts (#1489)
* Replaced list(<generator expression>) with list comprehension
* Replaced dict(<generator expression>) with dict comprehension
* Replaced set(<list literal>) with set literal
* Replaced builtin func(<list comprehension>) with func(<generator
  expression>) when supported (e.g. any(), all(), tuple(), min(), &
  max())
2017-05-18 07:35:54 -07:00
Senthil Kumaran
6dfcc81f6b Remove OSError related comment in urllib.request. (#1070) 2017-04-09 19:49:34 -07:00
Senthil Kumaran
a2a9ddd923 Remove invalid comment in urllib.request. (#1054) 2017-04-08 23:27:25 -07:00
Xiang Zhang
04c15d5bdc Issue #29142: Merge 3.6. 2017-01-09 11:52:10 +08:00
Xiang Zhang
c44d58a77a Issue #29142: Merge 3.5. 2017-01-09 11:50:02 +08:00
Xiang Zhang
959ff7f1c6 Issue #29142: Fix suffixes in no_proxy handling in urllib.
In urllib.request, suffixes in no_proxy environment variable with
leading dots could match related hostnames again (e.g. .b.c matches a.b.c).
Patch by Milan Oberkirch.
2017-01-09 11:47:55 +08:00
Serhiy Storchaka
70d28a184c Remove unused imports. 2016-12-16 20:00:15 +02:00
Christian Heimes
d04863771b Issue #28022: Deprecate ssl-related arguments in favor of SSLContext.
The deprecation include manual creation of SSLSocket and certfile/keyfile
(or similar) in ftplib, httplib, imaplib, smtplib, poplib and urllib.

ssl.wrap_socket() is not marked as deprecated yet.
2016-09-10 23:23:33 +02:00
Raymond Hettinger
b7f3c944d1 Merge 2016-09-09 16:44:53 -07:00
Raymond Hettinger
ae9e5f032d Issue #22450: Use "Accept: */*" in the default headers for urllib.request 2016-09-09 16:43:48 -07:00
Martin Panter
3c0d0baf2b Issue #12319: Support for chunked encoding of HTTP request bodies
When the body object is a file, its size is no longer determined with
fstat(), since that can report the wrong result (e.g. reading from a pipe).
Instead, determine the size using seek(), or fall back to chunked encoding
for unseekable files.

Also, change the logic for detecting text files to check for TextIOBase
inheritance, rather than inspecting the “mode” attribute, which may not
exist (e.g. BytesIO and StringIO).  The Content-Length for text files is no
longer determined ahead of time, because the original logic could have been
wrong depending on the codec and newline translation settings.

Patch by Demian Brecht and Rolf Krahl, with a few tweaks by me.
2016-08-24 06:33:33 +00:00
Senthil Kumaran
cde03fa038 [merge from 3.5] - Prevent HTTPoxy attack (CVE-2016-1000110)
Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which
indicates that the script is in CGI mode.

Issue #27568 Reported and patch contributed by Rémi Rampin.
2016-07-30 23:51:13 -07:00
Senthil Kumaran
17742f2d45 [merge from 3.4] - Prevent HTTPoxy attack (CVE-2016-1000110)
Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which
indicates that the script is in CGI mode.

Issue #27568 Reported and patch contributed by Rémi Rampin.
2016-07-30 23:39:06 -07:00
Senthil Kumaran
436fe5a447 [merge from 3.3] Prevent HTTPoxy attack (CVE-2016-1000110)
Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which
indicates that the script is in CGI mode.

Issue #27568 Reported and patch contributed by Rémi Rampin.
2016-07-30 23:34:34 -07:00
Senthil Kumaran
4cbb23f8f2 Prevent HTTPoxy attack (CVE-2016-1000110)
Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which
indicates that the script is in CGI mode.

Issue #27568 Reported and patch contributed by Rémi Rampin.
2016-07-30 23:24:16 -07:00
Martin Panter
29f256909f Issue #22797: Synchronize urlopen() doc string with RST documentation 2016-06-04 05:06:34 +00:00
Martin Panter
0f29ad1be5 More typo fixes for 3.6 2016-06-04 05:06:25 +00:00
R David Murray
d2367c651e Clean up urlopen doc string.
Clarifies what is returned when and that the methods are common between the two.

Patch by Alexander Liu as part of #22797.
2016-06-03 20:16:06 -04:00
Martin Panter
0b39a556e8 Issue #14132, Issue #17214: Merge two redirect handling fixes from 3.5 2016-05-16 07:45:28 +00:00
Martin Panter
e6f060903c Issue #17214: Percent-encode non-ASCII bytes in redirect targets
Some servers send Location header fields with non-ASCII bytes, but "http.
client" requires the request target to be ASCII-encodable, otherwise a
UnicodeEncodeError is raised. Based on patch by Christian Heimes.

Python 2 does not suffer any problem because it allows non-ASCII bytes in the
HTTP request target.
2016-05-16 01:14:20 +00:00
Martin Panter
ce6e06874b Issue #14132: Fix redirect handling when target is just a query string 2016-05-16 01:07:13 +00:00
Senthil Kumaran
5d1110a952 merge from 3.5
Issue #26892: Honor debuglevel flag in urllib.request.HTTPHandler.

Patch contributed by Chi Hsuan Yen.
2016-05-13 01:35:29 -07:00
Senthil Kumaran
9642eedc0a Issue #26892: Honor debuglevel flag in urllib.request.HTTPHandler.
Patch contributed by Chi Hsuan Yen.
2016-05-13 01:32:42 -07:00
Martin Panter
51b697b7f3 Issue #26864: Merge no_proxy fixes from 3.5 2016-04-30 01:30:57 +00:00
Martin Panter
aa27982ffc Issue #26864: Fix case insensitivity and suffix comparison with no_proxy
Patch by Xiang Zhang.
2016-04-30 01:03:40 +00:00
Senthil Kumaran
0996fa3bd8 merge 3.5
Issue #26804: urllib.request will prefer lower_case proxy environment variables
over UPPER_CASE or Mixed_Case ones.

Patch contributed by Hans-Peter Jansen. Reviewed by Martin Panter and Senthil Kumaran.
2016-04-25 08:18:07 -07:00
Senthil Kumaran
a7c0ff2f0b Issue #26804: urllib.request will prefer lower_case proxy environment variables
over UPPER_CASE or Mixed_Case ones.

Patch contributed by Hans-Peter Jansen. Reviewed by Martin Panter and Senthil Kumaran.
2016-04-25 08:16:23 -07:00
Berker Peksag
48238c7e37 Issue #2202: Fix UnboundLocalError in AbstractDigestAuthHandler.get_algorithm_impls
Raise ValueError if algorithm is not MD5 or SHA.

Initial patch by Mathieu Dupuy.
2016-03-06 16:17:47 +02:00
Berker Peksag
e88dd1c32c Issue #2202: Fix UnboundLocalError in AbstractDigestAuthHandler.get_algorithm_impls
Raise ValueError if algorithm is not MD5 or SHA.

Initial patch by Mathieu Dupuy.
2016-03-06 16:16:40 +02:00
Serhiy Storchaka
885bdc4946 Issue #25985: sys.version_info is now used instead of sys.version
to format short Python version.
2016-02-11 13:10:36 +02:00
Martin Panter
a3643c280f Issue #12923: Merge FancyURLopener fix from 3.5 2016-02-06 01:08:40 +00:00
Martin Panter
a03702252f Issue #12923: Reset FancyURLopener's redirect counter even on exception
Based on patches by Brian Brazil and Daniel Rocco.
2016-02-04 06:01:35 +00:00
Serhiy Storchaka
3fd4a735d8 Issue #25899: Converted non-ASCII characters in docstrings and manpage
to ASCII replacements.  Removed UTF-8 BOM from Misc/NEWS.
Original patch by Chris Angelico.
2015-12-18 13:10:37 +02:00
Martin Panter
f65dd1d4db Issue #25576: Apply fix to new urlopen() doc string 2015-11-24 23:00:37 +00:00
Raymond Hettinger
507343a2ef Add missing docstring 2015-08-18 00:35:52 -07:00
Robert Collins
1f9a29f31b Issue #24021: docstring for urllib.urlcleanup.
Patch from Daniel Andrade Groppe and Peter Lovett
2015-08-04 12:52:43 +12:00
Robert Collins
2fee5c9367 Issue #24021: docstring for urllib.urlcleanup.
Patch from Daniel Andrade Groppe and Peter Lovett
2015-08-04 12:52:06 +12:00
R David Murray
4c7f995e80 #7159: generalize urllib prior auth support.
This fix is a superset of the functionality introduced by the issue #19494
enhancement, and supersedes that fix.  Instead of a new handler, we have a new
password manager that tracks whether we should send the auth for a given uri.
This allows us to say "always send", satisfying #19494, or track that we've
succeeded in auth and send the creds right away on every *subsequent* request.
The support for using the password manager is added to AbstractBasicAuth,
which means the proxy handler also now can handle prior auth if passed
the new password manager.

Patch by Akshit Khurana, docs mostly by me.
2015-04-16 16:36:18 -04:00
Victor Stinner
a9dd680d23 (Merge 3.4) Issue #23881: urllib.request.ftpwrapper constructor now closes the
socket if the FTP connection failed to fix a ResourceWarning.
2015-04-07 12:50:24 +02:00
Victor Stinner
ab73e65032 Issue #23881: urllib.request.ftpwrapper constructor now closes the socket if
the FTP connection failed to fix a ResourceWarning.
2015-04-07 12:49:27 +02:00
Benjamin Peterson
b666697fa8 use context's check_hostname attribute rather than the HTTPSHandler check_hostname parameter 2014-12-07 13:46:02 -05:00
Benjamin Peterson
074b95da48 merge 3.4 2014-12-07 13:47:39 -05:00
Nick Coghlan
c216c48699 Close #19494: add urrlib.request.HTTPBasicPriorAuthHandler
This auth handler adds the Authorization header to the first
HTTP request rather than waiting for a HTTP 401 Unauthorized
response from the server as the default HTTPBasicAuthHandler
does.

This allows working with websites like https://api.github.com which do
not follow the strict interpretation of RFC, but more the dicta in the
end of section 2 of RFC 2617:

    > A client MAY preemptively send the corresponding Authorization
    > header with requests for resources in that space without receipt
    > of another challenge from the server.  Similarly, when a client
    > sends a request to a proxy, it may reuse a userid and password in
    > the Proxy-Authorization header field without receiving another
    > challenge from the proxy server. See section 4 for security
    > considerations associated with Basic authentication.

Patch by Matej Cepl.
2014-11-12 23:33:50 +10:00
Senthil Kumaran
8b7e161ac3 backport context argument of urlopen (#22366) for pep 476 2014-09-19 15:23:30 +08:00
Senthil Kumaran
a5c85b3f5f Issue #22366: urllib.request.urlopen will accept a context object (SSLContext)
as an argument which will then used be for HTTPS connection.

Patch by Alex Gaynor.
2014-09-19 15:23:30 +08:00
Serhiy Storchaka
f54c350160 Issue #19524: Fixed resource leak in the HTTP connection when an invalid
response is received.  Patch by Martin Panter.
2014-09-06 21:41:39 +03:00