Commit graph

27173 commits

Author SHA1 Message Date
Miss Islington (bot)
60ba0b6847
bpo-44022: Fix http client infinite line reading (DoS) after a HTTP 100 Continue (GH-25916) (GH-25931)
Fixes http.client potential denial of service where it could get stuck reading lines from a malicious server after a 100 Continue response.

Co-authored-by: Gregory P. Smith <greg@krypto.org>
(cherry picked from commit 47895e31b6)

Co-authored-by: Gen Xu <xgbarry@gmail.com>
2021-05-05 16:14:28 -07:00
Miss Islington (bot)
24f1d1a8a2
bpo-43882 Remove the newline, and tab early. From query and fragments. (GH-25936)
(cherry picked from commit 985ac01637)
2021-05-05 16:04:38 -07:00
Miss Islington (bot)
10d6f6bfd7
bpo-35753: Fix crash in doctest with unwrap-able functions (GH-22981) (#25926)
Ignore objects that inspect.unwrap throws due to
too many wrappers.  This is a very rare case, however
it can easily be surfaced when a module under doctest
imports unitest.mock.call into its namespace.

We simply skip any object that throws this exception.
This should handle the majority of cases.
(cherry picked from commit 565a31804c)

Co-authored-by: Alfred Perlstein <alfred@fb.com>
2021-05-05 20:01:21 +02:00
Miss Islington (bot)
7b4725a210
bpo-40297: Fix test_socket.CANTest.testSendFrame (GH-19548)
The address tuple for CAN_RAW no longer returns the address family
after the introduction of CAN ISO-TP support in a30f6d45ac. However,
updating test_socket.CANTest.testSendFrame was missed as part of the
change, so the test incorrectly attempts to index past the last tuple
item to retrieve the address family.

This removes the now-redundant check for equality against socket.AF_CAN,
as the tuple will not contain the address family.
(cherry picked from commit 355bae8882)

Co-authored-by: karl ding <karlding@users.noreply.github.com>
2021-05-04 14:01:42 -07:00
Miss Islington (bot)
2995bff426
bpo-44018: random.seed() no longer mutates its inputs (GH-25856) (GH-25872) 2021-05-03 19:45:30 -07:00
Miss Islington (bot)
d194e1eb93
fix enum and backslash warnings (GH-25861) (GH-25871)
- Enum warning in test_faulthandler
- backslash warning in test_ipaddress
(cherry picked from commit 652bcd9f9c)

Co-authored-by: Ethan Furman <ethan@stoneleaf.us>

Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
2021-05-03 19:17:23 -07:00
Miss Islington (bot)
756b7b9248
bpo-43822: Prioritize tokenizer errors over custom syntax errors when raising parser exceptions (GH-25866)
(cherry picked from commit 9142088e74)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-05-03 18:06:45 -07:00
Shreyan Avigyan
d3b9134ebb
Remove Enum warnings from test_httpservers (GH-25844) 2021-05-03 20:27:47 +01:00
Dennis Sweeney
80a2a4ed7d
bpo-38530: Refactor and improve AttributeError suggestions (GH-25776)
- Make case-swaps half the cost of any other edit
- Refactor Levenshtein code to not use memory allocator, and to bail early on no match.
- Add comments to Levenshtein distance code
- Add test cases for Levenshtein distance behind a debug macro
- Set threshold to `(name_size + item_size + 3) * MOVE_COST / 6`.
  - Reasoning: similar to `difflib.SequenceMatcher.ratio()` >= 2/3:
```
"Multiset Jaccard similarity" >= 2/3
matching letters / total letters >= 2/3
(name_size - distance + item_size - distance) / (name_size + item_size) >= 2/3
1 - (2*distance) / (name_size + item_size) >= 2/3
1/3 >= (2*distance) / (name_size + item_size)
(name_size + item_size) / 6 >= distance
With rounding:
(name_size + item_size + 3) // 6 >= distance
```

Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-05-03 16:47:27 +01:00
Christian Heimes
c715b52421
bpo-43943: ssl tests: Increase server socket timeout, backlog, debugging (GH-25850)
Signed-off-by: Christian Heimes <christian@python.org>
2021-05-03 17:45:02 +02:00
Pablo Galindo
7719953b30
bpo-44011: Revert "New asyncio ssl implementation (GH-17975)" (GH-25848)
This reverts commit 5fb06edbbb and all
subsequent dependent commits.
2021-05-03 16:21:59 +01:00
Andrew Svetlov
4ffd6fd2f1
Increase test timeout (GH-25842) 2021-05-03 12:06:17 +03:00
Batuhan Taskaya
ad106c68eb
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583) 2021-05-03 10:43:00 +03:00
Christian Heimes
37ebdf0a86
bpo-44011: Fix asyncio tests without ssl module (GH-25840)
Signed-off-by: Christian Heimes <christian@python.org>
2021-05-03 09:38:56 +02:00
Eric V. Smith
99ad742ea9
bpo-44015: dataclasses should allow KW_ONLY to be specified only once per class (GH-25841)
bpo-44015: Raise a TypeError if KW_ONLY is specified more than once.
2021-05-03 03:24:53 -04:00
Raymond Hettinger
8c598dbb94
bpo-25478: Add total() method to collections.Counter (GH-25829) 2021-05-02 20:19:51 -07:00
Ned Deily
8703178258
bpo-43568: Drop support for MACOSX_DEPLOYMENT_TARGET < 10.3 (GH-25827)
Only complain if the config target is >= 10.3 and the current target is
< 10.3. The check was originally added to ensure that incompatible
LDSHARED flags are not used, because -undefined dynamic_lookup is
used when building for 10.3 and later, and is not supported on older OS
versions. Apart from that, there should be no problem in general
with using an older target.

Authored-by: Joshua Root <jmr@macports.org>
2021-05-02 20:28:43 -04:00
Mark Shannon
33ec88ac81
bpo-43977: Make sure that tp_flags for pattern matching are inherited correctly. (GH-25813) 2021-05-03 00:38:22 +01:00
Andrew Svetlov
5fb06edbbb
bpo-44011: New asyncio ssl implementation (#17975) 2021-05-03 00:34:15 +03:00
Erlend Egeberg Aasland
c96cc089f6
bpo-43434: Move sqlite3.connect audit events to sqlite3.Connection.__init__ (GH-25818) 2021-05-02 22:25:17 +01:00
Jason R. Coombs
37e0c7850d
bpo-43926: Cleaner metadata with PEP 566 JSON support. (GH-25565)
* bpo-43926: Cleaner metadata with PEP 566 JSON support.

* Add blurb

* Add versionchanged and versionadded declarations for changes to metadata.

* Use descriptor for PEP 566
2021-05-02 17:03:40 -04:00
Brandt Bucher
0ad1e0384c
bpo-43754: Eliminate bindings for partial pattern matches (GH-25229) 2021-05-02 13:02:10 -07:00
Christian Heimes
60ce8f0be6
bpo-36384: Leading zeros in IPv4 addresses are no longer tolerated (GH-25099)
Reverts commit e653d4d8e8 and makes
parsing even more strict. Like socket.inet_pton() any leading zero
is now treated as invalid input.

Signed-off-by: Christian Heimes <christian@python.org>

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-05-02 14:00:35 +02:00
Christian Heimes
91554e4c5c
bpo-43908: Mark ssl, hash, and hmac types as immutable (GH-25792)
Signed-off-by: Christian Heimes <christian@python.org>
2021-05-02 09:47:45 +02:00
Ethan Furman
1ae05fdf2d
Revert "bpo-43989: Temporarily disable warnings in ssltests (GH-25780)" (GH-25793)
Reverts python/cpython#25780

Automerge-Triggered-By: GH:tiran
2021-05-01 13:53:57 -07:00
Christian Heimes
ddbef71a2c
bpo-43916: Rewrite new hashlib tests, fix typo (GH-25791)
* bpo-43916: Rewrite new hashlib tests, fix typo
* Flag test as cpython only
2021-05-01 22:42:36 +02:00
Christian Heimes
a5669b3c62
bpo-43998: Fix testing without ssl module (GH-25790)
Signed-off-by: Christian Heimes <christian@python.org>
2021-05-01 22:17:01 +02:00
Christian Heimes
e983252b51
bpo-43998: Default to TLS 1.2 and increase cipher suite security (GH-25778)
The ssl module now has more secure default settings. Ciphers without forward
secrecy or SHA-1 MAC are disabled by default. Security level 2 prohibits
weak RSA, DH, and ECC keys with less than 112 bits of security.
:class:`~ssl.SSLContext` defaults to minimum protocol version TLS 1.2.
Settings are based on Hynek Schlawack's research.

```
$ openssl version
OpenSSL 1.1.1k  FIPS 25 Mar 2021
$ openssl ciphers -v '@SECLEVEL=2:ECDH+AESGCM:ECDH+CHACHA20:ECDH+AES:DHE+AES:!aNULL:!eNULL:!aDSS:!SHA1:!AESCCM'
TLS_AES_256_GCM_SHA384  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(256) Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 Kx=any      Au=any  Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(128) Mac=AEAD
TLS_AES_128_CCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESCCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
```

Signed-off-by: Christian Heimes <christian@python.org>
2021-05-01 20:53:10 +02:00
Eric V. Smith
823fbf4e0e
If using a frozen class with slots, add __getstate__ and __setstate__ to set the instance values. (GH-25786) 2021-05-01 13:27:30 -04:00
Christian Heimes
f82fd77717
bpo-43989: Temporarily disable warnings in ssltests (GH-25780)
-Werror is currently broken.
2021-05-01 16:05:01 +02:00
Yurii Karabas
c24199184b
bpo-42269: Add slots parameter to dataclass decorator (GH-24171)
Add slots parameter to dataclass decorator and make_dataclass function.
2021-04-30 22:14:30 -04:00
Erlend Egeberg Aasland
e467ec476f
bpo-43995: Fix reference leak in test_grammar (GH-25764) 2021-05-01 00:23:14 +01:00
Victor Stinner
665c7746fc
bpo-43916: _md5.md5 uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25753)
The following types use Py_TPFLAGS_DISALLOW_INSTANTIATION flag:

* _md5.md5
* _sha1.sha1
* _sha256.sha224
* _sha256.sha256
* _sha512.sha384
* _sha512.sha512
2021-04-30 18:40:30 +02:00
Pablo Galindo
e374a40afa
bpo-43901: Fix refleaks in test_module (GH-25754) 2021-04-30 17:26:45 +01:00
Victor Stinner
7dcf0f6db3
bpo-43916: select.devpoll uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25751) 2021-04-30 18:19:57 +02:00
Erlend Egeberg Aasland
9746cda705
bpo-43916: Apply Py_TPFLAGS_DISALLOW_INSTANTIATION to selected types (GH-25748)
Apply Py_TPFLAGS_DISALLOW_INSTANTIATION to the following types:

* _dbm.dbm
* _gdbm.gdbm
* _multibytecodec.MultibyteCodec
* _sre..SRE_Scanner
* _thread._localdummy
* _thread.lock
* _winapi.Overlapped
* array.arrayiterator
* functools.KeyWrapper
* functools._lru_list_elem
* pyexpat.xmlparser
* re.Match
* re.Pattern
* unicodedata.UCD
* zlib.Compress
* zlib.Decompress
2021-04-30 16:04:57 +02:00
Erlend Egeberg Aasland
387397f8a4
bpo-43916: select.poll uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25750) 2021-04-30 15:49:17 +02:00
Mark Shannon
5979e81a21
bpo-43933: Set frame.f_lineno during call to __exit__ (GH-25719)
* Set line number of __exit__ call in a with statement to be that of the with keyword.
2021-04-30 14:32:47 +01:00
Erlend Egeberg Aasland
64141382ec
bpo-43908: check_set_special_type_attr() checks Py_TPFLAGS_IMMUTABLETYPE (GH-25743)
check_set_special_type_attr() and type_set_annotations()
now check for immutable flag (Py_TPFLAGS_IMMUTABLETYPE).

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-30 15:25:43 +02:00
Victor Stinner
4908fae3d5
bpo-43916: PyStdPrinter_Type uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25749)
The PyStdPrinter_Type type now uses the
Py_TPFLAGS_DISALLOW_INSTANTIATION flag to disallow instantiation,
rather than seting a tp_init method which always fail.

Write also unit tests for PyStdPrinter_Type.
2021-04-30 14:56:27 +02:00
Victor Stinner
0cad068ec1
bpo-43916: Remove _disabled_new() function (GH-25745)
posix and _hashlib use the new Py_TPFLAGS_DISALLOW_INSTANTIATION
flag on their heap types, rather than using a custom tp_new function
(_disabled_new).
2021-04-30 14:06:49 +02:00
Victor Stinner
3bb09947ec
bpo-43916: Add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag (GH-25721)
Add a new Py_TPFLAGS_DISALLOW_INSTANTIATION type flag to disallow
creating type instances: set tp_new to NULL and don't create the
"__new__" key in the type dictionary.

The flag is set automatically on static types if tp_base is NULL or
&PyBaseObject_Type and tp_new is NULL.

Use the flag on the following types:

* _curses.ncurses_version type
* _curses_panel.panel
* _tkinter.Tcl_Obj
* _tkinter.tkapp
* _tkinter.tktimertoken
* _xxsubinterpretersmodule.ChannelID
* sys.flags type
* sys.getwindowsversion() type
* sys.version_info type

Update MyStr example in the C API documentation to use
Py_TPFLAGS_DISALLOW_INSTANTIATION.

Add _PyStructSequence_InitType() function to create a structseq type
with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag set.

type_new() calls _PyType_CheckConsistency() at exit.
2021-04-30 12:46:15 +02:00
larryhastings
74613a46fc
bpo-43817: Add inspect.get_annotations(). (#25522)
Add inspect.get_annotations, which safely computes the annotations defined on an object.  It works around the quirks of accessing the annotations from various types of objects, and makes very few assumptions about the object passed in. inspect.get_annotations can also correctly un-stringize stringized annotations.

inspect.signature, inspect.from_callable, and inspect.from_function now call inspect.get_annotations to retrieve annotations.  This means inspect.signature and inspect.from_callable can now un-stringize stringized annotations, too.
2021-04-29 21:16:28 -07:00
larryhastings
175a54b2d8
Two minor fixes for accessing a module's name. (#25658)
While working on another issue, I noticed two minor nits in the C implementation of the module object.  Both are related to getting a module's name.

First, the C function module_dir() (module.__dir__) starts by ensuring the module dict is valid.  If the module dict is invalid, it wants to format an exception using the name of the module, which it gets from PyModule_GetName().  However, PyModule_GetName() gets the name of the module from the dict.  So getting the name in this circumstance will never succeed.

When module_dir() wants to format the error but can't get the name, it knows that PyModule_GetName() must have already raised an exception.  So it leaves that exception alone and returns an error.  The end result is that the exception raised here is kind of useless and misleading: dir(module) on a module with no __dict__ raises SystemError("nameless module").  I changed the code to actually raise the exception it wanted to raise, just without a real module name: TypeError("<module>.__dict__ is not a dictionary").  This seems more useful, and would do a better job putting the programmer who encountered this on the right track of figuring out what was going on.

Second, the C API function PyModule_GetNameObject() checks to see if the module has a dict.  If m->md_dict is not NULL, it calls _PyDict_GetItemIdWithError().  However, it's possible for m->md_dict to be None.  And if you call _PyDict_GetItemIdWithError(Py_None, ...) it will *crash*.

Unfortunately, this crash was due to my own bug in the other branch.  Fixing my code made the crash go away.  I assert that this is still possible at the API level.

The fix is easy: add a PyDict_Check() to PyModule_GetNameObject().

Unfortunately, I don't know how to add a unit test for this.  Having changed module_dir() above, I can't find any other interfaces callable from Python that eventually call PyModule_GetNameObject().  So I don't know how to trick the runtime into reproducing this error.

Since both these changes are minor--each entails only a small edit to only one line--I didn't bother with a news item.
2021-04-29 20:13:25 -07:00
larryhastings
2f2b69855d
bpo-43901: Lazy-create an empty annotations dict in all unannotated user classes and modules (#25623)
Change class and module objects to lazy-create empty annotations dicts on demand.  The annotations dicts are stored in the object's `__dict__` for backwards compatibility.
2021-04-29 20:09:08 -07:00
Brandt Bucher
dbe60ee09d
bpo-43892: Validate the first term of complex literal value patterns (GH-25735) 2021-04-29 17:19:28 -07:00
Mark Shannon
088a15c49d
bpo-43933: Show frame.f_lineno as None, rather than -1, if there is no line number. (GH-25717) 2021-04-29 19:28:50 +01:00
Senthil Kumaran
76cd81d603
bpo-43882 - urllib.parse should sanitize urls containing ASCII newline and tabs. (GH-25595)
* issue43882 - urllib.parse should sanitize urls containing ASCII newline and tabs.

Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2021-04-29 10:16:50 -07:00
Mark Shannon
c76da79b37
bpo-42739: Don't use sentinels to mark end of line table. (GH-25657)
* Add length parameter to PyLineTable_InitAddressRange and doen't use sentinel values at end of table. Makes the line number table more robust.

* Update PyCodeAddressRange to match PEP 626.
2021-04-29 13:12:51 +01:00
Inada Naoki
53dd6c99b3
bpo-43651: Fix test_logging (GH-25715) 2021-04-29 20:37:32 +09:00