Commit graph

13395 commits

Author SHA1 Message Date
Serhiy Storchaka
2da967ea14
[3.12] gh-99593: Add tests for Unicode C API (part 3) (GH-104728) (GH-106595)
Add tests for codecs.
(cherry picked from commit 51ea664d18)
2023-07-10 20:58:58 +03:00
Miss Islington (bot)
41057b2ffe
[3.12] gh-105227: Add PyType_GetDict() (GH-105747) (#106600)
gh-105227: Add PyType_GetDict() (GH-105747)

This compensates for static builtin types having `tp_dict` set to `NULL`.

(cherry picked from commit a840806d33)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2023-07-10 17:12:15 +00:00
Miss Islington (bot)
8fdb058793
[3.12] gh-106524: Fix a crash in _sre.template() (GH-106525) (GH-106544)
Some items remained uninitialized if _sre.template() was called with invalid
indices. Then attempt to clear them in the destructor led to dereferencing
of uninitialized pointer.
(cherry picked from commit 2ef1dc37f0)

Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
2023-07-08 08:12:33 +00:00
Miss Islington (bot)
887a7e6036
[3.12] gh-91053: make func watcher tests resilient to other func watchers (GH-106286) (#106365)
gh-91053: make func watcher tests resilient to other func watchers (GH-106286)
(cherry picked from commit 58906213cc)

Co-authored-by: Carl Meyer <carl@oddbird.net>
2023-07-03 09:13:21 -06:00
Miss Islington (bot)
04a165f4f9
[3.12] GH-106152: Add PY_THROW event to cProfile (GH-106256)
GH-106152: Add PY_THROW event to cProfile (GH-106161)
(cherry picked from commit cea9d4ea82)

Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
2023-06-29 23:39:18 +00:00
Kumar Aditya
9e6f8d4615
[3.12] gh-105987: Fix reference counting issue in `_asyncio._swap_cur… (#106099)
[3.12] gh-105987: Fix reference counting issue in `_asyncio._swap_current_task` (GH-105989).
(cherry picked from commit d2cbb6e918)

Co-authored-by: chgnrdv <52372310+chgnrdv@users.noreply.github.com>
2023-06-26 08:51:28 +00:00
Serhiy Storchaka
9cd366462b
[3.12] gh-106033: Get rid of new occurrences of PyDict_GetItem and Py… (#106041)
[3.12] gh-106033: Get rid of new occurrences of PyDict_GetItem and PyObject_HasAttr (GH-106034)

These functions are broken by design because they discard any exceptions raised
inside, including MemoryError and KeyboardInterrupt.  They should not be
used in new code.
(cherry picked from commit 1d33d53780)
2023-06-24 16:36:34 -07:00
Eric Snow
33d3069c45
[3.12] gh-104812: Run Pending Calls in any Thread (gh-104813) (gh-105752)
For a while now, pending calls only run in the main thread (in the main interpreter).  This PR changes things to allow any thread run a pending call, unless the pending call was explicitly added for the main thread to run.
(cherry picked from commit 757b402)
2023-06-14 00:50:08 +00:00
Miss Islington (bot)
c3a2cbb54d
[3.12] gh-105603: Change the PyInterpreterConfig.own gil Field (gh-105620) (gh-105731)
We are changing it to be more flexible that a strict bool can be for possible future expanded used cases.
(cherry picked from commit b97e14a806)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2023-06-13 17:42:56 +00:00
Erlend E. Aasland
56877e4560
[3.12] gh-105375: Improve error handling in _Unpickler_SetInputStream() (#105667) (#105720)
Prevent exceptions from possibly being overwritten in case of multiple
failures.

(cherry picked from commit 217589d4f3)
2023-06-13 09:15:19 +00:00
Miss Islington (bot)
017b9595d4
[3.12] gh-105375: Explicitly initialise all {Pickler,Unpickler}Object fields (GH-105686) (#105710)
All fields must be explicitly initialised to prevent manipulation of
uninitialised fields in dealloc.

Align initialisation order with the layout of the object structs.

(cherry picked from commit ca3cc4b95d)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-12 22:12:17 +00:00
Miss Islington (bot)
77bdeebdda
[3.12] gh-105436: The environment block should end with two null wchar_t values (GH-105495) (#105700)
gh-105436: The environment block should end with two null wchar_t values (GH-105495)
(cherry picked from commit 4f7d3b602d)

Co-authored-by: Dora203 <66343334+sku2000@users.noreply.github.com>
2023-06-12 16:50:58 +00:00
Miss Islington (bot)
2eed1f5868
[3.12] gh-103968: PyType_FromMetaclass: Allow metaclasses with tp_new=NULL (GH-105386) (GH-105697)
gh-103968: PyType_FromMetaclass: Allow metaclasses with tp_new=NULL (GH-105386)
(cherry picked from commit 2b90796be6)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2023-06-12 16:24:21 +00:00
Erlend E. Aasland
d310fc776e
[3.12] gh-105375: Harden pyexpat initialisation (#105606) (#105669)
(cherry picked from commit 20a56d8bec)

Add proper error handling to add_errors_module() to prevent exceptions
from possibly being overwritten.
2023-06-11 21:35:13 +00:00
Miss Islington (bot)
82ac2be6b3
[3.12] gh-105375: Improve error handling in _ctypes (GH-105593) (#105663)
Prevent repeated PyLong_FromVoidPtr() from possibly overwriting the
current exception.
(cherry picked from commit e8998e46a7)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 20:25:27 +00:00
Miss Islington (bot)
c14f6ea7e4
[3.12] gh-105375: Improve posix error handling (GH-105592) (#105598)
Fix a bug where an IndexError could end up being overwritten.
(cherry picked from commit f668f73bc8)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 21:33:30 +02:00
Miss Islington (bot)
b4b5565e47
[3.12] gh-105375: Improve _decimal error handling (GH-105605) (#105647)
Fix a bug where an exception could end up being overwritten.
(cherry picked from commit c932f72849)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 10:55:18 +00:00
Miss Islington (bot)
36ecbc3570
[3.12] gh-105375: Harden _datetime initialisation (GH-105604) (#105645)
Improve error handling so init bails on the first exception.
(cherry picked from commit 16d49680b5)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 10:46:33 +00:00
Miss Islington (bot)
85a1a0983f
[3.12] gh-105375: Harden _ssl initialisation (GH-105599) (#105642)
Add proper error handling to prevent reference leaks and overwritten
exceptions.
(cherry picked from commit 01f4230460)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 10:26:52 +00:00
Miss Islington (bot)
d3c69ed366
[3.12] gh-105375: Improve array.array exception handling (GH-105594) (#105644)
Fix a bug where 'tp_richcompare' could end up overwriting an exception.
(cherry picked from commit 35cff545db)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-11 10:24:22 +00:00
Miss Islington (bot)
8c4cf96a06
[3.12] gh-105375: Improve error handling in zoneinfo module (GH-105586) (#105612)
Fix bugs where exceptions could end up being overwritten
because of deferred error handling.

(cherry picked from commit 33c92c4f15)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-09 23:22:10 +00:00
Miss Islington (bot)
4fbbf699c0
[3.12] gh-105375: Harden error handling in _testcapi/heaptype.c (GH-105608) (#105615)
Bail on first error in heapctypesubclasswithfinalizer_finalize()
(cherry picked from commit d636d7dfe7)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-09 23:21:42 +00:00
Miss Islington (bot)
411366ccdb
[3.12] gh-105375: Improve error handling in _elementtree (GH-105591) (#105600)
Fix bugs where exceptions could end up being overwritten.
(cherry picked from commit 00b599ab5a)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-09 21:15:24 +00:00
Erlend E. Aasland
e0087df65d
[3.12] gh-105375: Improve errnomodule error handling (#105590) (#105596)
(cherry picked from commit eede1d2f48)

Bail immediately if an exception is set, to prevent exceptions from
being overwritten.
2023-06-09 20:36:53 +00:00
Erlend E. Aasland
04b76ece7f
[3.12] gh-105375: Improve _pickle error handling (#105475) (#105584)
(cherry picked from commit 89aac6f6b7)

Error handling was deferred in some cases, which could potentially lead
to exceptions being overwritten.
2023-06-09 18:08:32 +00:00
Miss Islington (bot)
1e128779be
[3.12] gh-94673: Ensure subtypes are readied only once in math.trunc() (gh-105465) (gh-105471)
Fixes a typo in d2e2e53.
(cherry picked from commit 5394bf92aa)

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
2023-06-07 20:11:10 +00:00
Miss Islington (bot)
d3092573a8
[3.12] Fix grammar and improve clarity for an deprecation message. (GH-105457) (#105458) 2023-06-07 12:41:04 -05:00
Miss Islington (bot)
bb6ea72003
[3.12] gh-105375: Improve error handling in sqlite3 collation callback (GH-105412) (#105440)
Check for error after each call to PyUnicode_FromStringAndSize().
(cherry picked from commit a24a780d93)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-07 11:43:18 +00:00
Miss Islington (bot)
27cbeb08b8
[3.12] gh-104399: Use newer libtommath APIs when necessary (GH-104407) (#105343)
gh-104399: Use newer libtommath APIs when necessary (GH-104407)
(cherry picked from commit 00d73caf80)

Co-authored-by: Christopher Chavez <chrischavez@gmx.us>
2023-06-06 10:01:28 -04:00
Victor Stinner
82ab13c49a
[3.12] gh-102304: doc: Add links to Stable ABI and Limited C API (#105345) (#105371)
* gh-102304: doc: Add links to Stable ABI and Limited C API (#105345)

* Add "limited-c-api" and "stable-api" references.
* Rename "stable-abi-list" reference to "limited-api-list".
* Makefile: Document files regenerated by "make regen-limited-abi"
* Remove first empty line in generated files:

  - Lib/test/test_stable_abi_ctypes.py
  - PC/python3dll.c

(cherry picked from commit bae415ad02)

* gh-102304: Fix up Simple ABI doc (GH-105351)

(cherry picked from commit 0202aa002e)
2023-06-06 13:11:28 +00:00
Miss Islington (bot)
c7a9d96a25
[3.12] gh-104690 Disallow thread creation and fork at interpreter finalization (GH-104826) (#105277)
gh-104690 Disallow thread creation and fork at interpreter finalization (GH-104826)

Disallow thread creation and fork at interpreter finalization.

in the following functions, check if interpreter is finalizing and raise `RuntimeError` with appropriate message:
* `_thread.start_new_thread` and thus `threading`
* `posix.fork`
* `posix.fork1`
* `posix.forkpty`
* `_posixsubprocess.fork_exec` when a `preexec_fn=` is supplied.

---------

(cherry picked from commit ce558e69d4)

Co-authored-by: chgnrdv <52372310+chgnrdv@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-06-04 04:32:00 +00:00
Miss Islington (bot)
72d5dfaa8f
[3.12] gh-104614: Fix potential ref. leak in _testcapimodule/get_basic_static_type() (GH-105225) (#105248)
(cherry picked from commit e01b04c907)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-02 17:09:51 +00:00
Miss Islington (bot)
d57ee813eb
[3.12] gh-104614: Make Sure ob_type is Always Set Correctly by PyType_Ready() (gh-105122) (gh-105211)
When I added the relevant condition to type_ready_set_bases() in gh-103912, I had missed that the function also sets tp_base and ob_type (if necessary).  That led to problems for third-party static types.

We fix that here, by making those extra operations distinct and by adjusting the condition to be more specific.
(cherry picked from commit 1469393)

Co-authored-by: Eric Snow ericsnowcurrently@gmail.com
2023-06-01 23:06:38 +00:00
Miss Islington (bot)
c38ceb032d
[3.12] gh-105020: Share tp_bases and tp_mro Between Interpreters For All Static Builtin Types (gh-105115) (gh-105124)
In gh-103912 we added tp_bases and tp_mro to each PyInterpreterState.types.builtins entry.  However, doing so ignored the fact that both PyTypeObject fields are public API, and not documented as internal (as opposed to tp_subclasses).  We address that here by reverting back to shared objects, making them immortal in the process.
(cherry picked from commit 7be667d)

Co-authored-by: Eric Snow ericsnowcurrently@gmail.com
2023-06-01 22:24:55 +00:00
Miss Islington (bot)
25543aecfe
[3.12] gh-103142: Upgrade binary builds and CI to OpenSSL 1.1.1u (GH-105174) (#105199)
gh-103142: Upgrade binary builds and CI to OpenSSL 1.1.1u (GH-105174)

Upgrade builds to OpenSSL 1.1.1u.

This OpenSSL version addresses a pile if less-urgent CVEs since 1.1.1t.

The Mac/BuildScript/build-installer.py was already updated.

Also updates _ssl_data_111.h from OpenSSL 1.1.1u, _ssl_data_300.h from 3.0.9, and adds a new _ssl_data_31.h file from 3.1.1 along with the ssl.c code to use it.

Manual edits to the _ssl_data_300.h file prevent it from removing any existing definitions in case those exist in some peoples builds and were important (avoiding regressions during backporting).

backports of this prior to 3.12 will not include the openssl 3.1 header.
(cherry picked from commit ede89af605)

Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
2023-06-01 17:13:35 +00:00
Miss Islington (bot)
6375287b37
[3.12] gh-89886: Rely on HAVE_SYS_TIME_H (GH-105058) (#105192)
Quoting autoconf (v2.71):

    All current systems provide time.h; it need not be checked for.
    Not all systems provide sys/time.h, but those that do, all allow
    you to include it and time.h simultaneously.

(cherry picked from commit 9ab587b714)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-01 12:08:37 +00:00
Miss Islington (bot)
8b516668f2
[3.12] gh-102251: Explicitly free state for test modules with state in test_import (GH-105085) (#105170)
(cherry picked from commit a99b9d911e)

Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-31 21:34:21 +00:00
Miss Islington (bot)
9ae49e3f3b
gh-88745: Add _winapi.CopyFile2 and update shutil.copy2 to use it (GH-105055)
(cherry picked from commit cda1bd3c9d)

Co-authored-by: Steve Dower <steve.dower@python.org>
2023-05-30 20:33:17 +01:00
Irit Katriel
b45df737d4
[3.12] gh-105071: add PyUnstable_Exc_PrepReraiseStar to expose except* implementation in the unstable API (GH-105072) (#105095)
(cherry picked from commit b7aadb4583)
2023-05-30 16:50:23 +01:00
Miss Islington (bot)
fd6b913535
[3.12] gh-102251: Fix reference leak in _testsinglephase initialization (GH-105082) (#105083)
Correctly decref 'initialized' in init_module()

(cherry picked from commit d14eb3433c)

Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-30 09:53:46 +00:00
Miss Islington (bot)
68bf3fe0e4
gh-104820: Fixes os.stat on Windows to better handle file systems that do not support FileIdInformation (GH-104892)
(cherry picked from commit 6031727a37)

Co-authored-by: Steve Dower <steve.dower@python.org>
2023-05-29 17:57:04 +01:00
Miss Islington (bot)
635ce29257
gh-104803: Implement ntpath.isdevdrive for checking whether a path is on a Windows Dev Drive (GH-104805)
(cherry picked from commit bfd20d257e)

Co-authored-by: Steve Dower <steve.dower@python.org>
2023-05-29 13:36:08 +01:00
Miss Islington (bot)
05189f3054
[3.12] GH-101588: Deprecate pickle/copy/deepcopy support in itertools (GH-104965) (GH-104997) 2023-05-26 16:02:42 -05:00
Miss Islington (bot)
5c2971b78f
[3.12] gh-104372: Drop the GIL around the vfork() call. (GH-104782) (#104942)
gh-104372: Drop the GIL around the vfork() call. (GH-104782)

On Linux where the `subprocess` module can use the `vfork` syscall for
faster spawning, prevent the parent process from blocking other threads
by dropping the GIL while it waits for the vfork'ed child process `exec`
outcome.  This prevents spawning a binary from a slow filesystem from
blocking the rest of the application.

Fixes GH-104372.
(cherry picked from commit d08679212d)

Co-authored-by: Gregory P. Smith <gps@python.org>
2023-05-25 20:44:29 +00:00
Miss Islington (bot)
c4bc97a0aa
[3.12] gh-99108: Refresh HACL* (GH-104808) (#104893)
gh-99108: Refresh HACL* (GH-104808)

Refresh HACL* from upstream to improve SHA2 performance and fix a 32-bit issue in SHA3.
(cherry picked from commit 160321e530)

Co-authored-by: Jonathan Protzenko <protz@microsoft.com>
2023-05-24 20:55:37 +00:00
Miss Islington (bot)
b719dd8725
[3.12] gh-104372: use == -1 before PyErr_Occurred (GH-104831) (#104833)
gh-104372: use == -1 before PyErr_Occurred (GH-104831)

The ideal pattern for this.  (already in the 3.11 backport)
(cherry picked from commit 7f963bfc79)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-24 04:40:21 +00:00
Miss Islington (bot)
22c45c49bb
[3.12] gh-103295: fix stack overwrite on 32-bit in perf map test harness (GH-104811) (#104823)
gh-103295: fix stack overwrite on 32-bit in perf map test harness (GH-104811)
(cherry picked from commit e0b3078705)

Co-authored-by: Carl Meyer <carl@oddbird.net>
2023-05-23 22:34:50 +00:00
Miss Islington (bot)
9aea1f28e2
[3.12] gh-99108: Release the GIL around hashlib built-in computation (GH-104675) (#104776)
gh-99108: Release the GIL around hashlib built-in computation (GH-104675)

This matches the GIL releasing behavior of our existing `_hashopenssl`
module, extending it to the HACL* built-ins.

Includes adding comments to better describe the ENTER/LEAVE macros
purpose and explain the lock strategy in both existing and new code.
(cherry picked from commit 2e5d8a90aa)

Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
2023-05-23 11:24:02 +00:00
Kumar Aditya
b9c807a260
GH-103092: isolate _ssl (#104725) 2023-05-22 06:14:48 +05:30
Serhiy Storchaka
f3466bc040
gh-98836: Extend PyUnicode_FromFormat() (GH-98838)
* Support for conversion specifiers o (octal) and X (uppercase hexadecimal).
* Support for length modifiers j (intmax_t) and t (ptrdiff_t).
* Length modifiers are now applied to all integer conversions.
* Support for wchar_t C strings (%ls and %lV).
* Support for variable width and precision (*).
* Support for flag - (left alignment).
2023-05-22 00:32:39 +03:00