Commit graph

14860 commits

Author SHA1 Message Date
Kumar Aditya
be763e550e
gh-127945: fix thread safety and add lock held assertions to paramfunc in ctypes (#132473) 2025-04-14 14:05:06 +05:30
Gregory P. Smith
7a29c9883f
GH-115322: fix ctypes call_function audit hook on 32-bit platforms (GH-132496)
* GH-115322: fix ctypes call_function audit hook on 32-bit platforms.

It was using a signed conversion to communicate the function id (pointer) value.
2025-04-14 06:22:29 +00:00
Serhiy Storchaka
1fc1df8dcc
gh-132099: Harmonize Bluetooth address handling (GH-132486)
Now all protocols always accept the Bluetooth address as string and
getsockname() always returns the Bluetooth address as string.

* BTPROTO_SCO now accepts not only bytes, but str.
* BTPROTO_SCO now checks address for embedded null.
* On *BSD, BTPROTO_HCI now accepts str instead of bytes.
* On FreeBSD, getsockname() for BTPROTO_HCI now returns str instead of bytes.
* On NetBSD and DragonFly BDS, BTPROTO_HCI now checks address for embedded null.
2025-04-14 08:58:56 +03:00
Robin Jadoul
2666a06d33
GH-115322: Add missing audit hooks (GH-115624)
Add extra audit hooks to catch C function calling from ctypes,
reading/writing files through readline and executing external
programs through _posixsubprocess.

* Make audit-tests for open pass when readline.append_history_file is unavailable
* Less direct testing of _posixsubprocess for audit hooks
* Also remove the audit hook from call_cdeclfunction now that _ctypes_callproc does it instead.
* reword the NEWS entry.
* mention readline in NEWS
* add versionchanged markers
* fix audit_events.rst versionadded
* doc lint

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2025-04-13 21:46:20 +00:00
Pieter Eendebak
9d127e83b9
gh-123471: Make concurrent iteration over itertools.repeat safe under free-threading (#131247) 2025-04-13 13:26:58 +05:30
Serhiy Storchaka
f2f86d3f45
gh-132429: Fix support of Bluetooth sockets on NetBSD and DragonFly BSD (GH-132431)
* Also add support for cid and bdaddr_type in the BTPROTO_L2CAP address on FreeBSD.
* Return cid in getsockname() for BTPROTO_L2CAP if it is not zero.
* Fix a compiler warning on FreeBSD.
2025-04-12 23:57:34 +03:00
Nicolas Trangez
2aab2db146
GH-132417: ctypes: Fix potential Py_DECREF(NULL) when handling functions returning PyObject * (#132418)
Some functions (such as `PyErr_Occurred`) with a `restype` set to `ctypes.py_object` may return NULL without setting an exception.
2025-04-12 07:40:34 +00:00
Bénédikt Tran
e0dffc54b8
gh-132097: fix runtime UB when calling expat handlers (#132265) 2025-04-12 09:20:11 +02:00
Serhiy Storchaka
e5f8b2c703
gh-85302: Add support for BTPROTO_SCO on FreeBSD (GH-131981)
BTPROTO_SCO has been supported on FreeBSD since 2008.
2025-04-11 19:38:17 +03:00
Xuehai Pan
26ae05e95c
gh-127405: Add ABIFLAGS to sysconfig variables on Windows (GH-131799) 2025-04-11 16:19:03 +01:00
Collin Funk
1b49c8c71b
gh-131127: Minimal build support on systems using LibreSSL (GH-131128) 2025-04-11 15:35:12 +03:00
Neil Schemenauer
d687900f98
gh-128384: Use a context variable for warnings.catch_warnings (gh-130010)
Make `warnings.catch_warnings()` use a context variable for holding
the warning filtering state if the `sys.flags.context_aware_warnings`
flag is set to true.  This makes using the context manager thread-safe in
multi-threaded programs.

Add the `sys.flags.thread_inherit_context` flag.  If true, starting a new
thread with `threading.Thread` will use a copy of the context
from the caller of `Thread.start()`.

Both these flags are set to true by default for the free-threaded build
and false for the default build.

Move the Python implementation of warnings.py into _py_warnings.py.

Make _contextvars a builtin module.

Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-04-09 16:18:54 -07:00
Kumar Aditya
d47584aae6
gh-131336: fix thread safety for ctypes functions (#132232) 2025-04-09 18:18:40 +00:00
Serhiy Storchaka
487cf3c170
gh-132305: Make Argument Clinic code compatible with Python 3.10 (#132306) 2025-04-09 19:10:02 +01:00
Serhiy Storchaka
f5f1ac84b3
gh-112068: C API: Add support of nullable arguments in PyArg_Parse (GH-121303) 2025-04-08 22:08:00 +03:00
Tian Gao
ab64130b57
gh-132250: Clear error in lsprof callback when method descriptor raises an excep… (#132251) 2025-04-08 13:36:47 -04:00
sobolevn
f7305a06c7
gh-115942: Add locked to several multiprocessing locks (#115944)
Co-authored-by: mpage <mpage@cs.stanford.edu>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-04-08 11:14:12 +03:00
Gregory P. Smith
c0de650024
gh-131809: Upgrade vendored expat to 2.7.1 (#132192) 2025-04-07 23:42:00 +03:00
Peter Bierma
8e260b384a
gh-128182: switch ctypes locking to critical sections (#132133)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-04-07 22:00:31 +05:30
sobolevn
3980718710
gh-132171: Fix _interpreters.run_string crash on string subclass (#132173) 2025-04-07 14:59:44 +03:00
sobolevn
f2daa96c81
gh-132174: Fix function name in error message of _interpreters.run_string (#132175)
Co-authored-by: Brian Schubert <brianm.schubert@gmail.com>
2025-04-07 11:23:52 +03:00
Sonny Ding
af8d1b9537
gh-122040: reword Modules/xxmodule.c module-level comment (#132201) 2025-04-07 08:03:03 +00:00
Bénédikt Tran
b865871486
gh-99108: add support for SIMD-accelerated HMAC-BLAKE2 (#132120) 2025-04-07 09:31:17 +02:00
Kumar Aditya
0dba59e689
gh-128182: add critical sections to ctypes arrays getters and setters (#132152) 2025-04-06 16:21:44 +05:30
Adam Turner
1755157207
GH-118761: Expose more core interpreter types in `_types` (#132103)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2025-04-05 18:11:07 +00:00
Chris Eibl
d827d4d018
gh-131591: Fix GENERATE_DEBUG_SECTION for clangcl on Windows (GH-132112) 2025-04-06 00:00:54 +08:00
Kumar Aditya
b9d8d99563
gh-128182: add critical section to _ctypes.Simple getters and setters (#132081) 2025-04-05 10:27:41 +00:00
Kumar Aditya
f7a8bc50db
gh-128182: add critical section to _ctypes.PyCData methods (#132082) 2025-04-05 15:24:58 +05:30
Adam Turner
231a50fa9a
gh-109599: Expose CapsuleType via the _types module (#131969) 2025-04-04 23:37:41 +01:00
Bénédikt Tran
0a97427ee5
gh-99108: Implement HACL* HMAC (#130157)
A new extension module, `_hmac`, now exposes the HACL* HMAC (formally verified) implementation.

The HACL* implementation is used as a fallback implementation when the OpenSSL implementation of HMAC
is not available or disabled. For now, only named hash algorithms are recognized and SIMD support provided
by HACL* for the BLAKE2 hash functions is not yet used.
2025-04-04 19:04:00 +02:00
Bénédikt Tran
04bc681e7c
gh-131938: Update exception message for Element.remove() when an element is not found (#131972)
The exception message for `xml.etree.ElementTree.Element.remove` when an element is not found
has been updated from "list.remove(x): x not in list" to "Element.remove(x): element not found".
2025-04-04 15:04:07 +00:00
Serhiy Storchaka
345baa77ba
gh-132075: Fix possible use of sockaddr structures with uninitialized members (GH-132076)
Now all structure members are initialized with zeroes by default.
2025-04-04 17:09:58 +03:00
Peter Bierma
6e91d1f9aa
gh-131974: Fix usages of locked_deref in ctypes (#131975) 2025-04-02 17:04:25 +02:00
Chris Eibl
e9556e1004
GH-131296: Disable MSVC-specific optimization in sre.c when compiling with clang-cl (GH-131593) 2025-04-01 17:29:45 +01:00
Bénédikt Tran
edbf7fb129
gh-111178: remove redundant casts for functions with correct signatures (#131673) 2025-04-01 17:18:11 +02:00
Victor Stinner
8cd29c2b53
gh-111178: Fix getsockaddrarg() undefined behavior (#131668)
Don't pass direct references to sockaddr members since their type may
not match PyArg_ParseTuple() types. Instead, use temporary 'int' and
'unsigned char' variables, and update sockaddr members afterwards.

On FreeBSD, treat BTPROTO_HCI node name as a bytes string,
not as an integer.
2025-04-01 13:54:27 +00:00
Peter Bierma
511d3440a0
gh-131936: Strengthen check in _suggestions._generate_suggestions (#131945) 2025-03-31 19:30:29 +02:00
Chris Eibl
9127b4602e
GH-131296: Fix incompatible pointer type warning in _winapi.c (GH-131600) 2025-03-31 17:33:55 +01:00
Chris Eibl
7cbcb6efd9
GH-131296: Fix parenthesis warnings on Windows-specific code (GH-131905) 2025-03-31 15:54:30 +01:00
Bénédikt Tran
bab1398a47
gh-126033: fix UAF in xml.etree.ElementTree.Element.remove when concurrent mutations happen (#126124) 2025-03-31 12:31:26 +02:00
Bénédikt Tran
c57623c221
gh-126037: fix UAF in xml.etree.ElementTree.Element.find* when concurrent mutations happen (#127964)
We fix a use-after-free in the `find`, `findtext` and `findall` methods of `xml.etree.ElementTree.Element`
objects that can be triggered when the tag to find implements an `__eq__` method that mutates the
element being queried.
2025-03-31 12:26:52 +02:00
Kumar Aditya
46ada1e5e0
gh-127945: acquire critical section around PyCFuncPtr_call (#131898) 2025-03-30 11:01:12 +00:00
Kumar Aditya
28e476f6a2
gh-127945: make initialization of error_object_name thread safe in ctypes (#131896) 2025-03-30 10:50:35 +00:00
Kumar Aditya
bc5a028c13
gh-127945: fix thread safety of creating instances of ctypes structures (#131716) 2025-03-30 15:22:30 +05:30
Steve Dower
d260631be0
gh-131423: Update to OpenSSL 3.0.16. (GH-131839)
The bin tag is 3.0.16.1 because we rebuilt without uplink support to fix gh-131804.
This PR also prevents making calls that are now unsafe without uplink, and updates
the tests to property interpret these failures as unsupported.
2025-03-28 15:07:57 +00:00
Michael Droettboom
8614f86b71
gh-131525: Cache the result of tuple_hash (#131529)
* gh-131525: Cache the result of tuple_hash

* Fix debug builds

* Add blurb

* Fix formatting

* Pre-compute empty tuple singleton

* Mostly set the cache within tuple_alloc

* Fixes for TSAN

* Pre-compute empty tuple singleton

* Fix for 32-bit platforms

* Assert that op != NULL in _PyTuple_RESET_HASH_CACHE

* Use FT_ATOMIC_STORE_SSIZE_RELAXED macro

* Update Include/internal/pycore_tuple.h

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* Fix alignment

* atomic load

* Update Objects/tupleobject.c

Co-authored-by: Chris Eibl <138194463+chris-eibl@users.noreply.github.com>

---------

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Chris Eibl <138194463+chris-eibl@users.noreply.github.com>
2025-03-27 09:57:06 -04:00
Sam Gross
67fbfb42bd
gh-131586: Avoid refcount contention in some "special" calls (#131588)
In the free threaded build, the `_PyObject_LookupSpecial()` call can lead to
reference count contention on the returned function object becuase it
doesn't use stackrefs. Refactor some of the callers to use
`_PyObject_MaybeCallSpecialNoArgs`, which uses stackrefs internally.

This fixes the scaling bottleneck in the "lookup_special" microbenchmark
in `ftscalingbench.py`. However, the are still some uses of
`_PyObject_LookupSpecial()` that need to be addressed in future PRs.
2025-03-26 14:38:47 -04:00
Peter Bierma
90b82f2b61
gh-129900: Fix SystemExit return codes when the REPL is started from the command line (#129901) 2025-03-25 19:48:46 +00:00
Kumar Aditya
f1967e7249
gh-127945: fix thread safety of ctypes state (#131710)
This fixes thread safety of `array_cache` and `swapped_suffix` by initializing them in module exec to make it thread safety.
2025-03-25 17:03:05 +05:30
Kumar Aditya
96ef4c511f
gh-127945: add locking to malloc closure in free-threading (#131662)
The freelist is not thread safe in free-threading so this adds lock around it make it thread safe in free-threading.
2025-03-25 16:48:46 +05:30