Commit graph

14993 commits

Author SHA1 Message Date
Adam Turner
d29ddbd90c
gh-132983: Convert zstd `__new__` methods to Argument Clinic (#133860) 2025-05-12 08:51:53 +00:00
Kumar Aditya
9b9cdb6440
gh-100926: use explicit stginfo lock for pointer cache (#133867) 2025-05-11 08:24:20 +00:00
Rogdham
878e0fb8b4
gh-132983: Remove leftovers from EndlessZstdDecompressor (#133856)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2025-05-11 02:04:25 +00:00
Adam Turner
1a87b6e9ae
gh-132983: Make zstd types immutable (#133784) 2025-05-10 22:37:17 +00:00
Adam Turner
1a548c0a50
gh-132983: Reduce the size of `_zstdmodule.h` (#133793) 2025-05-10 22:25:22 +01:00
Kumar Aditya
70f9b3de36
gh-100926: fix thread safety of ctypes __pointer_type__ (#133843) 2025-05-10 17:38:06 +00:00
Bénédikt Tran
116a9f9b37
gh-133009: fix UAF in xml.etree.ElementTree.Element.__deepcopy__ (#133010) 2025-05-10 09:32:39 +02:00
Adam Turner
1978904a2f
GH-132983: PEP 7 and Argument Clinic changes for zstd (#133791) 2025-05-10 00:33:45 +00:00
Adam Turner
98e2c3af47
GH-132983: remove empty_bytes from _zstd module state (#133785) 2025-05-09 20:17:12 +00:00
Adam Turner
bbe9c31edc
gh-132983: Simplify `_zstd_exec()` (#133775) 2025-05-09 20:15:19 +01:00
Adam Turner
c2a5d4b383
gh-132983: Clean-ups for `_zstd` (#133670) 2025-05-09 15:08:51 +01:00
Kumar Aditya
2cd24ebfe9
fix thread safety of io.StringIO.truncate (#133732) 2025-05-09 07:59:17 +00:00
Will Childs-Klein
6801bd32cb
gh-133623: Add ssl.HAS_PSK_TLS13 to detect external TLS 1.3 PSK support (#133624) 2025-05-09 09:09:09 +02:00
Tomasz Pytel
5dd3a3a58c
gh-132551: make io.BytesIO thread safe (#132616)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2025-05-08 18:51:36 +00:00
Adam Turner
bd7c5859c6
GH-132983: Remove subclassing support from zstd types (#133694)
For consistency with ``bz2``, ``lzma``, and ``zlib``.
2025-05-08 18:35:22 +00:00
Adam Turner
6f6f48d289
gh-103092: Support subinterpreters in `_zstd` (#133674) 2025-05-08 19:11:34 +01:00
Kumar Aditya
2d82ab761a
gh-132886: use relaxed atomics for sock_fd in gil builds in socket module (#133208) 2025-05-08 22:33:41 +05:30
Eric Snow
c81fa2b9cd
gh-132775: Add _PyCode_GetScriptXIData() (gh-133480)
This converts functions, code, str, bytes, bytearray, and memoryview objects to PyCodeObject,
and ensure that the object looks like a script.  That means no args, no return, and no closure.
_PyCode_GetPureScriptXIData() takes it a step further and ensures there are no globals.

We also add _PyObject_SupportedAsScript() to the internal C-API.
2025-05-08 15:07:46 +00:00
Xuehai Pan
f0f93ba5fa
gh-131942: Use the Python-specific Py_DEBUG macro rather than _DEBUG in Windows-related C code (GH-131944) 2025-05-08 15:01:25 +00:00
Bénédikt Tran
a2c4467d06
gh-133644: remove deprecated PyImport_ImportModuleNoBlock (#133655) 2025-05-08 13:08:43 +00:00
Serhiy Storchaka
dcf93c4c93
gh-133595: Clean up sqlite3.Connection APIs (GH-133605)
* All parameters of sqlite3.connect() except "database" are now keyword-only.
* The first three parameters of methods create_function() and
  create_aggregate() are now positional-only.
* The first parameter of methods set_authorizer(), set_progress_handler()
  and set_trace_callback() is now positional-only.
2025-05-08 15:42:00 +03:00
Serhiy Storchaka
4c914e7a36
gh-133583: Add support for fixed size unsigned integers in argument parsing (GH-133584)
* Add Argument Clinic converters: uint8, uint16, uint32, uint64.
* Add private C API: _PyLong_UInt8_Converter(),
  _PyLong_UInt16_Converter(), _PyLong_UInt32_Converter(),
  _PyLong_UInt64_Converter().
2025-05-08 12:27:50 +03:00
dgpb
afed5f8835
gh-125028: Prohibit placeholders in partial keywords (GH-126062) 2025-05-08 10:53:53 +03:00
Rogdham
2cc6de77bd
gh-132983: Remove pyzstd in identifiers (#133535) 2025-05-08 01:47:42 +01:00
Eric Snow
27128e4fa8
gh-132775: Unrevert "Add _PyCode_VerifyStateless()" (gh-133528)
This reverts commit 3c73cf5 (gh-133497), which itself reverted
the original commit d270bb5 (gh-133221).

We reverted the original change due to failing android tests.
The checks in _PyCode_CheckNoInternalState() were too strict,
so we've relaxed them.
2025-05-08 00:00:33 +00:00
Zachary Ware
74e2acddf6
Test fixes for 3.15 (GH-133599)
Followup to 942673ed19 (GH-133588)

* Update configure for Python 3.15

* Update magic number for 3.15

* Remove deprecated 'check_home' argument from sysconfig.is_python_build

* Add warningignore entries for Modules/_sqlite/clinic/connection.c.h

* Work around c-analyzer complaints about _testclinic deprecation tests

---------

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-05-07 14:50:39 -05:00
Max Bachmann
1460ccefd0
gh-133537: Avoid using console I/O in WinAPI partitions that don't support it (GH-133538) 2025-05-07 20:47:05 +01:00
Max Bachmann
427f8c366d
gh-133517: Remove os.listdrive, os.listvolumes and os.listmounts in non-desktop Windows builds (GH-133518) 2025-05-07 20:45:42 +01:00
Adam Turner
f8691901d7
GH-132983: Remove zstd version check in the header file (#133502) 2025-05-06 15:04:50 +03:00
Irit Katriel
296cd128bf
Revert "gh-133395: add option for extension modules to specialize BINARY_OP/SUBSCR, apply to arrays (#133396)" (#133498) 2025-05-06 13:12:26 +03:00
Petr Viktorin
3c73cf51df
gh-132775: Revert "gh-132775: Add _PyCode_VerifyStateless() (gh-133221)" (#133497) 2025-05-06 13:09:41 +03:00
Emma Smith
c273f59fb3
gh-132983: Add the compression.zstd pacakge and tests (#133365)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
Co-authored-by: Rogdham <contact@rogdham.net>
2025-05-06 01:38:08 +01:00
Adam Turner
e6f8e0a035
GH-132983: Build `_zstd` on Windows (#133366) 2025-05-06 00:58:47 +01:00
Eric Snow
ea598730ef
gh-132775: Add _PyCode_GetXIData() (gh-133475) 2025-05-05 23:46:03 +00:00
Brandt Bucher
b1aa515bd6
GH-133231: Add JIT utilities in sys._jit (GH-133233) 2025-05-05 15:25:22 -07:00
Eric Snow
d270bb5792
gh-132775: Add _PyCode_VerifyStateless() (gh-133221)
"Stateless" code is a function or code object which does not rely on external state or internal state.
It may rely on arguments and builtins, but not globals or a closure. I've left a comment in
pycore_code.h that provides more detail.

We also add _PyFunction_VerifyStateless(). The new functions will be used in several later changes
that facilitate "sharing" functions and code objects between interpreters.
2025-05-05 21:48:58 +00:00
Eric Snow
24ebb9ccfd
gh-132775: Unrevert "Add _PyCode_GetVarCounts()" (gh-133265)
This reverts commit 811edcf (gh-133232), which itself reverted the original commit 811edcf (gh-133128).

We reverted the original change due to failing s390 builds (a big-endian architecture).
It ended up that I had not accommodated op caches.
2025-05-05 13:24:29 -06:00
Peter Bierma
b275b8f342
gh-133140: Add PyUnstable_Object_IsUniquelyReferenced for free-threading (#133144) 2025-05-05 21:01:20 +02:00
Irit Katriel
082dbf7788
gh-133395: add option for extension modules to specialize BINARY_OP/SUBSCR, apply to arrays (#133396) 2025-05-05 17:46:56 +01:00
Wulian233
b6c2ef0c7a
gh-131296: Avoid posixmodule.c warning (GH-133142)
Clang-cl detects that on 32-bit builds the variable is always smaller than the value. But since the same code is used for other architectures, we can't just _fix_ it. This cast avoids the tautological-constant-out-of-range-compare warning.
2025-05-05 17:45:15 +01:00
Stan Ulbrych
f5b784741d
gh-110067: Make max heap methods public and add missing ones (GH-130725)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2025-05-05 17:52:49 +02:00
Sergey B Kirpichev
1bc16504ef
gh-61103: drop unused Py_HAVE_C_COMPLEX define (GH-133435)
Py_HAVE_C_COMPLEX was added in 3.14 so the removal doesn't need a deprecation
period even under a strict reading of PEP 387.

The Py_FFI_SUPPORT_C_COMPLEX check configure check implies support for
complex types in ctypes.
2025-05-05 15:50:57 +02:00
Sergey B Kirpichev
8d0e07eb89
gh-61103: don't use C _Complex types to implement F/D/G in ctypes (GH-133237)
According to the C standard, the memory representation of _Complex types
is equivalent to 2-element arrays. Unlike _Complex, arrays are always available.

- drop _complex.h header
- use appropriate real arrays to replace complex types

Co-authored-by: Lisandro Dalcin <dalcinl@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
2025-05-05 11:23:30 +02:00
Adam Turner
3f80165a26
GH-91048: Minor fixes for `_remotedebugging & rename to _remote_debugging` (#133398) 2025-05-05 02:30:14 +02:00
Pablo Galindo Salgado
3109c47be8
gh-91048: Add filename and line number to external inspection routines (GH-133385)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2025-05-04 23:33:37 +02:00
Serhiy Storchaka
5a57248b22
gh-81793: Always call linkat() from os.link(), if available (GH-132517)
This fixes os.link() on platforms (like Linux and OpenIndiana) where the
system link() function does not follow symlinks.

* On Linux, it now follows symlinks by default and if
  follow_symlinks=True is specified.
* On Windows, it now raises error if follow_symlinks=True is passed.
* On macOS, it now raises error if follow_symlinks=False is passed and
  the system linkat() function is not available at runtime.
* On other platforms, it now raises error if follow_symlinks is passed
  with a value that does not match the system link() function behavior
  if if the behavior is not known.

Co-authored-by: Joachim Henke <37883863+jo-he@users.noreply.github.com>
Co-authored-by: Thomas Kluyver <takowl@gmail.com>
2025-05-04 17:24:10 +03:00
Serhiy Storchaka
e9253ebf74
gh-122559: Synchronize C and Python implementation of the io module about pickling (GH-122628)
In the C implementation, remove __reduce__ and __reduce_ex__ methods
that always raise TypeError and restore __getstate__ methods that always
raise TypeErrori.

This restores fine details of the pre-3.12 behavior and unifies
both implementations.
2025-05-04 14:06:49 +00:00
Michael Forney
0af61fe2f4
bpo-44172: Keep reference to original window in curses subwindow objects (GH-26226)
The X/Open curses specification[0] and ncurses documentation[1]
both state that subwindows must be deleted before the main window.

Deleting the windows in the wrong order causes a double-free with
NetBSD's curses implementation.

To fix this, keep track of the original window object in the subwindow
object, and keep a reference to the original for the lifetime of
the subwindow.

[0] https://pubs.opengroup.org/onlinepubs/7908799/xcurses/delwin.html
[1] https://invisible-island.net/ncurses/man/curs_window.3x.html

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-05-04 13:29:44 +03:00
Emma Smith
3b4333583f
gh-132983: Introduce _zstd bindings module (GH-133027)
* Add _zstd module for https://peps.python.org/pep-0784/

This commit introduces the `_zstd` module, with bindings to libzstd from
the pyzstd project. It also includes the unix build system configuration.
Windows build system support will be integrated independently as it
depends on integration with cpython-source-deps.

* Add _zstd to modules

* Fix path for compression.zstd module

* Ignore _zstd module like _io

* Expand module state macros to improve code quality

Also removes module state references from the classes in the _zstd
module and instead uses PyType_GetModuleState()

* Remove backticks suggested in review

Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>

* Use critical sections to lock object state

This should avoid races and deadlocks.

* Remove compress/decompress and mark module as not reliant on the GIL

The `compress`/`decompress` functions will be moved to Python code for simplicity.
C implementations can always be re-added in the future.

Also, mark _zstd as not requiring the GIL.

* Lift critical section to avoid clang warning

* Respond to comments by picnixz

* Call out pyzstd explicitly in license description

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>

* Use a much more robust implementation...

... for `get_zstd_state_from_type`

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* Use PyList_GetItemRef for thread safety purposes

* Use a macro for the minimum supported version

* remove const from primivite types

* Use PyMem_New in another spot

* Simplify error handling in _get_frame_size

* Another simplification of error handling in get_frame_info

* Rename _module_state to mod_state

* Rewrite comment explaining the context of the code

* Add link to pyzstd

* Add TODO about refactoring dict training code

* Use PyModule_AddObjectRef over PyModule_AddObject

PyModule_AddObject is soft-deprecated, so we should use PyModule_AddObjectRef

* Check result of OutputBufferGrow

* Simplify return logic in `add_constant_to_type`

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* Ignore return value of _zstd_clear()

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* Remove redundant comments

* Remove __reduce__ from ZstdDict

We should instead document that to pickle a dictionary a user should use
the `.dict_content` attribute.

* Use PyUnicode_FromFormat instead of a buffer

* Don't use C constants/types in error messages

* Make error messages easier to understand for Python users

* Lower minimum required version 1.4.0

* Use casts and make slot function signatures correct

* Be consistent with CPython on const usage

* Make else clauses in line with PEP 7

* Fix over-indented blocks in argument clinic

* Add critical section around ZSTD_DCtx_setParameter

* Add a TODO about refactoring critical sections

* Use Py_UNREACHABLE

* Move bytes operations out of Py_BEGIN_ALLOW_THREADS

* Add TODO about ensuring a lock is held

* Remove asserts that may not be correct

* Add TODO to make ZstdDict and others GC objects

* Make objects GC tracked

* Remove unused include

* Fix some memory issues

* Fix refleaks on module and in ZstdDict

* Update configure to check for ZDICT_finalizeDictionary

* Properly check version in configure

* exit(1) if check fails

* Use AC_RUN_IFELSE

* Use a define() to re-use version check

* Actually properly set _zstd module status based on version

---------

Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-05-04 01:29:55 +00:00
Pablo Galindo Salgado
2bc8365231
GH-91048: Add utils for printing the call stack for asyncio tasks (#133284) 2025-05-04 00:51:57 +00:00