This reverts commit 3c73cf5 (gh-133497), which itself reverted
the original commit d270bb5 (gh-133221).
We reverted the original change due to failing android tests.
The checks in _PyCode_CheckNoInternalState() were too strict,
so we've relaxed them.
Followup to 942673ed19 (GH-133588)
* Update configure for Python 3.15
* Update magic number for 3.15
* Remove deprecated 'check_home' argument from sysconfig.is_python_build
* Add warningignore entries for Modules/_sqlite/clinic/connection.c.h
* Work around c-analyzer complaints about _testclinic deprecation tests
---------
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
"Stateless" code is a function or code object which does not rely on external state or internal state.
It may rely on arguments and builtins, but not globals or a closure. I've left a comment in
pycore_code.h that provides more detail.
We also add _PyFunction_VerifyStateless(). The new functions will be used in several later changes
that facilitate "sharing" functions and code objects between interpreters.
This reverts commit 811edcf (gh-133232), which itself reverted the original commit 811edcf (gh-133128).
We reverted the original change due to failing s390 builds (a big-endian architecture).
It ended up that I had not accommodated op caches.
For the free-threaded build, check the process resident set size (RSS)
increase before triggering a full automatic garbage collection. If the RSS
has not increased 10% since the last collection then it is deferred.
In certain cases it's possible for locals loaded by `LOAD_FAST` instructions
to be on the stack when the local is killed by `DEL_FAST`. These `LOAD_FAST`
instructions should not be optimized into `LOAD_FAST_BORROW` as the strong
reference in the frame is killed while there is still a reference on the stack.
* Track the current executor, not the previous one, on the thread-state.
* Batch executors for deallocation to avoid having to constantly incref executors; this is an ad-hoc form of deferred reference counting.
* Add _zstd module for https://peps.python.org/pep-0784/
This commit introduces the `_zstd` module, with bindings to libzstd from
the pyzstd project. It also includes the unix build system configuration.
Windows build system support will be integrated independently as it
depends on integration with cpython-source-deps.
* Add _zstd to modules
* Fix path for compression.zstd module
* Ignore _zstd module like _io
* Expand module state macros to improve code quality
Also removes module state references from the classes in the _zstd
module and instead uses PyType_GetModuleState()
* Remove backticks suggested in review
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
* Use critical sections to lock object state
This should avoid races and deadlocks.
* Remove compress/decompress and mark module as not reliant on the GIL
The `compress`/`decompress` functions will be moved to Python code for simplicity.
C implementations can always be re-added in the future.
Also, mark _zstd as not requiring the GIL.
* Lift critical section to avoid clang warning
* Respond to comments by picnixz
* Call out pyzstd explicitly in license description
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* Use a much more robust implementation...
... for `get_zstd_state_from_type`
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* Use PyList_GetItemRef for thread safety purposes
* Use a macro for the minimum supported version
* remove const from primivite types
* Use PyMem_New in another spot
* Simplify error handling in _get_frame_size
* Another simplification of error handling in get_frame_info
* Rename _module_state to mod_state
* Rewrite comment explaining the context of the code
* Add link to pyzstd
* Add TODO about refactoring dict training code
* Use PyModule_AddObjectRef over PyModule_AddObject
PyModule_AddObject is soft-deprecated, so we should use PyModule_AddObjectRef
* Check result of OutputBufferGrow
* Simplify return logic in `add_constant_to_type`
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* Ignore return value of _zstd_clear()
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* Remove redundant comments
* Remove __reduce__ from ZstdDict
We should instead document that to pickle a dictionary a user should use
the `.dict_content` attribute.
* Use PyUnicode_FromFormat instead of a buffer
* Don't use C constants/types in error messages
* Make error messages easier to understand for Python users
* Lower minimum required version 1.4.0
* Use casts and make slot function signatures correct
* Be consistent with CPython on const usage
* Make else clauses in line with PEP 7
* Fix over-indented blocks in argument clinic
* Add critical section around ZSTD_DCtx_setParameter
* Add a TODO about refactoring critical sections
* Use Py_UNREACHABLE
* Move bytes operations out of Py_BEGIN_ALLOW_THREADS
* Add TODO about ensuring a lock is held
* Remove asserts that may not be correct
* Add TODO to make ZstdDict and others GC objects
* Make objects GC tracked
* Remove unused include
* Fix some memory issues
* Fix refleaks on module and in ZstdDict
* Update configure to check for ZDICT_finalizeDictionary
* Properly check version in configure
* exit(1) if check fails
* Use AC_RUN_IFELSE
* Use a define() to re-use version check
* Actually properly set _zstd module status based on version
---------
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
There's some extra complexity due to making sure we we get things right when handling functions and classes defined in the __main__ module. This is also reflected in the tests, including the addition of extra functions in test.support.import_helper.
This helper is useful in a variety of ways, including in demonstrating how the different counts relate to one another.
It will be used in a later change to help identify if a function is "stateless", meaning it doesn't have any free vars or globals.
Note that a majority of this change is tests.
The function indicates whether or not the function has a return statement.
This is used by a later change related treating some functions like scripts.
In the free-threaded build, avoid data races caused by updating type slots
or type flags after the type was initially created. For those (typically
rare) cases, use the stop-the-world mechanism. Remove the use of atomics
when reading or writing type flags. The use of atomics is not sufficient to
avoid races (since flags are sometimes read without a lock and without
atomics) and are no longer required.
We replace it with _Py_GetMainModule(), and add _Py_CheckMainModule(), but both in the internal-only C-API. We also add _PyImport_GetModulesRef(), which is the equivalent of _PyImport_GetModules(), but which increfs before the lock is released.
This is used by a later change related to pickle and handling __main__.
The following are added to the internal C-API:
* _PyErr_FormatV()
* _PyErr_SetModuleNotFoundError()
* _PyXIData_GetNotShareableErrorType()
* _PyXIData_FormatNotShareableError()
We also drop _PyXIData_lookup_context_t and _PyXIData_GetLookupContext().
For the same reasons as running the GC, this will allow sections that
run in native code for long periods without executing bytecode to also
run the remote debugger protocol without having to wait until bytecode
is executed
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
