Document the architectures supported by macOS universal SDK configuration flags,
and add details on wheel tag naming.
(cherry picked from commit 474c388740)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
gh-129438: Update ``--enable-experimental-jit`` section with install requirements (GH-129450)
Add a note to the `JIT` docs that building CPython with `JIT` requires Python 3.11 or newer.
(cherry picked from commit 652f66ac38)
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
* gh-116510: Fix a Crash Due to Shared Immortal Interned Strings (gh-124865)
Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.
This is an un-revert of gh-124646 that then addresses the Py_TRACE_REFS
failures identified by gh-124785.
(cherry picked from commit f2cb399470)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.13] gh-125286: Share the Main Refchain With Legacy Interpreters (gh-125709)
They used to be shared, before 3.12. Returning to sharing them resolves a failure on Py_TRACE_REFS builds.
---------
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-126647: `Doc/using/configure.rst`: Add an entry for ``--enable-experimental-jit`` option (GH-126648)
Add an entry for the ``--enable-experimental-jit`` option in ``Doc/using/configure.rst``.
This was added as an experimental option in CPython 3.13.
Possible values for it:
* `no` - don't build the JIT.
* `yes` - build the JIT.
* `yes-off` - build the JIT but disable it by default.
* `interpreter` - don't build the JIT but enable tier 2 interpreter instead.
(cherry picked from commit f435de6765)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-112088: aclocal version is updated to 1.16.5 in docs (GH-125457)
(cherry picked from commit 45df264f3f)
Co-authored-by: Mikhail Efimov <efimov.mikhail@gmail.com>
gh-120522: Apply App Store compliance patch during installation (GH-121947)
Adds a --with-app-store-compliance configuration option that patches out code known to be an issue with App Store review processes. This option is applied automatically on iOS, and optionally on macOS.
(cherry picked from commit 728432c804)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
This reverts commit 0dfb437a32 prior
to the release of 3.13.0b4 to allow for additional review time.
(cherry picked from commit f27593a87c)
Co-authored-by: Ned Deily <nad@python.org>
gh-120522: Add a `--with-app-store-compliance` configure option to patch out problematic code (GH-120984)
* Add --app-store-compliance configuration option.
* Added blurb.
* Correct tab-vs-spaces formatting issue.
* Correct source file name in docs.
* Correct source code reference in Mac docs
* Only apply the patch forward, and ensure the working directory is correct.
* Make patching reslient to multiple builds.
* Documentation fixes found during review
* Documentation and configure.ac syntax improvements
* Regenerate configure script.
* Silence the patch echo output.
---------
(cherry picked from commit 48cd104b0c)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Jacob Coffee <jacob@z7x.org>
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Ned Deily <nad@python.org>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
If Py_NOGIL is defined and Py_SET_REFCNT() is called with a reference
count larger than UINT32_MAX, make the object immortal.
Set _Py_IMMORTAL_REFCNT constant type to Py_ssize_t to fix the
following compiler warning:
Include/internal/pycore_global_objects_fini_generated.h:14:24:
warning: comparison of integers of different signs: 'Py_ssize_t'
(aka 'long') and 'unsigned int' [-Wsign-compare]
if (Py_REFCNT(obj) < _Py_IMMORTAL_REFCNT) {
~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
The "Check if generated files are up to date" job of GitHub Actions
now runs the "autoreconf -ivf -Werror" command instead of the "make
regen-configure" command to avoid depending on the external quay.io
server.
Add Tools/build/regen-configure.sh script to regenerate the configure
with an Ubuntu container image. The
"quay.io/tiran/cpython_autoconf:271" container image
(https://github.com/tiran/cpython_autoconf) is no longer used.
* Add mimalloc v2.12
Modified src/alloc.c to remove include of alloc-override.c and not
compile new handler.
Did not include the following files:
- include/mimalloc-new-delete.h
- include/mimalloc-override.h
- src/alloc-override-osx.c
- src/alloc-override.c
- src/static.c
- src/region.c
mimalloc is thread safe and shares a single heap across all runtimes,
therefore finalization and getting global allocated blocks across all
runtimes is different.
* mimalloc: minimal changes for use in Python:
- remove debug spam for freeing large allocations
- use same bytes (0xDD) for freed allocations in CPython and mimalloc
This is important for the test_capi debug memory tests
* Don't export mimalloc symbol in libpython.
* Enable mimalloc as Python allocator option.
* Add mimalloc MIT license.
* Log mimalloc in Lib/test/pythoninfo.py.
* Document new mimalloc support.
* Use macro defs for exports as done in:
https://github.com/python/cpython/pull/31164/
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
Remove replace_stdout(): call sys.stdout.reconfigure() instead of set
the error handler to backslashreplace.
display_header() logs an empty line and flush stdout.
Remove encoding workaround in display_header() since stdout error
handler is now set to backslashreplace earlier.
* Add --fast-ci and --slow-ci options to libregrtest:
* --fast-ci uses a default timeout of 10 minutes and "-u all,-cpu"
(skip slowest tests).
* --slow-ci uses a default timeout of 20 minues and "-u all" (run
all tests).
* regrtest header now lists test resources.
* Makefile changes:
* "make test", "make hostrunnertest" and "make coverage-report" now
use --fast-ci option and TESTTIMEOUT variable.
* "make buildbottest" now uses "--slow-ci". Remove options which
became redundant with "--slow-ci".
* "make testall" and "make testuniversal" now use --slow-ci option
and TESTTIMEOUT variable.
* "make testall" now uses "find -exec rm ..." instead of
"find ... -print|xargs rm ...", same as "make clean".
* GitHub Actions workflow:
* Ubuntu and Address Sanitizer jobs now use "make test". Remove
options which became redundant with "--fast-ci".
* Windows jobs now use --fast-ci option.
* Use -j0 to detect the number of CPUs.
* Set Makefile TESTTIMEOUT default to an empty string, since
--slow-ci and --fast-ci use different default timeout. It's now
accepted to pass "--timeout=" to regrtest: treated as not timeout.
* Tools/scripts/run_tests.py now uses --fast-ci option.
* Tools/buildbot/test.bat now uses --slow-ci option. Remove
--timeout=1200 option, redundant with --slow-ci.
Document PyMODINIT_FUNC macro.
Remove links to PyAPI_FUNC() and PyAPI_DATA() macros since they are
not documented. These macros should only be used to define the Python
C API. They should not be used outside Python code base.
Fix a race condition in "make regen-all". The deepfreeze.c source and
files generated by Argument Clinic are now generated or updated
before generating "global objects". Previously, some identifiers may
miss depending on the order in which these files were generated.
* "make regen-global-objects": Make sure that deepfreeze.c is
generated and up to date, and always run "make clinic".
* "make clinic" no longer runs generate_global_objects.py script.
* "make regen-deepfreeze" now only updates deepfreeze.c (C file).
It doesn't build deepfreeze.o (object) anymore.
* Remove misleading messages in "make regen-global-objects" and
"make clinic". They are now outdated, these commands are now
safe to use.
* Document generates files in Doc/using/configure.rst.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Statistics gathering is now off by default. Use the "-X pystats"
command line option or set the new PYTHONSTATS environment variable
to 1 to turn statistics gathering on at Python startup.
Statistics are no longer dumped at exit if statistics gathering was
off or statistics have been cleared.
Changes:
* Add PYTHONSTATS environment variable.
* sys._stats_dump() now returns False if statistics are not dumped
because they are all equal to zero.
* Add PyConfig._pystats member.
* Add tests on sys functions and on setting PyConfig._pystats to 1.
* Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h
header files.
* Rename '_py_stats' variable to '_Py_stats'.
* Exclude Include/cpython/pystats.h from the Py_LIMITED_API.
* Move pystats.h include from object.h to Python.h.
* Add _Py_StatsOn() and _Py_StatsOff() functions. Remove
'_py_stats_struct' variable from the API: make it static in
specialize.c.
* Document API in Include/pystats.h and Include/cpython/pystats.h.
* Complete pystats documentation in Doc/using/configure.rst.
* Don't write "all zeros" stats: if _stats_off() and _stats_clear()
or _stats_dump() were called.
* _PyEval_Fini() now always call _Py_PrintSpecializationStats() which
does nothing if stats are all zeros.
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
Python built with "configure --with-trace-refs" (tracing references)
is now ABI compatible with Python release build and debug build.
Moreover, it now also supports the Limited API.
Change Py_TRACE_REFS build:
* Remove _PyObject_EXTRA_INIT macro.
* The PyObject structure no longer has two extra members (_ob_prev
and _ob_next).
* Use a hash table (_Py_hashtable_t) to trace references (all
objects): PyInterpreterState.object_state.refchain.
* Py_TRACE_REFS build is now ABI compatible with release build and
debug build.
* Limited C API extensions can now be built with Py_TRACE_REFS:
xxlimited, xxlimited_35, _testclinic_limited.
* No longer rename PyModule_Create2() and PyModule_FromDefAndSpec2()
functions to PyModule_Create2TraceRefs() and
PyModule_FromDefAndSpec2TraceRefs().
* _Py_PrintReferenceAddresses() is now called before
finalize_interp_delete() which deletes the refchain hash table.
* test_tracemalloc find_trace() now also filters by size to ignore
the memory allocated by _PyRefchain_Trace().
Test changes for Py_TRACE_REFS:
* Add test.support.Py_TRACE_REFS constant.
* Add test_sys.test_getobjects() to test sys.getobjects() function.
* test_exceptions skips test_recursion_normalizing_with_no_memory()
and test_memory_error_in_PyErr_PrintEx() if Python is built with
Py_TRACE_REFS.
* test_repl skips test_no_memory().
* test_capi skisp test_set_nomemory().
Apply BOLT optimizations to libpython for shared builds. Most of the C
code is in libpython so it is critical to apply BOLT there fully realize
BOLT benefits.
This change also reworks how BOLT instrumentation is applied. It
effectively removes the readelf based logic added in gh-101525 and
replaces it with a mechanism that saves a copy of the pre-bolt binary
and restores that copy when necessary. This allows us to perform BOLT
optimizations without having to manually delete the output binary to
force a new bolt run.
Also:
- add a clean-bolt target for purging BOLT files and hook that up to the
clean target
- .gitignore BOLT related files
Before and after this refactor, `make` will no-op after a previous run.
Both versions should also share common make DAG deficiencies where
targets fail to trigger as often as they need to or can trigger
prematurely in certain scenarios. e.g. after this change you may need to
`rm profile-bolt-stamp` to force a BOLT run because there aren't
appropriate non-phony targets for BOLT's make target to depend on.
To make it easier to iterate on custom BOLT settings, the flags to pass
to instrumentation and application are now defined in configure and can
be overridden by passing BOLT_INSTRUMENT_FLAGS and BOLT_APPLY_FLAGS.
It has had no effect on non-macOS platforms for a long time, and has had
the non-obvious effect of invoking `pkg_config` and not setting
`-DUSING_APPLE_OS_LIBFFI` on macOS since GH-22855.
Add COMPILEALL_OPTS variable in Makefile to override compileall
options (default: -j0) in "make install". Also merge the compileall
commands into a single command building PYC files for the all
optimization levels (0, 1, 2) at once.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
This adds support for comparing pystats collected from two different builds.
- The `--json-output` can be used to load in a set of raw stats and output a
JSON file.
- Two of these JSON files can be provided on the next run, and then comparative
results between the two are output.
