Commit graph

482 commits

Author SHA1 Message Date
Miss Islington (bot)
15c3d0013d
[3.13] gh-117398: Add datetime C-API type check test for subinterpreters (gh-120463)
Check if the DateTime C-API type matches the datetime.date type on main and shared/isolated subinterpreters.

(cherry picked from commit 50a389565a, AKA gh-119604)

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
2024-06-13 18:30:42 +00:00
Sam Gross
ae705319fc
[3.13] gh-117657: Fix race involving immortalizing objects (GH-119927) (#120005)
The free-threaded build currently immortalizes objects that use deferred
reference counting (see gh-117783). This typically happens once the
first non-main thread is created, but the behavior can be suppressed for
tests, in subinterpreters, or during a compile() call.

This fixes a race condition involving the tracking of whether the
behavior is suppressed.

(cherry picked from commit 47fb4327b5)
2024-06-03 22:21:32 +00:00
Sam Gross
723d4d2fe8
gh-118527: Intern code consts in free-threaded build (#118667)
We already intern and immortalize most string constants. In the
free-threaded build, other constants can be a source of reference count
contention because they are shared by all threads running the same code
objects.
2024-05-06 20:12:39 -04:00
Hugo van Kemenade
3b3f8dea57
gh-117225: Move colorize functionality to own internal module (#118283) 2024-05-01 12:27:06 -06:00
Russell Keith-Magee
21336aa127
gh-118201: Accomodate flaky behavior of os.sysconf on iOS (GH-118453) 2024-04-30 22:31:00 -04:00
Guido van Rossum
7d83f7bcc4
gh-118335: Configure Tier 2 interpreter at build time (#118339)
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.

We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.

On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.

In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
2024-04-30 18:26:34 -07:00
Sam Gross
7ccacb220d
gh-117783: Immortalize objects that use deferred reference counting (#118112)
Deferred reference counting is not fully implemented yet. As a temporary
measure, we immortalize objects that would use deferred reference
counting to avoid multi-threaded scaling bottlenecks.

This is only performed in the free-threaded build once the first
non-main thread is started. Additionally, some tests, including refleak
tests, suppress this behavior.
2024-04-29 14:36:02 -04:00
Kirill Podoprigora
51c70de998
gh-118351: Adapt support.TEST_MODULES_ENABLED for builds without the config variable (GH-118354) 2024-04-29 16:50:11 +01:00
Pablo Galindo Salgado
345e1e04ec
gh-112730: Make the test suite resilient to color-activation environment variables (#117672) 2024-04-24 21:25:22 +01:00
Eric Snow
03e3e31723
gh-76785: Rename _xxsubinterpreters to _interpreters (gh-117791)
See https://discuss.python.org/t/pep-734-multiple-interpreters-in-the-stdlib/41147/26.
2024-04-24 16:18:24 +00:00
Hugo van Kemenade
975081b11e
gh-117225: Add color to doctest output (#117583)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-04-24 14:27:40 +03:00
Irit Katriel
c179c0e6cb
gh-117680: make _PyInstructionSequence a PyObject and use it in tests (#117629) 2024-04-17 16:42:04 +01:00
Eric Snow
fd259fdabe
gh-76785: Handle Legacy Interpreters Properly (gh-117490)
This is similar to the situation with threading._DummyThread.  The methods (incl. __del__()) of interpreters.Interpreter objects must be careful with interpreters not created by interpreters.create().  The simplest thing to start with is to disable any method that modifies or runs in the interpreter.  As part of this, the runtime keeps track of where an interpreter was created.  We also handle interpreter "refcounts" properly.
2024-04-11 23:23:25 +00:00
Sam Gross
25f6ff5d3e
gh-117649: Raise ImportError for unsupported modules in free-threaded build (#117651)
The free-threaded build does not currently support the combination of
single-phase init modules and non-isolated subinterpreters. Ensure that
`check_multi_interp_extensions` is always `True` for subinterpreters in
the free-threaded build so that importing these modules raises an
`ImportError`.
2024-04-11 15:00:54 -04:00
Eric Snow
993c3cca16
gh-76785: Add More Tests to test_interpreters.test_api (gh-117662)
In addition to the increase test coverage, this is a precursor to sorting out how we handle interpreters created directly via the C-API.
2024-04-10 18:37:01 -06:00
Erlend E. Aasland
ca62ffd1a5
gh-116303: Skip tests if C recursion limit is unavailable (GH-117368)
The test suite fetches the C recursion limit from the _testcapi
extension module. Test extension modules can be disabled using the
--disable-test-modules configure option.
2024-04-08 14:45:25 +02:00
Sam Gross
42205143f8
gh-117478: Add @support.requires_gil_enabled decorator (#117479)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2024-04-04 14:10:46 -04:00
Eric Snow
976bcb2379
gh-76785: Raise InterpreterError, Not RuntimeError (gh-117489)
I had meant to switch everything to InterpreterError when I added it a while back.  At the time I missed a few key spots.

As part of this, I've added print-the-exception to _PyXI_InitTypes() and fixed an error case in `_PyStaticType_InitBuiltin().
2024-04-03 10:58:39 -06:00
Erlend E. Aasland
ea94b3b149
gh-116303: Skip test module dependent tests if test modules are unavailable (#117341) 2024-04-03 15:11:36 +02:00
Eric Snow
857d3151c9
gh-76785: Consolidate Some Interpreter-related Testing Helpers (gh-117485)
This eliminates the duplication of functionally identical helpers in the _testinternalcapi and _xxsubinterpreters modules.
2024-04-02 23:16:50 +00:00
Eric Snow
f341d6017d
gh-76785: Add PyInterpreterConfig Helpers (gh-117170)
These helpers make it easier to customize and inspect the config used to initialize interpreters.  This is especially valuable in our tests.  I found inspiration from the PyConfig API for the PyInterpreterConfig dict conversion stuff.  As part of this PR I've also added a bunch of tests.
2024-04-02 20:35:52 +00:00
Erlend E. Aasland
6c8ac8a32f
gh-116303: Handle disabled test modules in test.support helpers (#116482)
Make sure test.support helpers skip iso. failing if test extension
modules are disabled. Also log TEST_MODULES in test.pythoninfo.
2024-03-28 08:40:37 +00:00
Malcolm Smith
74c8568d07
gh-71042: Add platform.android_ver (#116674) 2024-03-27 17:53:27 +01:00
Victor Stinner
a114d08a89
gh-92906: Enable test_cext and test_cppext on Windows (#117000)
On Windows in release mode, the test_cext and test_cppext can now
build C and C++ extensions.

* test_cext now also builds the C extension without options.
* test_cppext now also builds the C++ extension without options.
* Add C++14 test to test_cppext; C++11 is not supported by MSVC.
* Make setup_venv_with_pip_setuptools_wheel() quiet when
  support.verbose is false. Only show stdout and stderr on failure.
2024-03-19 14:03:27 +00:00
Antoine Pitrou
86bc40dd41
gh-112536: Add test_threading to TSAN tests (#116898) 2024-03-16 11:55:46 +00:00
Sam Gross
8e2aab7ad5
gh-116604: Fix test_gc on free-threaded build (#116662)
The free-threaded GC only does full collections, so it uses a threshold that
is a maximum of a fixed value (default 2000) and proportional to the number of
live objects. If there were many live objects after the previous collection,
then the threshold may be larger than 10,000 causing
`test_indirect_calls_with_gc_disabled` to fail.

This manually sets the threshold to `(1000, 0, 0)` for the test. The `0`
disables the proportional scaling.
2024-03-13 13:27:36 +00:00
Jason R. Coombs
a254807761
gh-116307: Proper fix for 'mod' leaking across importlib tests (#116680)
gh-116307: Create a new import helper 'isolated modules' and use that instead of 'Clean Import' to ensure that tests from importlib_resources don't leave modules in sys.modules.
2024-03-12 21:36:21 +00:00
Malcolm Smith
872c0714fc
gh-71052: Change Android's sys.platform from "linux" to "android"
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-11 19:25:39 +00:00
Victor Stinner
d9bcdda39c
gh-116417: Add _testlimitedcapi C extension (#116419)
Add a new C extension "_testlimitedcapi" which is only built with the
limited C API.

Move heaptype_relative.c and vectorcall_limited.c from
Modules/_testcapi/ to Modules/_testlimitedcapi/.

* configure: add _testlimitedcapi test extension.
* Update generate_stdlib_module_names.py.
* Update make check-c-globals.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-07 18:31:12 +00:00
Eric Snow
4402b3cbcf
gh-76785: Minor Improvements to "interpreters" Module (gh-116328)
This includes adding pickle support to various classes, and small changes to improve the maintainability of the low-level _xxinterpqueues module.
2024-03-05 08:54:46 -07:00
Eric Snow
936d4611d6
gh-76785: Fix interpreters.Queue.get_nowait() (gh-116166)
I missed this change in gh-115566.
2024-03-01 16:36:35 +00:00
Malcolm Smith
41d5391c55
gh-71052: Add test exclusions to support running the test suite on Android (#115918) 2024-02-29 22:32:50 +01:00
Eric Snow
e80abd57a8
gh-76785: Update test.support.interpreters to Align With PEP 734 (gh-115566)
This brings the code under test.support.interpreters, and the corresponding extension modules, in line with recent updates to PEP 734.

(Note: PEP 734 has not been accepted at this time.  However, we are using an internal copy of the implementation in the test suite to exercise the existing subinterpreters feature.)
2024-02-28 16:08:08 -07:00
Petr Viktorin
7acf1fb5a7
gh-114911: Add CPUStopwatch test helper (GH-114912)
A few of our tests measure the time of CPU-bound operation, mainly
to avoid quadratic or worse behaviour.
Add a helper to ignore GC and time spent in other processes.
2024-02-28 12:53:48 +01:00
Donghee Na
a2d4281415
gh-112087: Make __sizeof__ and listiter_{len, next} to be threadsafe (gh-114843) 2024-02-15 02:00:50 +09:00
Russell Keith-Magee
391659b3da
gh-114099: Add test exclusions to support running the test suite on iOS (#114889)
Add test annotations required to run the test suite on iOS (PEP 730).

The majority of the change involve annotating tests that use subprocess,
but are skipped on Emscripten/WASI for other reasons, and including
iOS/tvOS/watchOS under the same umbrella as macOS/darwin checks.

`is_apple` and `is_apple_mobile` test helpers have been added to
identify *any* Apple platform, and "any Apple platform except macOS",
respectively.
2024-02-05 01:04:57 +01:00
Nikita Sobolev
ba253a4794
gh-108303: Move .whl test files to Lib/test/wheeldata/ (#114343) 2024-01-23 16:33:12 +01:00
Nikita Sobolev
8f5e7d739f
gh-108303: Move smtpd to test.support (#114368)
Update test_logging.py and test_smtplib.py.
2024-01-22 14:36:29 +01:00
Ronald Oussoren
5adff3f86f
gh-80931: Skip some socket tests while hunting for refleaks on macOS (#114057)
Some socket tests related to sending file descriptors cause a file descriptor leak on macOS, all of them tests that send one or more descriptors than cannot be received on the read end.  This appears to be a platform bug.

This PR skips those tests when doing a refleak test run to avoid hiding other problems.
2024-01-21 11:25:15 +01:00
Nikita Sobolev
9c93350f58
gh-108303: Move all doctest related files and tests to Lib/test/test_doctest/ (#112109)
Co-authored-by: Brett Cannon <brett@python.org>
2024-01-18 16:58:11 +01:00
Mark Shannon
17b73ab99e
GH-113655: Lower the C recursion limit on various platforms (GH-113944) 2024-01-16 09:32:01 +00:00
Samet YASLAN
88cb972000
gh-112536: Add support for thread sanitizer (TSAN) (gh-112648) 2023-12-30 17:17:02 +09:00
Mark Shannon
45e09f921b
GH-112215: Increase C recursion limit for non debug builds (GH-113397) 2023-12-22 14:25:25 +00:00
Serhiy Storchaka
c6e953be12
gh-113090: Fix test.support.os_support.can_chmod() on Windows (GH-113091) 2023-12-14 13:27:43 +02:00
Irit Katriel
428c9812cb
gh-112962: in dis module, put cache information in the Instruction instead of creating fake Instructions to represent it (#113016) 2023-12-13 12:00:21 +00:00
Eric Snow
c6e614fd81
gh-76785: Avoid Pickled TracebackException for Propagated Subinterpreter Exceptions (gh-113036)
We need the TracebackException of uncaught exceptions for a single purpose: the error display.  Thus we only need to pass the formatted error display between interpreters.  Passing a pickled TracebackException is overkill.
2023-12-13 00:31:30 +00:00
Eric Snow
8a4c1f3ff1
gh-76785: Show the Traceback for Uncaught Subinterpreter Exceptions (gh-113034)
When an exception is uncaught in Interpreter.exec_sync(), it helps to show that exception's error display if uncaught in the calling interpreter.  We do so here by generating a TracebackException in the subinterpreter and passing it between interpreters using pickle.
2023-12-13 00:00:54 +00:00
Eric Snow
9898e61041
gh-76785: Add Interpreter.prepare_main() (gh-113021)
This is one of the last pieces to get test.support.interpreters in sync with PEP 734.
2023-12-12 18:06:06 +00:00
Eric Snow
a49b427b02
gh-76785: More Fixes for test.support.interpreters (gh-113012)
This brings the module (along with the associated extension modules) mostly in sync with PEP 734.  There are only a few small things to wrap up.
2023-12-12 17:43:30 +00:00
Eric Snow
86a77f4e1a
gh-76785: Fixes for test.support.interpreters (gh-112982)
This involves a number of changes for PEP 734.
2023-12-12 08:24:31 -07:00