cpython/Lib
T. Wouters 388e1ca9f0
gh-115999: Make list and tuple iteration more thread-safe. (#128637)
Make tuple iteration more thread-safe, and actually test concurrent iteration of tuple, range and list. (This is prep work for enabling specialization of FOR_ITER in free-threaded builds.) The basic premise is:

Iterating over a shared iterable (list, tuple or range) should be safe, not involve data races, and behave like iteration normally does.

Using a shared iterator should not crash or involve data races, and should only produce items regular iteration would produce. It is not guaranteed to produce all items, or produce each item only once. (This is not the case for range iteration even after this PR.)

Providing stronger guarantees is possible for some of these iterators, but it's not always straight-forward and can significantly hamper the common case. Since iterators in general aren't shared between threads, and it's simply impossible to concurrently use many iterators (like generators), better to make sharing iterators without explicit synchronization clearly wrong.

Specific issues fixed in order to make the tests pass:

 - List iteration could occasionally fail an assertion when a shared list was shrunk and an item past the new end was retrieved concurrently. There's still some unsafety when deleting/inserting multiple items through for example slice assignment, which uses memmove/memcpy.

 - Tuple iteration could occasionally crash when the iterator's reference to the tuple was cleared on exhaustion. Like with list iteration, in free-threaded builds we can't safely and efficiently clear the iterator's reference to the iterable (doing it safely would mean extra, slow refcount operations), so just keep the iterable reference around.
2025-02-18 16:52:46 -08:00
..
__phello__
_pyrepl gh-127349: Add check for correct resizing in REPL (#127387) 2025-01-30 19:34:09 +00:00
asyncio gh-130145: fix loop.run_forever when loop is already running (#130146) 2025-02-15 15:01:53 +05:30
collections Speed-up lazy heapq import in collections (gh-127538) 2024-12-02 20:45:36 -06:00
concurrent gh-88110: Clear concurrent.futures.thread._threads_queues after fork to avoid joining parent process' threads (GH-126098) 2024-11-22 18:20:34 +02:00
ctypes gh-119349: Add ctypes.util.dllist -- list loaded shared libraries (GH-122946) 2025-02-08 14:02:36 +01:00
curses
dbm
email gh-80222: Fix email address header folding with long quoted-string (#122753) 2025-01-18 19:50:52 -05:00
encodings gh-123803: Support arbitrary code page encodings on Windows (GH-123804) 2024-11-18 17:45:25 +00:00
ensurepip gh-129583: update bundled pip to 25.0.1 (#129909) 2025-02-10 12:21:04 +02:00
html
http gh-129408: http: Fix typo 'RFF' to RFC' in documentation (#129411) 2025-01-28 13:25:51 -05:00
idlelib gh-68400: Remove outdated 2.7 comment (#129919) 2025-02-09 23:53:32 -05:00
importlib gh-130094: Fix race conditions in importlib (gh-130101) 2025-02-18 18:02:42 -05:00
json gh-125660: Reject invalid unicode escapes for Python implementation of JSON decoder (GH-125683) 2024-10-18 15:29:47 +03:00
logging gh-127712: Fix secure argument of logging.handlers.SMTPHandler (GH-127726) 2025-02-10 12:34:27 +00:00
multiprocessing gh-129205: Update multiprocessing.forkserver to use os.readinto() (#129425) 2025-01-30 22:24:52 +00:00
pathlib GH-127381: pathlib ABCs: remove ReadablePath.rglob() (#130207) 2025-02-17 19:15:59 +00:00
pydoc_data Remove trailing whitespace in Lib/pydoc_data/topics.py (#130014) 2025-02-11 23:28:28 +02:00
re gh-126505: Fix bugs in compiling case-insensitive character classes (GH-126557) 2024-11-11 18:27:26 +02:00
site-packages
sqlite3 gh-105875: amend sqlite3 docstring wrt. SQLite requirement (#129599) 2025-02-02 23:03:54 +00:00
sysconfig GH-107956: install build-details.json (PEP 739) (#130069) 2025-02-13 13:58:00 +00:00
test gh-115999: Make list and tuple iteration more thread-safe. (#128637) 2025-02-18 16:52:46 -08:00
tkinter gh-128562: Fix generation of the tkinter widget names (GH-128604) 2025-01-13 15:54:46 +00:00
tomllib gh-118761: Improve import time of tomllib (#128907) 2025-01-16 18:48:42 -08:00
turtledemo gh-128062: Fix the font size and shortcut display of the turtledemo menu (#128063) 2024-12-19 15:24:47 -05:00
unittest gh-128595: Default to stdout isatty for colour detection instead of stderr (#128498) 2025-01-20 12:52:42 +02:00
urllib gh-105704: Disallow square brackets ([ and ]) in domain names for parsed URLs (#129418) 2025-01-31 09:41:34 -08:00
venv GH-127724: don't use sysconfig to calculate the venv local include path (#127731) 2024-12-12 21:41:46 +00:00
wsgiref gh-126133: Only use start year in PSF copyright, remove end years (#126236) 2024-11-12 15:59:19 +02:00
xml gh-128302: Fix bugs in xml.dom.xmlbuilder (GH-128284) 2025-01-07 12:40:41 +02:00
xmlrpc
zipfile gh-118761: Improve import time for pstats and zipfile (#128981) 2025-01-23 14:49:36 +00:00
zoneinfo
__future__.py
__hello__.py
_aix_support.py
_android_support.py Fix typo in Lib/_android_support.py (#127699) 2024-12-06 19:50:58 +02:00
_apple_support.py gh-126925: Modify how iOS test results are gathered (#127592) 2024-12-09 13:28:57 +08:00
_collections_abc.py gh-116938: Clarify documentation of dict and dict.update regarding the positional argument they accept (#125213) 2024-10-11 23:05:13 +00:00
_colorize.py gh-129061: Fix FORCE_COLOR and NO_COLOR when empty strings (#129140) 2025-01-27 16:24:10 +02:00
_compat_pickle.py
_compression.py
_ios_support.py
_markupbase.py _markupbase.py: Use a permalink for the analysis of MS-Word extensions (GH-129017) 2025-02-06 11:40:43 +01:00
_opcode_metadata.py gh-100239: replace BINARY_SUBSCR & family by BINARY_OP with oparg NB_SUBSCR (#129700) 2025-02-07 22:39:54 +00:00
_osx_support.py
_py_abc.py
_pydatetime.py gh-127260: Improve error consistency in both fromisoformat implementations (#130134) 2025-02-18 15:49:28 +00:00
_pydecimal.py Fix typos in Lib/_pydecimal.py (#127700) 2024-12-12 22:27:29 +02:00
_pyio.py gh-129005: Align FileIO.readall between _pyio and _io (#129705) 2025-02-07 12:06:11 +01:00
_pylong.py Fix typos in comments and test code (#122846) 2024-08-11 21:16:41 -07:00
_sitebuiltins.py
_strptime.py gh-127552: Remove comment questioning 4-digit restriction for ‘Y’ in datetime.strptime patterns (#127590) 2024-12-04 10:30:51 -08:00
_threading_local.py
_weakrefset.py gh-89967: make WeakKeyDictionary and WeakValueDictionary thread safe (#125325) 2024-10-13 21:05:05 +05:30
abc.py
annotationlib.py gh-119180: Add VALUE_WITH_FAKE_GLOBALS format to annotationlib (#124415) 2024-11-26 15:40:13 +00:00
antigravity.py
argparse.py GH-127133: Remove ability to nest argument groups & mutually exclusive groups (#127186) 2024-11-24 15:20:37 +00:00
ast.py gh-127975: Avoid reusing quote types in ast.unparse if not needed (#127980) 2025-01-31 08:49:06 +00:00
base64.py gh-118761: substitute re import in base64.b16decode for a more efficient alternative (#128736) 2025-01-14 13:25:33 +00:00
bdb.py gh-128991: Release the enter frame reference within bdb callback (#128992) 2025-01-18 16:53:06 -05:00
bisect.py
bz2.py
calendar.py gh-128317: Move CLI calendar highlighting to private class (#129625) 2025-02-08 15:56:57 +00:00
cmd.py gh-118761: Improve import time of cmd module (#130056) 2025-02-17 20:06:08 +00:00
code.py gh-125666: Avoid PyREPL exiting when a null byte is in input (#125732) 2024-10-27 01:23:53 +00:00
codecs.py
codeop.py gh-124960: Fixed barry_as_FLUFL future flag does not work in new REPL (#124999) 2024-10-14 15:53:50 +02:00
colorsys.py
compileall.py
configparser.py gh-129678: ConfigParser: do not write an empty unnamed section (GH-129679) 2025-02-17 14:24:57 +01:00
contextlib.py
contextvars.py gh-126451: Register contextvars.Context to collections.abc.Mapping (#126452) 2024-11-07 01:12:45 +03:00
copy.py gh-128118: Speed up copy.copy with fast lookup for atomic and container types (#128119) 2024-12-30 18:18:42 +01:00
copyreg.py gh-125767: Fix pickling and copying of super objects (GH-125781) 2024-10-21 21:30:45 +03:00
cProfile.py
csv.py gh-118761: Improve import time for csv (#128858) 2025-01-18 11:45:18 +01:00
dataclasses.py gh-118974: Add decorator argument to make_dataclass (gh-122723) 2024-10-01 09:51:51 -04:00
datetime.py gh-123843: Remove broken links to the Zope DateTimeWiki (#123846) 2024-09-08 22:39:23 -04:00
decimal.py gh-123339: Fix cases of inconsistency of __module__ and __firstlineno__ in classes (GH-123613) 2024-09-28 20:51:49 +03:00
difflib.py gh-129939: Add darkmode support for difflib's comparison pages (#129940) 2025-02-11 10:58:57 +02:00
dis.py Revert "GH-128914: Remove conditional stack effects from bytecodes.c and the code generators (GH-128918)" (GH-129202) 2025-01-23 09:26:25 +00:00
doctest.py gh-128595: Default to stdout isatty for colour detection instead of stderr (#128498) 2025-01-20 12:52:42 +02:00
enum.py gh-112328: Make EnumDict usable on its own and document it (GH-123669) 2024-12-20 11:40:58 -08:00
filecmp.py gh-122400: Handle ValueError in filecmp (GH-122401) 2024-07-30 08:50:30 +00:00
fileinput.py
fnmatch.py gh-122288: Improve performances of fnmatch.translate (#122289) 2024-11-27 16:42:45 +00:00
fractions.py gh-121797: Add class method Fraction.from_number() (GH-121800) 2024-10-14 07:54:59 +00:00
ftplib.py
functools.py gh-127750: Fix and optimize functools.singledispatchmethod() (GH-130008) 2025-02-17 11:11:20 +02:00
genericpath.py
getopt.py gh-126946: Improve error message in getopt.do_longs based on existing comment (GH-126871) 2024-11-26 10:54:02 +02:00
getpass.py
gettext.py gh-118761: Improve the import time of `gettext` (#128898) 2025-01-20 00:01:20 +00:00
glob.py GH-129835: Yield path with trailing slash from ReadablePath.glob('') (#129836) 2025-02-08 06:47:09 +00:00
graphlib.py gh-129892: Doc: Remove unnecessary role directive in graphlib.py (#129896) 2025-02-09 18:22:57 +03:00
gzip.py gh-125260: Change the default `gzip.compress()` mtime to 0 (#125261) 2024-10-12 18:18:48 +01:00
hashlib.py Fix typos in warnings, docstrings, comments and text files (#123597) 2024-09-03 02:20:40 +02:00
heapq.py
hmac.py
imaplib.py gh-55454: Add IMAP4 IDLE support to imaplib (#122542) 2025-02-06 19:15:11 -08:00
inspect.py gh-130164: Fix inspect.Signature.bind() handling of positional-only args without defaults (GH-130192) 2025-02-18 17:19:04 +02:00
io.py
ipaddress.py gh-124217, ipaddress: Add RFC 9637 reserved IPv6 block 3fff::/20 (#124240) 2024-09-20 11:01:28 +00:00
keyword.py
linecache.py gh-126775: make linecache.checkcache threadsafe and GC re-entrency safe (#126776) 2024-12-10 08:40:54 +01:00
locale.py gh-129646: Update the locale alias mapping (#129647) 2025-02-04 16:22:05 +01:00
lzma.py
mailbox.py
mimetypes.py gh-128703: Fix mimetypes.guess_type for empty Content-Type in registry (GH-128854) 2025-02-14 17:26:26 +00:00
modulefinder.py
netrc.py
ntpath.py gh-119826: Improved fallback for ntpath.abspath() on Windows (GH-119938) 2024-11-12 21:18:03 +00:00
nturl2path.py GH-127236: pathname2url(): generate RFC 1738 URL for absolute POSIX path (#127194) 2024-11-25 19:59:20 +00:00
numbers.py
opcode.py Revert "GH-128914: Remove conditional stack effects from bytecodes.c and the code generators (GH-128918)" (GH-129202) 2025-01-23 09:26:25 +00:00
operator.py gh-115808: Add `is_none and is_not_none to operator` (#115814) 2024-08-10 20:16:34 +01:00
optparse.py gh-118761: Redudce the import time of `optparse` (#128899) 2025-01-20 00:03:19 +00:00
os.py gh-120057: Add os.reload_environ() function (#126268) 2024-11-05 08:43:34 +01:00
pdb.py gh-126944: Show explicit errors when required arguments of pdb commands are missing (#130240) 2025-02-18 10:58:15 -05:00
pickle.py gh-118761: Improve import time of the pickle module. (#128732) 2025-01-14 12:26:26 +01:00
pickletools.py gh-126997: Fix support of non-ASCII strings in pickletools (GH-127062) 2024-11-21 13:15:12 +02:00
pkgutil.py gh-97850: remove `find_loader and get_loader from pkgutil` (#119656) 2024-11-01 18:35:22 +02:00
platform.py gh-127732: Add Windows Server 2025 detection to platform module (GH-127733) 2024-12-09 12:24:26 +00:00
plistlib.py
poplib.py Give poplib.POP3.rpop a proper docstring (#127370) 2024-12-07 18:13:49 +02:00
posixpath.py GH-118289: Fix handling of non-directories in posixpath.realpath() (#120127) 2024-11-13 22:59:32 +00:00
pprint.py gh-118761: Improve import time of pprint (#122725) 2024-08-07 22:46:54 +03:00
profile.py
pstats.py gh-118761: Improve import time for pstats and zipfile (#128981) 2025-01-23 14:49:36 +00:00
pty.py gh-124405: Fix NameError in openpty (#124406) 2024-09-24 20:06:39 +02:00
py_compile.py
pyclbr.py
pydoc.py gh-128772: Fix pydoc for methods with __module__ is None (GH-129177) 2025-02-04 16:25:49 +02:00
queue.py
quopri.py
random.py gh-123968: Fix lower bound for python -m random --float (#123971) 2024-09-12 16:54:18 +03:00
reprlib.py gh-113570: reprlib.repr does not use builtin __repr__ for reshadowed builtins (GH-113577) 2024-10-17 16:34:37 +00:00
rlcompleter.py
runpy.py
sched.py
secrets.py
selectors.py
shelve.py
shlex.py
shutil.py gh-81340: Use copy_file_range in shutil.copyfile copy functions (GH-93152) 2025-02-03 10:23:27 +00:00
signal.py
site.py gh-128779: Fix site venv() for system site-packages (#129184) 2025-01-30 00:02:31 +00:00
smtplib.py
socket.py gh-128916: Do not set SO_REUSEPORT on non-AF_INET* sockets (GH-128933) 2025-01-17 16:49:16 -08:00
socketserver.py gh-128916: Do not set SO_REUSEPORT on non-AF_INET* sockets (GH-128933) 2025-01-17 16:49:16 -08:00
sre_compile.py
sre_constants.py
sre_parse.py
ssl.py gh-128035: Add ssl.HAS_PHA to detect libssl PHA support (GH-128036) 2024-12-24 18:29:27 +00:00
stat.py
statistics.py Minor code beautifications in statistics.py (gh-124866) 2024-10-01 15:55:36 -05:00
string.py gh-71494: string.Formatter: support keys/attributes in unnumbered fields (GH-21767) 2025-01-31 13:16:24 +01:00
stringprep.py
struct.py
subprocess.py gh-118761: Revert "Improve import time of subprocess (GH-129427)" (#130201) 2025-02-16 18:40:08 +00:00
symtable.py
tabnanny.py
tarfile.py gh-121267: Improve performance of tarfile (#121267) (#121269) 2024-10-30 15:08:30 -07:00
tempfile.py gh-126639: Add ResourceWarning to NamedTemporaryFile (#126677) 2024-12-18 10:12:24 +00:00
textwrap.py gh-126133: Only use start year in PSF copyright, remove end years (#126236) 2024-11-12 15:59:19 +02:00
this.py
threading.py gh-129403: Fix ValueError messages in asyncio.Barrier and threading.Barrier (#129419) 2025-01-30 11:11:12 +03:00
timeit.py
token.py gh-127303: Add docs for token.EXACT_TOKEN_TYPES (#127304) 2024-11-29 04:00:50 -05:00
tokenize.py gh-125553: Fix backslash continuation in untokenize (#126010) 2025-01-21 19:58:44 +00:00
trace.py gh-112192: Increase the trace module coverage precision to one decimal (#126972) 2024-12-03 23:33:13 +01:00
traceback.py gh-96092: Fix traceback.walk_stack(None) skipping too many frames (#129330) 2025-02-13 01:43:09 +00:00
tracemalloc.py
tty.py
turtle.py gh-126349: Add 'fill', 'poly', and 'no_animation' context managers to turtle (#126350) 2025-01-18 11:27:22 +01:00
types.py
typing.py gh-128661: Fix typing.evaluate_forward_ref not showing deprecation (#128663) 2025-01-09 18:15:13 +03:00
uuid.py gh-128427: Add uuid.NIL and uuid.MAX (#128429) 2025-01-27 18:17:17 +02:00
warnings.py gh-129842: warnings.py: Remove obsolete requirement reference (GH-129845) 2025-02-08 13:57:17 +01:00
wave.py
weakref.py gh-89967: make WeakKeyDictionary and WeakValueDictionary thread safe (#125325) 2024-10-13 21:05:05 +05:30
webbrowser.py gh-123494: Improve documentation for `webbrowser` return types (#123495) 2024-09-01 06:17:03 +01:00
zipapp.py
zipimport.py gh-125746: Delay deprecated zipimport.zipimporter.load_module removal time to 3.15 (#125748) 2025-02-11 23:59:09 +00:00