Commit graph

6981 commits

Author SHA1 Message Date
Dennis Sweeney
08069bac3b
[3.10] GH-93964: Harden overflow checks before _PyBytes_Resize in compile.c (GH-94045) 2022-06-22 09:37:56 +01:00
Miss Islington (bot)
968b238b5e
[3.11] gh-93741: Add private C API _PyImport_GetModuleAttrString() (GH-93742) (GH-93792)
It combines PyImport_ImportModule() and PyObject_GetAttrString()
and saves 4-6 lines of code on every use.

Add also _PyImport_GetModuleAttr() which takes Python strings as arguments.
(cherry picked from commit 6fd4c8ec77)
(cherry picked from commit d42b3689f4)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2022-06-16 00:19:29 -07:00
Éric
c649526f92
[3.10] gh-93217: fix some issues in man page and --help (GH-93219) (#93261) 2022-05-29 14:04:23 -04:00
Victor Stinner
9369942054
[3.10] gh-91924: Fix __ltrace__ for non-UTF-8 stdout encoding (#93214)
Fix __ltrace__ debug feature if the stdout encoding is not UTF-8.

If the stdout encoding is not UTF-8, the first call to
lltrace_resume_frame() indirectly sets lltrace to 0 when calling
unicode_check_encoding_errors() which calls
encodings.search_function().

Add test_lltrace.test_lltrace() test.
2022-05-26 00:16:32 +02:00
Miss Islington (bot)
a4bea26ee4
gh-93065: Fix HAMT to iterate correctly over 7-level deep trees (GH-93066) (GH-93146)
Also while there, clarify a few things about why we reduce the hash to 32 bits.

Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>

(cherry picked from commit c1f5c903a7)
2022-05-24 10:52:29 +02:00
Dennis Sweeney
c1b12495f6
gh-93061: Mark as artificial: backwards jump after async for (GH-93120) 2022-05-23 19:53:38 -04:00
Miss Islington (bot)
89697f7374
bpo-47260: Fix os.closerange() potentially being a no-op in a seccomp sandbox (GH-32418)
_Py_closerange() currently assumes that close_range() closes
all file descriptors even if it returns an error (other than ENOSYS).
This assumption can be wrong on Linux if a seccomp sandbox denies
the underlying syscall, pretending that it returns EPERM or EACCES.
In this case _Py_closerange() won't close any descriptors at all,
which in the worst case can be a security issue.

Fix this by falling back to other methods in case of any close_range()
error. Note that fallbacks will not be triggered on any problems with
closing individual file descriptors because close_range() is documented
to ignore such errors on both Linux[1] and FreeBSD[2].

[1] https://man7.org/linux/man-pages/man2/close_range.2.html
[2] https://www.freebsd.org/cgi/man.cgi?query=close_range&sektion=2
(cherry picked from commit 1c8b3b5d66)

Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
2022-04-08 11:10:38 -07:00
Pablo Galindo Salgado
3594ebca2c
[3.10] bpo-46940: Don't override existing AttributeError suggestion information (GH-31710) (GH-31724)
When an exception is created in a nested call to PyObject_GetAttr, any
external calls will override the context information of the
AttributeError that we have already placed in the most internal call.
This will cause the suggestions we create to nor work properly as the
attribute name and object that we will be using are the incorrect ones.

To avoid this, we need to check first if these attributes are already
set and bail out if that's the case..
(cherry picked from commit 3b3be05a16)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2022-03-07 13:18:36 +00:00
Miss Islington (bot)
fa8c5ed9c8
bpo-46831: Update __build_class__ comment (GH-31522)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
(cherry picked from commit 81d968b7c3)

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2022-03-02 22:02:59 -08:00
Mark Shannon
d4e4ef107a
[3.10] bpo-46724: Use JUMP_ABSOLUTE for all backward jumps. (GH-31326) (GH-31354) 2022-02-16 11:26:02 +00:00
Miss Islington (bot)
ff6948b128
bpo-45773: Remove invalid peephole optimizations (GH-31066)
(cherry picked from commit e0433c1e70)

Co-authored-by: Brandt Bucher <brandt@python.org>
2022-02-03 07:54:51 -08:00
Petr Viktorin
5c39e474db
[3.10] bpo-45703: Invalidate _NamespacePath cache on importlib.invalidate_cache (GH-29384) (GH-30922)
Consider the following directory structure:

    .
    └── PATH1
        └── namespace
            └── sub1
                └── __init__.py

And both PATH1 and PATH2 in sys path:

    $ PYTHONPATH=PATH1:PATH2 python3.11
    >>> import namespace
    >>> import namespace.sub1
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> ...

While this interpreter still runs, PATH2/namespace/sub2 is created:

    .
    ├── PATH1
    │   └── namespace
    │       └── sub1
    │           └── __init__.py
    └── PATH2
        └── namespace
            └── sub2
                └── __init__.py

The newly created module cannot be imported:

    >>> ...
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> import namespace.sub2
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ModuleNotFoundError: No module named 'namespace.sub2'

Calling importlib.invalidate_caches() now newly allows to import it:

    >>> import importlib
    >>> importlib.invalidate_caches()
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> import namespace.sub2
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace', '.../PATH2/namespace'])

This was not previously possible.
2022-01-27 06:00:23 -08:00
Yury Selivanov
6f9ca53a6a
bpo-46347: Fix PyEval_EvalCodeEx to correctly cleanup in error paths (#30553) 2022-01-11 16:17:42 -08:00
Miss Islington (bot)
b1a94f1fab
bpo-46347: Fix memory leak in PyEval_EvalCodeEx. (GH-30546)
First introduced in 0332e569c1
(cherry picked from commit 607d8a838f)

Co-authored-by: Yury Selivanov <yury@edgedb.com>
2022-01-11 15:09:22 -08:00
Miss Islington (bot)
bea3f42bb7
bpo-46289: Make conversion of FormattedValue not optional on ASDL (GH-30467)
Automerge-Triggered-By: GH:isidentical
(cherry picked from commit d382f7ee0b)

Co-authored-by: Batuhan Taskaya <batuhan@python.org>
2022-01-07 14:30:18 -08:00
Miss Islington (bot)
35955e4ade
[3.10] Update copyright year to 2022. (GH-30335) (GH-30336)
Automerge-Triggered-By: GH:benjaminp
(cherry picked from commit ba00f0d93a)


Co-authored-by: Benjamin Peterson <benjamin@python.org>
2022-01-02 13:13:04 -08:00
Miss Islington (bot)
438817fdd5
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059) (GH-30064)
(cherry picked from commit 59435eea08)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-12-12 01:24:12 +00:00
Mark Shannon
99c72326d2
[3.10] bpo-46009: Do not exhaust generator when send() method raises (GH-29986). (GH-29988)
* [3.10] bpo-46009: Do not exhaust generator when send() method raises (GH-29986).
(cherry picked from commit 69806b9516)

Co-authored-by: Mark Shannon <mark@hotpy.org>

* Rename variable after cherry-pick.

* Add NULL check.
2021-12-08 14:46:32 +00:00
Irit Katriel
4d2cc3ed46
bpo-45614: Fix traceback display for exceptions with invalid module name (GH-29726) (GH-29826)
(cherry picked from commit 4dfae6f38e)
2021-11-29 10:07:24 +00:00
Łukasz Langa
904af3de2b
[3.10] bpo-45848: Allow the parser to get error lines from encoded files (GH-29646) (GH-29661)
(cherry picked from commit fdcc46d955)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-11-20 16:34:56 +01:00
Miss Islington (bot)
6d66de97f8
bpo-44959: Add fallback to extension modules with '.sl' suffix on HP-UX (GH-27857) (GH-29152)
(cherry picked from commit 2396fa6537)

Co-authored-by: Florin Spătar <florin.spatar@gmail.com>
2021-11-18 17:19:19 +01:00
Łukasz Langa
8eabe60108
[3.10] bpo-45826: Fix a crash in suggestions.c by checking for traceback is None (GH-29590) (GH-29602)
(cherry picked from commit 5d90c467c0)

Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
2021-11-18 01:28:04 +01:00
Miss Islington (bot)
4ffde90dcc
bpo-45831: _Py_DumpASCII() uses a single write() call if possible (GH-29596) (GH-29598)
If the string is ASCII only and doesn't need to escape characters,
write the whole string with a single write() syscall.
(cherry picked from commit b919d8105c)

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-11-17 22:59:19 +01:00
Miss Islington (bot)
1079b3e3cb
bpo-42540: reallocation of id_mutex should not force default allocator (GH-29564)
Unlike the other locks reinitialized by _PyRuntimeState_ReInitThreads,
the "interpreters.main->id_mutex" is not freed by _PyRuntimeState_Fini
and should not force the default raw allocator.
(cherry picked from commit 736684b1bb)

Co-authored-by: Sam Gross <colesbury@gmail.com>
2021-11-17 13:16:01 -08:00
Brandt Bucher
a89bbde83f
[3.10] bpo-45773: Stop "optimizing" certain jump patterns (GH-29526) 2021-11-11 13:52:43 -08:00
Miss Islington (bot)
b2ae631619
[3.10] bpo-45688: Add _scproxy to sys.stdlib_module_names (GH-29358) (GH-29361)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-02 12:32:36 +01:00
Miss Islington (bot)
854db7e821
Fix format string in _PyImport_LoadDynamicModuleWithSpec() (GH-28863)
(cherry picked from commit f79f3b41c8)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2021-10-12 10:10:59 -07:00
Miss Islington (bot)
56825b697e
Handle error when PyUnicode_GetLength returns a negative value. (GH-28859)
(cherry picked from commit 560a79f94e)

Co-authored-by: Dong-hee Na <donghee.na@python.org>
2021-10-11 04:40:43 -07:00
Miss Islington (bot)
3a58d60620
Fix a leak in _PyImport_LoadDynamicModuleWithSpec() after failing PySys_Audit() (GH-28862)
(cherry picked from commit 9883ca498d)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2021-10-11 02:22:29 -07:00
Christian Clauss
ccd82a0800
[3.10] Fix typos in the Python directory (GH-28767) (GH-28799)
(cherry picked from commit db693df3e1)
2021-10-07 17:30:08 +02:00
Miss Islington (bot)
d0d29655ff
bpo-44050: Extension modules can share state when they don't support sub-interpreters. (GH-27794) (GH-28738)
Automerge-Triggered-By: GH:encukou
(cherry picked from commit b9bb74871b)

Co-authored-by: Hai Shi <shihai1992@gmail.com>
2021-10-05 18:34:59 +02:00
Serhiy Storchaka
1670d590fa
[3.10] bpo-45355: More use of sizeof(_Py_CODEUNIT) (GH-28720). (GH-28721)
(cherry picked from commit 252b7bcb23)
2021-10-04 17:07:21 +03:00
Serhiy Storchaka
b5499784ec
[3.10] bpo-45355: Use sizeof(_Py_CODEUNIT) instead of literal 2 for the size of the code unit (GH-28711). (GH-28718)
(cherry picked from commit 60b9e040c9)
2021-10-04 15:01:11 +03:00
Serhiy Storchaka
93242d7a2a
[3.10] Remove trailing spaces (GH-28709) 2021-10-03 20:03:49 +03:00
Victor Stinner
6df8c32753
bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait() (GH-28671)
On Unix, if the sem_clockwait() function is available in the C
library (glibc 2.30 and newer), the threading.Lock.acquire() method
now uses the monotonic clock (time.CLOCK_MONOTONIC) for the timeout,
rather than using the system clock (time.CLOCK_REALTIME), to not be
affected by system clock changes.

configure now checks if the sem_clockwait() function is available.
2021-10-01 18:22:49 +02:00
Mark Shannon
2be4c370c6
Move predispatch logic from DISPATCH macro to juts before switch. Reduces size of each opocde in interpreter. (GH-28475) 2021-09-29 13:16:13 +01:00
Serhiy Storchaka
ec4e2ec241
[3.10] bpo-45307: Restore private C API function _PyImport_FindExtensionObject() (GH-28594)
py2exe and PyOxidizer rely on this API.
It will be removed in Python 3.11.

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-09-28 23:12:50 +02:00
Brett Cannon
e1bdecb6dc
[3.10] bpo-45183: don't raise an exception when calling zipimport.zipimporter.find_spec() when the zip file is missing and the internal cache has been reset (GH-28435) (#28438)
This can occur when the zip file gets deleted, you call zipimport.zipimporter.invalidate_cache(), and then try to use zipimport.zipimporter.find_spec() (i.e. you left the zip file path on sys.path).
(cherry picked from commit 209b7035f7)

Co-authored-by: Brett Cannon <brett@python.org>
2021-09-17 17:46:22 -07:00
Łukasz Langa
2563dd2d0a
[3.10] bpo-34602: Quadruple stack size on macOS when compiling with UBSAN (GH-27309) (GH-28280)
(cherry picked from commit be9de8721d)

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-09-15 20:34:57 +02:00
Miss Islington (bot)
23c46778d6
bpo-44219: Release the GIL during isatty syscalls (GH-28250) (GH-28255)
Release the GIL while performing isatty() system calls on arbitrary
file descriptors. In particular, this affects os.isatty(),
os.device_encoding() and io.TextIOWrapper. By extension,
io.open() in text mode is also affected.
(cherry picked from commit 06148b1870)

Co-authored-by: Vincent Michel <vxgmichel@gmail.com>
2021-09-09 18:35:43 +02:00
Miss Islington (bot)
6b996d61c9
[3.10] bpo-45083: Include the exception class qualname when formatting an exception (GH-28119) (GH-28134)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
(cherry picked from commit b4b6342848)

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>

* Use a private version of _PyType_GetQualName

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-09-08 18:32:19 +02:00
Łukasz Langa
d41abe8970
[3.10] bpo-45056: Remove trailing unused constants from co_consts (GH-28109) (GH-28125)
(cherry picked from commit 55c4a92fc1)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2021-09-08 18:25:09 +02:00
Miss Islington (bot)
53257cf19f
bpo-45123: PyAiter_Check and PyObject_GetAiter fix & rename. (GH-28194) (GH-28199)
Fix PyAiter_Check to only check for the `__anext__` presense (not for
`__aiter__`). Rename `PyAiter_Check()` to `PyAIter_Check()`,
`PyObject_GetAiter()` -> `PyObject_GetAIter()`.
2021-09-07 12:43:33 +01:00
Miss Islington (bot)
ebbd0ac5d8
bpo-45039: Consistently use ADDOP_LOAD_CONST in compiler rather than ADDOP_O(c, LOAD_CONST,...) (GH-28015)
(cherry picked from commit 70ccee418d)

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2021-08-31 11:08:32 -07:00
Miss Islington (bot)
c4c57e5c0e
bpo-45061: Enhance faulthandler traceback wit no Python frame (GH-28090)
Fix indentation of <no Python frame> message in a faulthandler
traceback or a Fatal Python error traceback. Example:

Current thread 0x00007f03896fb740 (most recent call first):
  Garbage-collecting
  <no Python frame>
(cherry picked from commit 888d4cc06b)

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-08-31 08:53:17 -07:00
Victor Stinner
fe997e1a67
bpo-44449: faulthandler don't modify frame refcnt (GH-27850)
Fix a crash in the signal handler of the faulthandler module: no
longer modify the reference count of frame objects.
2021-08-30 15:24:39 +02:00
Dong-hee Na
32c1caa87f
bpo-45000: Raise SyntaxError when try to delete __debug__ (GH-27947) (GH-27957)
(cherry picked from commit 551da597a0)
2021-08-26 10:52:21 +01:00
Miss Islington (bot)
d86bbe3cff
bpo-25782: avoid hang in PyErr_SetObject when current exception has a cycle in its context chain (GH-27626)
Co-authored-by: Dennis Sweeney 36520290+sweeneyde@users.noreply.github.com
(cherry picked from commit d5c217475c)

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2021-08-10 06:47:23 -07:00
Mark Shannon
0f02993b2c
Remove unused variable. (GH-27677) (#27680) 2021-08-09 15:25:40 +01:00
Mark Shannon
762ef85f44
bpo-44840: Compiler: Move duplication of exit blocks with no line numbers to after CFG optimization. (GH-27656) (#27673)
(cherry picked from commit b854557b49)
2021-08-09 10:54:48 +01:00