Commit graph

1488 commits

Author SHA1 Message Date
Sergey B Kirpichev
9619e517d4
[3.12] gh-111495: add stub files for C API test modules (GH-111586) (GH-111592)
This is to reduce merge conflicts (Modules/Setup.stdlib.in) for
subsequent pull requests for the issue.
(cherry picked from commit 33ed5fa69d)
2023-11-01 09:43:56 +02:00
Serhiy Storchaka
54e93d36d0
[3.12] gh-111065: Add more tests for the C API with the PySys_ prefix (GH-111067) (GH-111305)
* Move existing tests for PySys_GetObject() and PySys_SetObject() into
  specialized files.
* Add test for PySys_GetXOptions() using _testcapi.
* Add tests for PySys_FormatStdout(), PySys_FormatStderr(),
  PySys_WriteStdout() and PySys_WriteStderr() using ctypes.

(cherry picked from commit b2ba298527)
2023-10-25 09:10:03 +00:00
Miss Islington (bot)
13ad1219c9
[3.12] gh-110437: Allow overriding VCRuntimeDLL with a semicolon separated list of DLLs to bundle (GH-110470)
gh-110437: Allow overriding VCRuntimeDLL with a semicolon separated list of DLLs to bundle (GH-110470)
(cherry picked from commit 12cc6792d0)

Co-authored-by: Steve Dower <steve.dower@python.org>
2023-10-09 11:42:04 +00:00
Nikita Sobolev
3ed5cb0de3
[3.12] gh-110525: Add CAPI tests for set and frozenset objects (GH-110526). (GH-110547)
(cherry picked from commit c49edd7d9c)
2023-10-09 10:04:09 +00:00
Miss Islington (bot)
d1528233b8
[3.12] gh-109286: Update Windows installer to use SQLite 3.43.1 (GH-110403) (#110478)
gh-109286: Update Windows installer to use SQLite 3.43.1 (GH-110403)
(cherry picked from commit 201dc11aeb)

Co-authored-by: jtranquilli <76231120+jtranquilli@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-10-09 11:12:15 +02:00
Miss Islington (bot)
74d0b60811
[3.12] gh-108494: Document how to add a project in PCbuild/readme.txt (GH-110077) (#110230)
gh-108494: Document how to add a project in PCbuild/readme.txt (GH-110077)

(cherry picked from commit 6387b5313c)

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-10-02 19:48:31 +00:00
Miss Islington (bot)
a6773ec099
[3.12] gh-109991: Update Windows build to use OpenSSL 3.0.11 (GH-110054) (#110056)
gh-109991: Update Windows build to use OpenSSL 3.0.11 (GH-110054)
(cherry picked from commit cf4c297256)

Co-authored-by: Zachary Ware <zach@python.org>
2023-10-02 11:59:52 +02:00
Miss Islington (bot)
3d0827e397
[3.12] gh-107565: Update Windows build to use OpenSSL 3.0.10 (GH-108928) (#108930)
gh-107565: Update Windows build to use OpenSSL 3.0.10 (GH-108928)
(cherry picked from commit deea7c8268)

Co-authored-by: Zachary Ware <zach@python.org>
2023-09-05 20:19:06 +02:00
Serhiy Storchaka
9c8dfcec64
[3.12] gh-107178: Add the C API tests for the Abstract Objects Layer (GH-107179) (#107728)
Cover all the Mapping Protocol, almost all the Sequence Protocol
(except PySequence_Fast) and a part of the Object Protocol.

Move existing tests to Lib/test/test_capi/test_abstract.py and
Modules/_testcapi/abstract.c.

Add also tests for PyDict C API..
(cherry picked from commit 16c9415fba)
2023-08-16 14:29:08 +02:00
Miss Islington (bot)
8cb750df13
[3.12] gh-91795: Update build optimization part of PCbuild/readme.txt (GH-91849) (#107776)
gh-91795: Update build optimization part of PCbuild/readme.txt (GH-91849)
(cherry picked from commit 906b73be5e)

Co-authored-by: Fatih <77548106+fatihkabakk@users.noreply.github.com>
2023-08-16 14:24:47 +02:00
Miss Islington (bot)
a71500cd24
[3.12] gh-107814: Avoid output from Nuget installation in find_python.bat (GH-107815) (#107823)
gh-107814: Avoid output from Nuget installation in find_python.bat (GH-107815)
(cherry picked from commit 1e229e2c3d)

Co-authored-by: Max Bachmann <kontakt@maxbachmann.de>
2023-08-10 11:19:14 +02:00
Miss Islington (bot)
b4355de947
[3.12] gh-104629: Build _testclinic extension module on Windows (GH-104723) (#107393)
(cherry picked from commit 3a1d819ebc)

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-07-28 20:57:26 +00:00
Miss Islington (bot)
57ef065eb3
[3.12] gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-105258) (gh-107303)
The _xxsubinterpreters module was meant to only use public API.  Some internal C-API usage snuck in over the last few years (e.g. gh-28969).  This fixes that.
(cherry picked from commit e6373c0d8b)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2023-07-27 13:15:47 -06:00
Miss Islington (bot)
6de9cffcd3
[3.12] gh-99079: Update Windows build to use OpenSSL 3.0.9 (GH-106649) (#106680)
gh-99079: Update Windows build to use OpenSSL 3.0.9 (GH-106649)
(cherry picked from commit e2d7366fb3)

Co-authored-by: Steve Dower <steve.dower@python.org>
2023-07-12 16:30:52 +02:00
Miss Islington (bot)
9e3e5d533e
[3.12] ARM64 clamping bug also exists in MSVC 14.35 (GH-105679) (#105695)
ARM64 clamping bug also exists in MSVC 14.35 (GH-105679)
(cherry picked from commit 58f0bda341)

Co-authored-by: Steve Dower <steve.dower@python.org>
2023-06-12 17:02:30 +01:00
Miss Islington (bot)
25543aecfe
[3.12] gh-103142: Upgrade binary builds and CI to OpenSSL 1.1.1u (GH-105174) (#105199)
gh-103142: Upgrade binary builds and CI to OpenSSL 1.1.1u (GH-105174)

Upgrade builds to OpenSSL 1.1.1u.

This OpenSSL version addresses a pile if less-urgent CVEs since 1.1.1t.

The Mac/BuildScript/build-installer.py was already updated.

Also updates _ssl_data_111.h from OpenSSL 1.1.1u, _ssl_data_300.h from 3.0.9, and adds a new _ssl_data_31.h file from 3.1.1 along with the ssl.c code to use it.

Manual edits to the _ssl_data_300.h file prevent it from removing any existing definitions in case those exist in some peoples builds and were important (avoiding regressions during backporting).

backports of this prior to 3.12 will not include the openssl 3.1 header.
(cherry picked from commit ede89af605)

Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
2023-06-01 17:13:35 +00:00
Miss Islington (bot)
c4bc97a0aa
[3.12] gh-99108: Refresh HACL* (GH-104808) (#104893)
gh-99108: Refresh HACL* (GH-104808)

Refresh HACL* from upstream to improve SHA2 performance and fix a 32-bit issue in SHA3.
(cherry picked from commit 160321e530)

Co-authored-by: Jonathan Protzenko <protz@microsoft.com>
2023-05-24 20:55:37 +00:00
Erlend E. Aasland
aab2a366b7
gh-104623: Update Windows installer to use SQLite 3.42.0 (#104625) 2023-05-19 00:59:40 +02:00
Mark Shannon
f7df173949
GH-101520: Move tracemalloc functionality into core, leaving interface in Modules. (#104508) 2023-05-17 14:17:16 +01:00
Jelle Zijlstra
24d8b88420
gh-103763: Implement PEP 695 (#103764)
This implements PEP 695, Type Parameter Syntax. It adds support for:

- Generic functions (def func[T](): ...)
- Generic classes (class X[T](): ...)
- Type aliases (type X = ...)
- New scoping when the new syntax is used within a class body
- Compiler and interpreter changes to support the new syntax and scoping rules 

Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: Eric Traut <eric@traut.com>
Co-authored-by: Larry Hastings <larry@hastings.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-15 20:36:23 -07:00
Jurica Bradarić
19ee53d52e
gh-93649: Split gc- and allocation tests from _testcapimodule.c (GH-104403) 2023-05-12 10:26:07 +01:00
Jonathan Protzenko
15665d896b
gh-99108: Replace SHA3 implementation HACL* version (#103597)
Replaces our built-in SHA3 implementation with a verified one from the HACL* project.

This implementation is used when OpenSSL does not provide SHA3 or is not present.

3.11 shiped with a very slow tiny sha3 implementation to get off of the <=3.10 reference implementation that wound up having serious bugs. This brings us back to a reasonably performing built-in implementation consistent with what we've just replaced our other guaranteed available standard hash algorithms with: code from the HACL* project.

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-07 20:50:04 -07:00
Carl Meyer
f5c38382f9
gh-103963: fix 'make regen-opcode' in out-of-tree builds (#104177) 2023-05-04 17:45:56 +00:00
Jelle Zijlstra
04f6733275
gh-102500: Implement PEP 688 (#102521)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-05-04 07:59:46 -07:00
Petr Viktorin
cd9a56c2b0
gh-103509: PEP 697 -- Limited C API for Extending Opaque Types (GH-103511)
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-04 09:56:53 +02:00
Dong-hee Na
d81ca7ec02
gh-84436: Add integration C API tests for immortal objects (gh-103962) 2023-05-03 00:05:30 +09:00
Mariusz Felisiak
f6314b92dc
gh-102997: Update Windows installer to SQLite 3.41.2. (#102999) 2023-05-02 09:37:57 +02:00
Mark Shannon
411b169281
GH-103082: Implementation of PEP 669: Low Impact Monitoring for CPython (GH-103083)
* The majority of the monitoring code is in instrumentation.c

* The new instrumentation bytecodes are in bytecodes.c

* legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
2023-04-12 12:04:55 +01:00
Stanislav Syekirin
ebc8103427
gh-103088: Sanitize venv paths when using MSYS or Cygwin Bash (GH-103325) 2023-04-11 20:20:46 +01:00
Irit Katriel
33822d037a
gh-87092: move assembler related code from compile.c to assemble.c (#103277) 2023-04-11 11:15:09 +01:00
Eric Snow
03089fdccc
gh-101659: Add _Py_AtExit() (gh-103298)
The function is like Py_AtExit() but for a single interpreter.  This is a companion to the atexit module's register() function, taking a C callback instead of a Python one.

We also update the _xxinterpchannels module to use _Py_AtExit(), which is the motivating case.  (This is inspired by pain points felt while working on gh-101660.)
2023-04-05 18:42:02 -06:00
Irit Katriel
80163e17d3
gh-87092: move CFG related code from compile.c to flowgraph.c (#103021) 2023-03-31 18:17:59 +01:00
Steve Dower
24ba507b1d
gh-103097: Add workaround for Windows ARM64 compiler bug (GH-103098)
See https://developercommunity.visualstudio.com/t/Regression-in-MSVC-1433-1434-ARM64-co/10224361 for details of the issue. It only applies to version 14.34.
2023-03-29 00:47:13 +01:00
Artem Mukhin
0a60deaeaf
GH-94808: Cover PyOS_mystrnicmp and PyOS_mystricmp (gh-102469) 2023-03-22 20:35:27 +09:00
Eric Snow
ad77d16a62
gh-102304: Move _Py_RefTotal to _PyRuntimeState (gh-102543)
The essentially eliminates the global variable, with the associated benefits. This is also a precursor to isolating this bit of state to PyInterpreterState.

Folks that currently read _Py_RefTotal directly would have to start using _Py_GetGlobalRefTotal() instead.

https://github.com/python/cpython/issues/102304
2023-03-20 10:03:04 -06:00
Steve Dower
f33b33eb31
Increase stack reserve size for Windows debug builds to avoid test crashes (GH-102764) 2023-03-17 01:07:07 +00:00
Steve Dower
0f175766e2
gh-99726: Improves correctness of stat results for Windows, and uses faster API when available (GH-102149)
This deprecates `st_ctime` fields on Windows, with the intent to change them to contain the correct value in 3.14. For now, they should keep returning the creation time as they always have.
2023-03-16 17:27:21 +00:00
Petr Viktorin
6b2d7c0ddb
gh-101101: Unstable C API tier (PEP 689) (GH-101102) 2023-02-28 09:31:01 +01:00
Erlend E. Aasland
efc985a714
gh-93649: Split exception tests from _testcapimodule.c (GH-102173)
Automerge-Triggered-By: GH:erlend-aasland
2023-02-23 07:03:13 -08:00
Jonathan Protzenko
fcadc7e405
gh-99108: Import MD5 and SHA1 from HACL* (#102089)
Replaces our fallback non-OpenSSL MD5 and SHA1 implementations with those from HACL* as we've already done with SHA2.
2023-02-22 13:18:43 -08:00
Gregory P. Smith
0b13575e74
gh-99108: Refactor _sha256 & _sha512 into _sha2. (#101924)
This merges their code. They're backed by the same single HACL* static library, having them be a single module simplifies maintenance.

This should unbreak the wasm enscripten builds that currently fail due to linking in --whole-archive mode and the HACL* library appearing twice.

Long unnoticed error fixed: _sha512.SHA384Type was doubly assigned and was actually SHA512Type. Nobody depends on those internal names.

Also rename LIBHACL_ make vars to LIBHACL_SHA2_ in preperation for other future HACL things.
2023-02-15 22:08:20 -08:00
Steve Dower
e1aadedf09
gh-101763: Update bundled copy of libffi to 3.4.4 on Windows (GH-101784) 2023-02-10 16:57:30 +00:00
Erlend E. Aasland
5d15224011
gh-101759: Update Windows installer to SQLite 3.40.1 (#101762) 2023-02-10 17:38:26 +01:00
Gregory P. Smith
b41c47cd06
gh-101726: Update the OpenSSL version to 1.1.1t (GH-101727)
Fixes CVE-2023-0286 (High) and a couple of Medium security issues.
https://www.openssl.org/news/secadv/20230207.txt
2023-02-09 17:40:51 +00:00
Steve Dower
f23371fbc9
LibFFI build requires x64 Cygwin, and skip the ARM build (GH-101743) 2023-02-09 17:15:19 +00:00
Jonathan Protzenko
1fcc0efdaa
gh-99108: Replace SHA2-224 & 256 with verified code from HACL* (#99109)
replacing hashlib primitives (for the non-OpenSSL case) with verified implementations from HACL*. This is the first PR in the series, and focuses specifically on SHA2-256 and SHA2-224.

This PR imports Hacl_Streaming_SHA2 into the Python tree. This is the HACL* implementation of SHA2, which combines a core implementation of SHA2 along with a layer of buffer management that allows updating the digest with any number of bytes. This supersedes the previous implementation in the tree.

@franziskuskiefer was kind enough to benchmark the changes: in addition to being verified (thus providing significant safety and security improvements), this implementation also provides a sizeable performance boost!

```
---------------------------------------------------------------
Benchmark                     Time             CPU   Iterations
---------------------------------------------------------------
Sha2_256_Streaming            3163 ns      3160 ns       219353     // this PR
LibTomCrypt_Sha2_256          5057 ns      5056 ns       136234     // library used by Python currently
``` 

The changes in this PR are as follows:
- import the subset of HACL* that covers SHA2-256/224 into `Modules/_hacl`
- rewire sha256module.c to use the HACL* implementation

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-02-06 18:11:01 -08:00
Eric Snow
c67b00534a
gh-101524: Split Up the _xxsubinterpreters Module (gh-101526)
This is step 1 in potentially dropping all the "channel"-related code. Channels have already been removed from PEP 554.

https://github.com/python/cpython/issues/101524
2023-02-03 18:14:43 -07:00
Steve Dower
f6c53b80a1
gh-101522: Allow overriding Windows dependencies versions and paths using MSBuild properties (GH-101523) 2023-02-03 18:08:34 +00:00
Steve Dower
737d367b1f
gh-77532: Minor tweaks to allow compiling with PlatformToolset=ClangCL on Windows (GH-101352)
To use this, ensure that clang support was selected in Visual Studio Installer, then set the PlatformToolset environment variable to "ClangCL" and build as normal from the command line.
It remains unsupported, but at least is possible now for experimentation.
2023-01-27 14:45:08 +00:00
Steve Dower
8d18d1ffd5
gh-99834: Update bundled copy of Tcl/Tk to 8.6.13.0 on Windows (GH-101307) 2023-01-26 20:47:24 +00:00