Commit graph

3540 commits

Author SHA1 Message Date
Eric Snow
52f91c642b
gh-90868: Adjust the Generated Objects (gh-99223)
We do the following:

* move the generated _PyUnicode_InitStaticStrings() to its own file
* move the generated _PyStaticObjects_CheckRefcnt() to its own file
* include pycore_global_objects.h in extension modules instead of pycore_runtime_init.h

These changes help us avoid including things that aren't needed.

https://github.com/python/cpython/issues/90868
2022-11-08 10:03:03 -07:00
Eric Snow
d45cc80452
gh-98627: Add the _testsinglephase Module (gh-99039)
This makes it more clear that a given test is definitely testing against a single-phase init (legacy) extension module.  The new module is a companion to _testmultiphase.

https://github.com/python/cpython/issues/98627
2022-11-08 09:58:11 -07:00
Guido van Rossum
f1a654648b
GH-98831: Simple input-output stack effects for bytecodes.c (#99120) 2022-11-08 08:22:56 -08:00
Nikita Sobolev
e56e33d271
gh-72719: Remove asyncore and asynchat modules (#96580)
Remove modules asyncore and asynchat, which were deprecated by PEP 594.

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-11-08 16:48:58 +01:00
Guido van Rossum
7dcd28eb41
GH-98831: Implement super-instruction generation (#99084)
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
2022-11-06 09:40:47 -08:00
Guido van Rossum
d04899abb0
GH-99104: Update headers for bytecodes.c and generate_cases.py (#99112)
Also tweak the labels near the end of bytecodes.c.
2022-11-04 17:40:43 -07:00
Guido van Rossum
c885623e9f
GH-98831: Auto-generate PREDICTED() macro calls (#99102)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2022-11-04 15:30:17 -07:00
Michael Droettboom
2844aa6a8e
Support comparing two sets of pystats (GH-98816)
This adds support for comparing pystats collected from two different builds.

- The `--json-output` can be used to load in a set of raw stats and output a
  JSON file.
- Two of these JSON files can be provided on the next run, and then comparative
  results between the two are output.
2022-11-04 10:15:54 +00:00
Victor Stinner
0faa0ba240
gh-92584: Remove the distutils package (#99061)
Remove the distutils package. It was deprecated in Python 3.10 by PEP
632 "Deprecate distutils module". For projects still using distutils
and cannot be updated to something else, the setuptools project can
be installed: it still provides distutils.

* Remove Lib/distutils/ directory
* Remove test_distutils
* Remove references to distutils
* Skip test_check_c_globals and test_peg_generator since they use
  distutils
2022-11-03 19:27:27 +01:00
Victor Stinner
e99c507013
gh-92584: Remove references to removed _bootsubprocess (#99062)
The _bootsubprocess module was removed in gh-93939
by commit 81dca70d70.
2022-11-03 18:12:45 +01:00
Victor Stinner
a60ddd31be
gh-98401: Invalid escape sequences emits SyntaxWarning (#99011)
A backslash-character pair that is not a valid escape sequence now
generates a SyntaxWarning, instead of DeprecationWarning.  For
example, re.compile("\d+\.\d+") now emits a SyntaxWarning ("\d" is an
invalid escape sequence), use raw strings for regular expression:
re.compile(r"\d+\.\d+"). In a future Python version, SyntaxError will
eventually be raised, instead of SyntaxWarning.

Octal escapes with value larger than 0o377 (ex: "\477"), deprecated
in Python 3.11, now produce a SyntaxWarning, instead of
DeprecationWarning. In a future Python version they will be
eventually a SyntaxError.

codecs.escape_decode() and codecs.unicode_escape_decode() are left
unchanged: they still emit DeprecationWarning.

* The parser only emits SyntaxWarning for Python 3.12 (feature
  version), and still emits DeprecationWarning on older Python
  versions.
* Fix SyntaxWarning by using raw strings in Tools/c-analyzer/ and
  wasm_build.py.
2022-11-03 17:53:25 +01:00
Guido van Rossum
916af11a97
GH-98831: Remove redundant extract_cases.py script (GH-99065)
Debt I owe from PR GH-98830.
2022-11-03 09:50:35 -07:00
serge-sans-paille
8c4de57de9
gh-98948: Remove obsolete readelf dependency (#98949)
This got introduced in commit 5884449539
to determine if readline is already linked against curses or tinfo in
the setup.py, which is no longer present.
2022-11-03 14:31:37 +01:00
Guido van Rossum
41bc101dd6
GH-98831: "Generate" the interpreter (#98830)
The switch cases (really TARGET(opcode) macros) have been moved from ceval.c to generated_cases.c.h. That file is generated from instruction definitions in bytecodes.c (which impersonates a C file so the C code it contains can be edited without custom support in e.g. VS Code).

The code generator lives in Tools/cases_generator (it has a README.md explaining how it works). The DSL used to describe the instructions is a work in progress, described in https://github.com/faster-cpython/ideas/blob/main/3.12/interpreter_definition.md.

This is surely a work-in-progress. An easy next step could be auto-generating super-instructions.

**IMPORTANT: Merge Conflicts**

If you get a merge conflict for instruction implementations in ceval.c, your best bet is to port your changes to bytecodes.c. That file looks almost the same as the original cases, except instead of `TARGET(NAME)` it uses `inst(NAME)`, and the trailing `DISPATCH()` call is omitted (the code generator adds it automatically).
2022-11-02 21:31:26 -07:00
Serhiy Storchaka
f520d720f6
gh-99016: Make build scripts compatible with Python 3.8 (GH-99017) 2022-11-02 20:30:09 +02:00
Brandt Bucher
276d77724f
GH-98686: Quicken everything (GH-98687) 2022-11-02 10:42:57 -07:00
Neil Schemenauer
de6981680b
gh-90716: add _pylong.py module (#96673)
Add Python implementations of certain longobject.c functions. These use
asymptotically faster algorithms that can be used for operations on
integers with many digits. In those cases, the performance overhead of
the Python implementation is not significant since the asymptotic
behavior is what dominates runtime. Functions provided by this module
should be considered private and not part of any public API.

Co-author: Tim Peters <tim.peters@gmail.com>
Co-author: Mark Dickinson <dickinsm@gmail.com>
Co-author: Bjorn Martinsson
2022-10-25 22:00:50 -07:00
Pablo Galindo Salgado
1f737edb67
gh-96143: Move the perf trampoline files to the Python directory (#98675) 2022-10-25 23:34:22 +01:00
Victor Stinner
1863302d61
gh-97669: Create Tools/build/ directory (#97963)
Create Tools/build/ directory. Move the following scripts from
Tools/scripts/ to Tools/build/:

* check_extension_modules.py
* deepfreeze.py
* freeze_modules.py
* generate_global_objects.py
* generate_levenshtein_examples.py
* generate_opcode_h.py
* generate_re_casefix.py
* generate_sre_constants.py
* generate_stdlib_module_names.py
* generate_token.py
* parse_html5_entities.py
* smelly.py
* stable_abi.py
* umarshal.py
* update_file.py
* verify_ensurepip_wheels.py

Update references to these scripts.
2022-10-17 12:01:00 +02:00
Jakub Kuczys
120b4ab2b6
gh-95731: Fix module docstring extraction in pygettext (#95732) 2022-10-15 07:57:53 -07:00
Victor Stinner
d09d2c7c91
gh-97669: Remove Tools/scripts/startuptime.py (#98214)
The "pyperf command" tool be used instead. Example:

    $ python3 -m pyperf command -- python3 -c pass
    .....................
    command: Mean +- std dev: 17.8 ms +- 0.4 ms

pyperf also computes the standard deviation which gives an idea of
the benchmark looks reliable or not.
2022-10-12 17:53:46 +02:00
Victor Stinner
0895c2a066
gh-97669: Create Tools/patchcheck/ directory (#98186)
Move patchcheck.py, reindent.py and untabify.py scripts to a new
Tools/patchcheck/ directory.
2022-10-12 10:09:21 +02:00
Victor Stinner
454a6d61bc
gh-97669: Remove abitype.py and pep384_macrocheck.py (#98165)
Remove abitype.py and pep384_macrocheck.py scripts of Tools/scripts/.
2022-10-11 13:18:53 +02:00
Victor Stinner
e0ae9ddffe
gh-97669: Remove outdated example scripts (#97675) (#98167)
Remove outdated example scripts of the Tools/scripts/ directory:

* gprof2html.py
* md5sum.py
* nm2def.py
* pathfix.py
* win_add2path.py

Remove test_gprof2html, test_md5sum and test_pathfix of test_tools.
2022-10-11 10:07:57 +02:00
Ken Jin
b399115ef1
gh-95756: Lazily created cached co_* attrs (GH-97791) 2022-10-11 11:26:08 +08:00
Joannah Nanjekye
4ed00be98f
gh-68686: Retire eptag ptag scripts (#98064)
* Retire eptag ptag scripts

* 📜🤖 Added by blurb_it.

* fix news entry error

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-10-08 07:57:47 -07:00
Irit Katriel
34e6f3567e
gh-92886: [clinic.py] raise exception on invalid input instead of assertion (GH-98051)
Tests should pass with -O (assertions off).

Automerge-Triggered-By: GH:iritkatriel
2022-10-07 14:41:35 -07:00
Victor Stinner
002252c4ad
gh-97669: Move difflib examples to Doc/includes/ (#97964)
Remove diff.py and ndiff.py scripts of Tools/scripts/: move them to
Doc/includes/.

* diff.py and ndiff.py files are no longer executable. Remove also
  their shebang ("#!/usr/bin/env python3").
* Remove the -profile command from ndiff.py to simply the code.
* Remove ndiff.py copyright and history command. The Python
  documentation examples are distributed under the "Zero Clause BSD
  License".
2022-10-07 10:57:48 -07:00
Łukasz Langa
bbc7cd649a
gh-97008: Add a Python implementation of AttributeError and NameError suggestions (#97022)
Relevant tests moved from test_exceptions to test_traceback to be able to
compare both implementations.

Co-authored-by: Carl Friedrich Bolz-Tereick <cfbolz@gmx.de>
2022-10-04 15:31:16 -07:00
Victor Stinner
116fa62c6e
gh-97670: Remove sys.getdxp() and analyze_dxp.py script (#97671)
Remove the sys.getdxp() function and the Tools/scripts/analyze_dxp.py
script. DXP stands for "dynamic execution pairs". They were related
to DYNAMIC_EXECUTION_PROFILE and DXPAIRS macros which have been
removed in Python 3.11. Python can now be built with "./configure
--enable-pystats" to gather statistics on Python opcodes.
2022-10-04 15:28:57 +02:00
Victor Stinner
6cbbc26a73
gh-97669: Remove outdated example scripts (#97675)
Remove outdated example scripts of the Tools/scripts/ directory. A
copy can be found in the old-demos project:
https://github.com/gvanrossum/old-demos

Removed scripts (39):

* byext.py
* byteyears.py
* cleanfuture.py
* copytime.py
* crlf.py
* db2pickle.py
* dutree.doc
* dutree.py
* find-uname.py
* find_recursionlimit.py
* finddiv.py
* findlinksto.py
* findnocoding.py
* fixcid.py
* fixdiv.py
* fixheader.py
* fixnotice.py
* fixps.py
* get-remote-certificate.py
* google.py
* highlight.py
* ifdef.py
* import_diagnostics.py
* lfcr.py
* linktree.py
* lll.py
* mailerdaemon.py
* make_ctype.py
* mkreal.py
* objgraph.py
* pdeps.py
* pickle2db.py
* pindent.py
* pysource.py
* reindent-rst.py
* rgrep.py
* suff.py
* texi2html.py
* which.py

Changes:

* Remove test_fixcid, test_lll, test_pdeps and test_pindent
  of test.test_tools.
* Remove get-remote-certificate.py changelog entry, since the script
  was removed.

Note: there is a copy of crlf.py in Lib/test/test_lib2to3/data/.
2022-10-04 10:49:00 +02:00
Victor Stinner
64fe343717
gh-97681: Remove Tools/demo/ directory (#97682)
Remove the Tools/demo/ directory which contained old demo scripts. A
copy can be found in the old-demos project:
https://github.com/gvanrossum/old-demos

Remove the following old demo scripts:

* beer.py
* eiffel.py
* hanoi.py
* life.py
* markov.py
* mcast.py
* queens.py
* redemo.py
* rpython.py
* rpythond.py
* sortvisu.py
* spreadsheet.py
* vector.py

Changes:

* Remove a reference to the redemo.py script in the regex howto
  documentation.
* Remove a reference to the removed Tools/demo/ directory in the
  curses documentation.
* Update PC/layout/ to remove the reference to Tools/demo/ directory.
2022-10-03 17:09:02 +02:00
Serhiy Storchaka
0ee9619a4c
gh-97728: Argument Clinic: Fix uninitialized variable in the Py_UNICODE converter (GH-97729)
It affects function os.system() on Windows and Windows-specific modules
winreg, _winapi, _overlapped, and _msi.
2022-10-03 10:42:54 +03:00
Eric Snow
273a819ed2
gh-90110: Update the c-analyzer Tool (gh-97695)
https://github.com/python/cpython/issues/90110
2022-09-30 19:32:46 -06:00
Steve Dower
ff54dd96cb
gh-90989: Install Windows launcher per-user, and clarify some installer text (GH-97655) 2022-09-30 10:29:31 +01:00
Steve Dower
73942e475c
gh-97649: The Tools directory is no longer installed on Windows (GH-97653) 2022-09-30 10:25:00 +01:00
Victor Stinner
83a0f44ffd
gh-97612: Fix shell injection in get-remote-certificate.py (#97613)
Fix a shell code injection vulnerability in the
get-remote-certificate.py example script. The script no longer uses a
shell to run "openssl" commands. Issue reported and initial fix by
Caleb Shortt.

Remove the Windows code path to send "quit" on stdin to the "openssl
s_client" command: use DEVNULL on all platforms instead.

Co-authored-by: Caleb Shortt <caleb@rgauge.com>
2022-09-29 01:17:27 +02:00
LiarPrincess
0c1d7a06ed
bpo-47243: Duplicate entry in 'Objects/unicodetype_db.h' (GH-32376)
Fix for duplicate 1st entry in 'Objects/unicodetype_db.h':

```c
/* a list of unique character type descriptors */
const _PyUnicode_TypeRecord _PyUnicode_TypeRecords[] = {
    {0, 0, 0, 0, 0, 0},
    {0, 0, 0, 0, 0, 0}, <--- HERE
    {0, 0, 0, 0, 0, 32},
    {0, 0, 0, 0, 0, 48},
    …
```

https://bugs.python.org/issue47243

Automerge-Triggered-By: GH:isidentical
2022-09-28 06:57:14 -07:00
Christian Heimes
0b62964b04
gh-96883: browser: include concurrent.futures (GH-96886) 2022-09-17 13:23:39 +02:00
Benjamin Peterson
fd1e477f53
closes gh-96734: Update to Unicode 15.0.0. (GH-96809) 2022-09-13 15:45:12 -07:00
Eric Snow
1756ffd66a
gh-90110: Fix the c-analyzer Tool (gh-96731)
This includes:

* update the whitelists
* fixes so we can stop ignoring some of the files
* ensure Include/cpython/*.h get analyzed
2022-09-12 11:09:31 -06:00
adang1345
662782e95f
gh-96729: Ensure installers built with Tools/msi/buildrelease.bat have matching UUIDs with official releases (GH-96755) 2022-09-12 15:23:13 +01:00
Christian Heimes
a36235d5c7
gh-84461: Omit resource mod and getresuid funcs on Emscripten (GH-96303) 2022-09-11 10:10:00 +02:00
Christian Heimes
1fc8bd3710
gh-95853: Multiple ops and debug for wasm_build.py (#96744) 2022-09-11 09:51:23 +02:00
Kumar Aditya
4e4bfffe2d
GH-90699: use statically allocated interned strings in typeobject's slotdefs (GH-94706) 2022-09-07 15:02:08 -07:00
Steve Dower
de33df27aa
gh-89545: Updates platform module to use new internal _wmi module on Windows to directly query OS properties (GH-96289) 2022-09-07 21:09:20 +01:00
Kumar Aditya
6dab8c95bd
GH-96458: Statically initialize utf8 representation of static strings (#96481) 2022-09-02 23:43:08 -07:00
Christian Heimes
2eb9008d72
gh-95853: Improve WASM build script (GH-96389)
- pre-build Emscripten ports and system libraries
- check for broken EMSDK versions
- use EMSDK's node for wasm32-emscripten
- warn when PKG_CONFIG_PATH is set
- add support level information
2022-08-30 07:56:26 +02:00
Christian Heimes
d0b3d235db
gh-96320: WASI socket fixes (#96388)
* gh-96320: WASI socket fixes

- ignore missing functions in ``socket.__repr__``
- bundle network files with assets

* blurb
2022-08-30 06:36:11 +02:00
Christian Heimes
873554ef84
gh-94682: Build and test with OpenSSL 1.1.1q (gh-94683) 2022-08-29 18:19:15 +02:00