cpython/Include
Petr Viktorin 49f6beb56a
[3.12] gh-113993: Make interned strings mortal (GH-120520, GH-121364, GH-121903, GH-122303) (#123065)
This backports several PRs for gh-113993, making interned strings mortal so they can be garbage-collected when no longer needed.

* Allow interned strings to be mortal, and fix related issues (GH-120520)

  * Add an InternalDocs file describing how interning should work and how to use it.

  * Add internal functions to *explicitly* request what kind of interning is done:
    - `_PyUnicode_InternMortal`
    - `_PyUnicode_InternImmortal`
    - `_PyUnicode_InternStatic`

  * Switch uses of `PyUnicode_InternInPlace` to those.

  * Disallow using `_Py_SetImmortal` on strings directly.
    You should use `_PyUnicode_InternImmortal` instead:
    - Strings should be interned before immortalization, otherwise you're possibly
      interning a immortalizing copy.
    - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
      `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
      backports, as they are now part of public API and version-specific ABI.

  * Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.

   Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
    - `_Py_ID`
    - `_Py_STR` (including the empty string)
    - one-character latin-1 singletons

    Now, when you intern a singleton, that exact singleton will be interned.

  * Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).

  * Intern `_Py_STR` singletons at startup.

  * Beef up the tests. Cover internal details (marked with `@cpython_only`).

  * Add lots of assertions

* Don't immortalize in PyUnicode_InternInPlace; keep immortalizing in other API (GH-121364)

  * Switch PyUnicode_InternInPlace to _PyUnicode_InternMortal, clarify docs

  * Document immortality in some functions that take `const char *`

  This is PyUnicode_InternFromString;
  PyDict_SetItemString, PyObject_SetAttrString;
  PyObject_DelAttrString; PyUnicode_InternFromString;
  and the PyModule_Add convenience functions.

  Always point out a non-immortalizing alternative.

  * Don't immortalize user-provided attr names in _ctypes

* Immortalize names in code objects to avoid crash (GH-121903)

* Intern latin-1 one-byte strings at startup (GH-122303)

There are some 3.12-specific changes, mainly to allow statically allocated strings in deepfreeze. (In 3.13, deepfreeze switched to the general `_Py_ID`/`_Py_STR`.)

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-09-27 13:28:48 -07:00
..
cpython [3.12] gh-116767: fix crash on 'async with' with many context managers (GH-118348) (#118477) 2024-05-01 18:23:29 +01:00
internal [3.12] gh-113993: Make interned strings mortal (GH-120520, GH-121364, GH-121903, GH-122303) (#123065) 2024-09-27 13:28:48 -07:00
abstract.h gh-98586: Add vector call APIs to the Limited API (GH-98587) 2022-10-27 11:45:42 +02:00
bltinmodule.h
boolobject.h [3.12] gh-106560: Fix redundant declarations in Include/ (#112611) (#112650) 2023-12-03 11:45:32 +00:00
bytearrayobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
bytesobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
ceval.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
codecs.h bpo-41842: Add codecs.unregister() function (GH-22360) 2020-09-28 23:41:11 +02:00
compile.h [3.12] gh-109596: Ensure repeated rules in the grammar are not allowed and fix incorrect soft keywords (GH-109606). (#109752) 2023-10-02 17:22:07 +02:00
complexobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
datetime.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
descrobject.h gh-103509: PEP 697 -- Limited C API for Extending Opaque Types (GH-103511) 2023-05-04 09:56:53 +02:00
dictobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
dynamic_annotations.h gh-96959: Update HTTP links which are redirected to HTTPS (GH-96961) 2022-09-24 14:38:53 +03:00
enumobject.h
errcode.h [3.12] gh-107450: Check for overflow in the tokenizer and fix overflow test (GH-110832) (#110931) 2023-10-16 18:59:18 +02:00
exports.h gh-88267: Avoid DLL exporting functions from static builds on Windows(GH-99888) 2022-12-09 11:16:15 +00:00
fileobject.h [3.12] gh-77782: Deprecate Py_HasFileSystemDefaultEncoding (GH-106272) (#106274) 2023-06-30 10:21:36 +00:00
fileutils.h bpo-45434: Remove useless space in includes (GH-28963) 2021-10-15 01:50:04 +02:00
floatobject.h [3.12] gh-121905: Consistently use "floating-point" instead of "floating point" (GH-121907) (GH-122013) 2024-07-19 09:08:33 +00:00
frameobject.h bpo-45434: Remove useless space in includes (GH-28963) 2021-10-15 01:50:04 +02:00
genericaliasobject.h bpo-44654: Refactor and clean up the union type implementation (GH-27196) 2021-07-17 22:44:10 +03:00
import.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
interpreteridobject.h [3.12] gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-105258) (gh-107303) 2023-07-27 13:15:47 -06:00
intrcheck.h bpo-41713: Remove PyOS_InitInterrupts() function (GH-23342) 2020-11-17 16:22:23 +01:00
iterobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
listobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
longobject.h [3.12] gh-117021: Fix integer overflow in PyLong_AsPid() on non-Windows 64-bit platforms (GH-117064) (GH-117070) 2024-03-20 16:44:05 +02:00
marshal.h bpo-45474: Exclude all of marshal.h if Py_LIMITED_API is defined (GH-29061) 2021-10-20 02:32:14 -07:00
memoryobject.h gh-79315: Add Include/cpython/memoryobject.h header (#99723) 2022-11-23 15:44:42 +01:00
methodobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
modsupport.h [3.12] gh-107226: PyModule_AddObjectRef() should only be in the limited API 3.10 (GH-107227) (GH-107260) 2023-07-25 23:01:18 +03:00
moduleobject.h [3.12] gh-111698: Restrict Py_mod_multiple_interpreters to 3.12+ Under Py_LIMITED_API (gh-111707) (gh-111787) 2023-11-27 19:42:27 -07:00
object.h [3.12] gh-116869: Make C API compatible with ISO C90 (GH-116950) (#117011) 2024-03-19 16:06:15 +00:00
objimpl.h gh-102013: Add PyUnstable_GC_VisitObjects (#102014) 2023-03-14 01:35:54 +00:00
opcode.h gh-103865: add monitoring support to LOAD_SUPER_ATTR (#103866) 2023-05-16 10:29:00 -06:00
osdefs.h
osmodule.h
patchlevel.h Post 3.12.6 2024-09-07 00:22:27 +02:00
py_curses.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
pybuffer.h gh-102500: Implement PEP 688 (#102521) 2023-05-04 07:59:46 -07:00
pycapsule.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
pydtrace.d
pydtrace.h Fix pydtrace.d path comment in Include/pydtrace.h (#28539) 2022-12-31 22:40:25 +05:30
pyerrors.h GH-101578: Normalize the current exception (GH-101607) 2023-02-08 09:31:12 +00:00
pyexpat.h [3.12] gh-115398: Expose Expat >=2.6.0 reparse deferral API (CVE-2023-52425) (GH-115623) (GH-116248) 2024-03-06 22:01:45 +00:00
pyframe.h gh-93937, C API: Move PyFrame_GetBack() to Python.h (#93938) 2022-06-19 12:02:33 +02:00
pyhash.h bpo-29410: Change the default hash algorithm to SipHash13. (GH-28752) 2021-10-10 17:29:46 +09:00
pylifecycle.h bpo-43931: Export Python version as API data (GH-25577) 2021-12-09 17:52:05 -08:00
pymacconfig.h [3.12] gh-110820: Disable test_signal.test_stress_modifying_handlers on macOS (GH-112834) 2023-12-09 15:53:16 +01:00
pymacro.h [3.12] gh-118124: fix assert related C++ checks on Solaris/Illumos (GH-121974) (#122109) 2024-07-22 07:45:17 +00:00
pymath.h gh-104263: Rely on Py_NAN and introduce Py_INFINITY (GH-104202) 2023-05-10 17:44:52 +01:00
pymem.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
pyport.h [3.12] gh-112536: Define _Py_THREAD_SANITIZER on GCC when TSan is enabled (GH-117702) (#117713) 2024-04-10 14:38:10 +00:00
pystate.h bpo-45459: C API uses type names rather than structure names (GH-31528) 2022-02-24 17:51:59 +01:00
pystats.h Remove useless symbol in pystats.h (#101864) 2023-04-09 14:13:21 +05:30
pystrcmp.h
pystrtod.h bpo-45995: add "z" format specifer to coerce negative 0 to zero (GH-30049) 2022-04-11 15:34:18 +01:00
Python.h [3.12] gh-117021: Fix integer overflow in PyLong_AsPid() on non-Windows 64-bit platforms (GH-117064) (GH-117070) 2024-03-20 16:44:05 +02:00
pythonrun.h gh-102755: PyErr_DisplayException only in ABI >= 3.12. Tests cover PyErr_Display as well (GH-102849) 2023-03-21 10:36:18 +01:00
pythread.h bpo-42047: Add native thread ID for DragonFlyBSD (#22714) 2022-05-18 15:10:10 +02:00
pytypedefs.h bpo-45459: Fix PyModuleDef_Slot type in the limited C API (GH-31668) 2022-03-03 23:06:55 +01:00
rangeobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
README.rst gh-101101: Unstable C API tier (PEP 689) (GH-101102) 2023-02-28 09:31:01 +01:00
setobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
sliceobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
structmember.h gh-47146: Soft-deprecate structmember.h, expose its contents via Python.h (GH-99014) 2022-11-22 08:25:43 +01:00
structseq.h gh-87347: Add parenthesis around macro arguments (#93915) 2022-06-20 16:04:52 +02:00
sysmodule.h gh-103295: expose API for writing perf map files (#103546) 2023-05-21 11:12:24 +01:00
traceback.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
tracemalloc.h GH-101520: Move tracemalloc functionality into core, leaving interface in Modules. (#104508) 2023-05-17 14:17:16 +01:00
tupleobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00
typeslots.h bpo-45459: Add Py_buffer to limited API (GH-29991) 2022-02-02 07:03:10 -08:00
unicodeobject.h Fix typo in "expected" word in few source files (#104034) 2023-05-01 09:45:50 -06:00
warnings.h bpo-35134: Split warnings.h and weakrefobject.h (GH-29042) 2021-10-19 01:31:57 +02:00
weakrefobject.h gh-87347: Add parenthesis around PyXXX_Check() arguments (#92815) 2022-06-16 13:49:43 +02:00

The Python C API
================

The C API is divided into these sections:

1. ``Include/``: Limited API
2. ``Include/cpython/``: CPython implementation details
3. ``Include/cpython/``, names with the ``PyUnstable_`` prefix: API that can
   change between minor releases
4. ``Include/internal/``, and any name with ``_`` prefix: The internal API

Information on changing the C API is available `in the developer guide`_

.. _in the developer guide: https://devguide.python.org/c-api/