mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-07-24 11:44:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

gh-113993: Allow interned strings to be mortal, and fix related issues (GH-120520)

Browse source 
* Add an InternalDocs file describing how interning should work and how to use it.

* Add internal functions to *explicitly* request what kind of interning is done:
  - `_PyUnicode_InternMortal`
  - `_PyUnicode_InternImmortal`
  - `_PyUnicode_InternStatic`

* Switch uses of `PyUnicode_InternInPlace` to those.

* Disallow using `_Py_SetImmortal` on strings directly.
  You should use `_PyUnicode_InternImmortal` instead:
  - Strings should be interned before immortalization, otherwise you're possibly
    interning a immortalizing copy.
  - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
    `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
    backports, as they are now part of public API and version-specific ABI.

* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.

* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
  - `_Py_ID`
  - `_Py_STR` (including the empty string)
  - one-character latin-1 singletons

  Now, when you intern a singleton, that exact singleton will be interned.

* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).

* Intern `_Py_STR` singletons at startup.

* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.

* Beef up the tests. Cover internal details (marked with `@cpython_only`).

* Add lots of assertions

Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
This commit is contained in:
Petr Viktorin 2024-06-21 17:19:31 +02:00 • committed by GitHub
parent 7595e6743a
commit 6f1d448bc1
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
42 changed files with 2464 additions and 1140 deletions
Download patch file Download diff file Expand all files Collapse all files

11
Modules/_operator.c
View file

@ -2,6 +2,7 @@
#include "pycore_modsupport.h" // _PyArg_NoKwnames()
#include "pycore_moduleobject.h" // _PyModule_GetState()
#include "pycore_runtime.h" // _Py_ID()
#include "pycore_pystate.h" // _PyInterpreterState_GET()
#include "clinic/_operator.c.h"
@ -1236,6 +1237,7 @@ attrgetter_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
return NULL;
/* prepare attr while checking args */
PyInterpreterState *interp = _PyInterpreterState_GET();
for (idx = 0; idx < nattrs; ++idx) {
PyObject *item = PyTuple_GET_ITEM(args, idx);
int dot_count;
@ -1259,7 +1261,7 @@ attrgetter_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
if (dot_count == 0) {
Py_INCREF(item);
PyUnicode_InternInPlace(&item);
_PyUnicode_InternMortal(interp, &item);
PyTuple_SET_ITEM(attr, idx, item);
} else { /* make it a tuple of non-dotted attrnames */
PyObject *attr_chain = PyTuple_New(dot_count + 1);
@ -1285,7 +1287,7 @@ attrgetter_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
Py_DECREF(attr);
return NULL;
}
PyUnicode_InternInPlace(&attr_chain_item);
_PyUnicode_InternMortal(interp, &attr_chain_item);
PyTuple_SET_ITEM(attr_chain, attr_chain_idx, attr_chain_item);
++attr_chain_idx;
unibuff_till = unibuff_from = unibuff_till + 1;
@ -1299,7 +1301,7 @@ attrgetter_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
Py_DECREF(attr);
return NULL;
}
PyUnicode_InternInPlace(&attr_chain_item);
_PyUnicode_InternMortal(interp, &attr_chain_item);
PyTuple_SET_ITEM(attr_chain, attr_chain_idx, attr_chain_item);
PyTuple_SET_ITEM(attr, idx, attr_chain);
@ -1662,7 +1664,8 @@ methodcaller_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
}
Py_INCREF(name);
PyUnicode_InternInPlace(&name);
PyInterpreterState *interp = _PyInterpreterState_GET();
_PyUnicode_InternMortal(interp, &name);
mc->name = name;
mc->xargs = Py_XNewRef(args); // allows us to use borrowed references