[3.5] bpo-29537: Tolerate legacy invalid bytecode (#169)

bpo-27286 fixed a problem where BUILD_MAP_UNPACK_WITH_CALL could
be emitted with an incorrect oparg value, causing the eval loop
to access the wrong stack entry when attempting to read the
function name.

The associated magic number change caused significant problems when
attempting to upgrade to 3.5.3 for anyone that relies on pre-cached
bytecode remaining valid across maintenance releases.

This patch restores the ability to import legacy bytecode generated
by 3.5.0, 3.5.1 or 3.5.2, and modifies the eval loop to
avoid any harmful consequences from the potentially malformed legacy
bytecode.

Original import patch by Petr Viktorin, eval loop patch by Serhiy Storchaka,
and tests and integration by Nick Coghlan.
This commit is contained in:
Nick Coghlan 2017-03-08 16:41:01 +10:00 committed by GitHub
parent bef209d449
commit 93602e3af7
13 changed files with 2828 additions and 2612 deletions

View file

@ -1263,6 +1263,11 @@ eq_mtime(time_t t1, time_t t2)
return d <= 1;
}
/* Issue #29537: handle issue27286 bytecode incompatibility
* See Lib/importlib/_bootstrap_external.py for general discussion
*/
extern PY_UINT32_T _Py_BACKCOMPAT_MAGIC_NUMBER;
/* Given the contents of a .py[co] file in a buffer, unmarshal the data
and return the code object. Return None if it the magic word doesn't
match (we do this instead of raising an exception as we fall back
@ -1274,6 +1279,7 @@ unmarshal_code(PyObject *pathname, PyObject *data, time_t mtime)
PyObject *code;
unsigned char *buf = (unsigned char *)PyBytes_AsString(data);
Py_ssize_t size = PyBytes_Size(data);
PY_UINT32_T magic;
if (size < 12) {
PyErr_SetString(ZipImportError,
@ -1281,7 +1287,10 @@ unmarshal_code(PyObject *pathname, PyObject *data, time_t mtime)
return NULL;
}
if (get_uint32(buf) != (unsigned int)PyImport_GetMagicNumber()) {
magic = get_uint32(buf);
if (magic != (unsigned int)PyImport_GetMagicNumber()
/* Issue #29537: handle issue27286 bytecode incompatibility */
&& magic != _Py_BACKCOMPAT_MAGIC_NUMBER) {
if (Py_VerboseFlag) {
PySys_FormatStderr("# %R has bad magic\n",
pathname);