gh-131591: Implement PEP 768 (#131937)

Co-authored-by: Ivona Stojanovic <stojanovic.i@hotmail.com>
Co-authored-by: Matt Wozniski <godlygeek@gmail.com>
This commit is contained in:
Pablo Galindo Salgado 2025-04-03 16:20:01 +01:00 committed by GitHub
parent 275056a7fd
commit 943cc1431e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
31 changed files with 1796 additions and 2 deletions

View file

@ -1835,6 +1835,28 @@ always available. Unless explicitly noted otherwise, all variables are read-only
.. versionadded:: 3.12
.. function:: remote_exec(pid, script)
Executes *script*, a file containing Python code in the remote
process with the given *pid*.
This function returns immediately, and the code will be executed by the
target process's main thread at the next available opportunity, similarly
to how signals are handled. There is no interface to determine when the
code has been executed. The caller is responsible for making sure that
the file still exists whenever the remote process tries to read it and that
it hasn't been overwritten.
The remote process must be running a CPython interpreter of the same major
and minor version as the local process. If either the local or remote
interpreter is pre-release (alpha, beta, or release candidate) then the
local and remote interpreters must be the same exact version.
.. availability:: Unix, Windows.
.. versionadded:: next
.. function:: _enablelegacywindowsfsencoding()
Changes the :term:`filesystem encoding and error handler` to 'mbcs' and

View file

@ -603,6 +603,17 @@ Miscellaneous options
.. versionadded:: 3.13
* ``-X disable_remote_debug`` disables the remote debugging support as described
in :pep:`768`. This includes both the functionality to schedule code for
execution in another process and the functionality to receive code for
execution in the current process.
This option is only available on some platforms and will do nothing
if is not supported on the current system. See also
:envvar:`PYTHON_DISABLE_REMOTE_DEBUG` and :pep:`768`.
.. versionadded:: next
* :samp:`-X cpu_count={n}` overrides :func:`os.cpu_count`,
:func:`os.process_cpu_count`, and :func:`multiprocessing.cpu_count`.
*n* must be greater than or equal to 1.
@ -1160,7 +1171,16 @@ conflict.
.. versionadded:: 3.13
.. envvar:: PYTHON_DISABLE_REMOTE_DEBUG
If this variable is set to a non-empty string, it disables the remote
debugging feature described in :pep:`768`. This includes both the functionality
to schedule code for execution in another process and the functionality to
receive code for execution in the current process.
See also the :option:`-X disable_remote_debug` command-line option.
.. versionadded:: next
.. envvar:: PYTHON_CPU_COUNT

View file

@ -660,6 +660,17 @@ also be used to improve performance.
Add ``-fstrict-overflow`` to the C compiler flags (by default we add
``-fno-strict-overflow`` instead).
.. option:: --without-remote-debug
Deactivate remote debugging support described in :pep:`768` (enabled by default).
When this flag is provided the code that allows the interpreter to schedule the
execution of a Python file in a separate process as described in :pep:`768` is
not compiled. This includes both the functionality to schedule code to be executed
and the functionality to receive code to be executed.
.. versionadded:: next
.. _debug-build:

View file

@ -90,6 +90,63 @@ If you encounter :exc:`NameError`\s or pickling errors coming out of
New features
============
.. _whatsnew314-pep678:
PEP 768: Safe external debugger interface for CPython
-----------------------------------------------------
:pep:`768` introduces a zero-overhead debugging interface that allows debuggers and profilers
to safely attach to running Python processes. This is a significant enhancement to Python's
debugging capabilities allowing debuggers to forego unsafe alternatives.
The new interface provides safe execution points for attaching debugger code without modifying
the interpreter's normal execution path or adding runtime overhead. This enables tools to
inspect and interact with Python applications in real-time without stopping or restarting
them — a crucial capability for high-availability systems and production environments.
For convenience, CPython implements this interface through the :mod:`sys` module with a
:func:`sys.remote_exec` function::
sys.remote_exec(pid, script_path)
This function allows sending Python code to be executed in a target process at the next safe
execution point. However, tool authors can also implement the protocol directly as described
in the PEP, which details the underlying mechanisms used to safely attach to running processes.
Here's a simple example that inspects object types in a running Python process:
.. code-block:: python
import os
import sys
import tempfile
# Create a temporary script
with tempfile.NamedTemporaryFile(mode='w', suffix='.py', delete=False) as f:
script_path = f.name
f.write(f"import my_debugger; my_debugger.connect({os.getpid()})")
try:
# Execute in process with PID 1234
print("Behold! An offering:")
sys.remote_exec(1234, script_path)
finally:
os.unlink(script_path)
The debugging interface has been carefully designed with security in mind and includes several
mechanisms to control access:
* A :envvar:`PYTHON_DISABLE_REMOTE_DEBUG` environment variable.
* A :option:`-X disable-remote-debug` command-line option.
* A :option:`--without-remote-debug` configure flag to completely disable the feature at build time.
A key implementation detail is that the interface piggybacks on the interpreter's existing evaluation
loop and safe points, ensuring zero overhead during normal execution while providing a reliable way
for external processes to coordinate debugging operations.
See :pep:`768` for more details.
(Contributed by Pablo Galindo Salgado, Matt Wozniski, and Ivona Stojanovic in :gh:`131591`.)
.. _whatsnew314-pep758:
PEP 758 Allow except and except* expressions without parentheses

View file

@ -143,6 +143,7 @@ typedef struct PyConfig {
int faulthandler;
int tracemalloc;
int perf_profiling;
int remote_debug;
int import_time;
int code_debug_ranges;
int show_ref_count;

View file

@ -29,6 +29,13 @@ typedef int (*Py_tracefunc)(PyObject *, PyFrameObject *, int, PyObject *);
#define PyTrace_C_RETURN 6
#define PyTrace_OPCODE 7
/* Remote debugger support */
#define MAX_SCRIPT_PATH_SIZE 512
typedef struct _remote_debugger_support {
int32_t debugger_pending_call;
char debugger_script_path[MAX_SCRIPT_PATH_SIZE];
} _PyRemoteDebuggerSupport;
typedef struct _err_stackitem {
/* This struct represents a single execution context where we might
* be currently handling an exception. It is a per-coroutine state
@ -202,6 +209,7 @@ struct _ts {
The PyThreadObject must hold the only reference to this value.
*/
PyObject *threading_local_sentinel;
_PyRemoteDebuggerSupport remote_debugger_support;
};
# define Py_C_RECURSION_LIMIT 5000

View file

@ -347,6 +347,18 @@ void _Py_unset_eval_breaker_bit_all(PyInterpreterState *interp, uintptr_t bit);
PyAPI_FUNC(_PyStackRef) _PyFloat_FromDouble_ConsumeInputs(_PyStackRef left, _PyStackRef right, double value);
#ifndef Py_SUPPORTS_REMOTE_DEBUG
#if defined(__APPLE__)
# if !defined(TARGET_OS_OSX)
// Older macOS SDKs do not define TARGET_OS_OSX
# define TARGET_OS_OSX 1
# endif
#endif
#if ((defined(__APPLE__) && TARGET_OS_OSX) || defined(MS_WINDOWS) || (defined(__linux__) && HAVE_PROCESS_VM_READV))
# define Py_SUPPORTS_REMOTE_DEBUG 1
#endif
#endif
#ifdef __cplusplus
}
#endif

View file

@ -73,6 +73,7 @@ typedef struct _Py_DebugOffsets {
uint64_t id;
uint64_t next;
uint64_t threads_head;
uint64_t threads_main;
uint64_t gc;
uint64_t imports_modules;
uint64_t sysdict;
@ -206,6 +207,15 @@ typedef struct _Py_DebugOffsets {
uint64_t gi_iframe;
uint64_t gi_frame_state;
} gen_object;
struct _debugger_support {
uint64_t eval_breaker;
uint64_t remote_debugger_support;
uint64_t remote_debugging_enabled;
uint64_t debugger_pending_call;
uint64_t debugger_script_path;
uint64_t debugger_script_path_size;
} debugger_support;
} _Py_DebugOffsets;
@ -223,6 +233,7 @@ typedef struct _Py_DebugOffsets {
.id = offsetof(PyInterpreterState, id), \
.next = offsetof(PyInterpreterState, next), \
.threads_head = offsetof(PyInterpreterState, threads.head), \
.threads_main = offsetof(PyInterpreterState, threads.main), \
.gc = offsetof(PyInterpreterState, gc), \
.imports_modules = offsetof(PyInterpreterState, imports.modules), \
.sysdict = offsetof(PyInterpreterState, sysdict), \
@ -326,6 +337,14 @@ typedef struct _Py_DebugOffsets {
.gi_iframe = offsetof(PyGenObject, gi_iframe), \
.gi_frame_state = offsetof(PyGenObject, gi_frame_state), \
}, \
.debugger_support = { \
.eval_breaker = offsetof(PyThreadState, eval_breaker), \
.remote_debugger_support = offsetof(PyThreadState, remote_debugger_support), \
.remote_debugging_enabled = offsetof(PyInterpreterState, config.remote_debug), \
.debugger_pending_call = offsetof(_PyRemoteDebuggerSupport, debugger_pending_call), \
.debugger_script_path = offsetof(_PyRemoteDebuggerSupport, debugger_script_path), \
.debugger_script_path_size = MAX_SCRIPT_PATH_SIZE, \
}, \
}

View file

@ -1195,6 +1195,7 @@ _PyStaticObjects_CheckRefcnt(PyInterpreterState *interp) {
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(salt));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(sched_priority));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(scheduler));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(script));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(second));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(security_attributes));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(seek));

View file

@ -686,6 +686,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(salt)
STRUCT_FOR_ID(sched_priority)
STRUCT_FOR_ID(scheduler)
STRUCT_FOR_ID(script)
STRUCT_FOR_ID(second)
STRUCT_FOR_ID(security_attributes)
STRUCT_FOR_ID(seek)

View file

@ -1193,6 +1193,7 @@ extern "C" {
INIT_ID(salt), \
INIT_ID(sched_priority), \
INIT_ID(scheduler), \
INIT_ID(script), \
INIT_ID(second), \
INIT_ID(security_attributes), \
INIT_ID(seek), \

View file

@ -24,6 +24,8 @@ extern int _PySys_ClearAttrString(PyInterpreterState *interp,
extern int _PySys_SetFlagObj(Py_ssize_t pos, PyObject *new_value);
extern int _PySys_SetIntMaxStrDigits(int maxdigits);
extern int _PySysRemoteDebug_SendExec(int pid, int tid, const char *debugger_script_path);
#ifdef __cplusplus
}
#endif

View file

@ -2532,6 +2532,10 @@ _PyUnicode_InitStaticStrings(PyInterpreterState *interp) {
_PyUnicode_InternStatic(interp, &string);
assert(_PyUnicode_CheckConsistency(string, 1));
assert(PyUnicode_GET_LENGTH(string) != 1);
string = &_Py_ID(script);
_PyUnicode_InternStatic(interp, &string);
assert(_PyUnicode_CheckConsistency(string, 1));
assert(PyUnicode_GET_LENGTH(string) != 1);
string = &_Py_ID(second);
_PyUnicode_InternStatic(interp, &string);
assert(_PyUnicode_CheckConsistency(string, 1));

View file

@ -73,6 +73,7 @@ class CAPITests(unittest.TestCase):
("program_name", str, None),
("pycache_prefix", str | None, "pycache_prefix"),
("quiet", bool, None),
("remote_debug", int, None),
("run_command", str | None, None),
("run_filename", str | None, None),
("run_module", str | None, None),

View file

@ -626,6 +626,7 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase):
'write_bytecode': True,
'verbose': 0,
'quiet': False,
'remote_debug': True,
'user_site_directory': True,
'configure_c_stdio': False,
'buffered_stdio': True,
@ -975,7 +976,7 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase):
'verbose': True,
'quiet': True,
'buffered_stdio': False,
'remote_debug': True,
'user_site_directory': False,
'pathconfig_warnings': False,
}
@ -1031,6 +1032,7 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase):
'write_bytecode': False,
'verbose': 1,
'quiet': True,
'remote_debug': True,
'configure_c_stdio': True,
'buffered_stdio': False,
'user_site_directory': False,

View file

@ -7,17 +7,22 @@ import locale
import operator
import os
import random
import socket
import struct
import subprocess
import sys
import sysconfig
import test.support
from io import StringIO
from unittest import mock
from test import support
from test.support import os_helper
from test.support.script_helper import assert_python_ok, assert_python_failure
from test.support.socket_helper import find_unused_port
from test.support import threading_helper
from test.support import import_helper
from test.support import force_not_colorized
from test.support import SHORT_TIMEOUT
try:
from test.support import interpreters
except ImportError:
@ -1944,5 +1949,235 @@ class SizeofTest(unittest.TestCase):
self.assertEqual(out, b"")
self.assertEqual(err, b"")
def _supports_remote_attaching():
PROCESS_VM_READV_SUPPORTED = False
try:
from _testexternalinspection import PROCESS_VM_READV_SUPPORTED
except ImportError:
pass
return PROCESS_VM_READV_SUPPORTED
@unittest.skipIf(not sys.is_remote_debug_enabled(), "Remote debugging is not enabled")
@unittest.skipIf(sys.platform != "darwin" and sys.platform != "linux" and sys.platform != "win32",
"Test only runs on Linux, Windows and MacOS")
@unittest.skipIf(sys.platform == "linux" and not _supports_remote_attaching(),
"Test only runs on Linux with process_vm_readv support")
@test.support.cpython_only
class TestRemoteExec(unittest.TestCase):
def tearDown(self):
test.support.reap_children()
def _run_remote_exec_test(self, script_code, python_args=None, env=None, prologue=''):
# Create the script that will be remotely executed
script = os_helper.TESTFN + '_remote.py'
self.addCleanup(os_helper.unlink, script)
with open(script, 'w') as f:
f.write(script_code)
# Create and run the target process
target = os_helper.TESTFN + '_target.py'
self.addCleanup(os_helper.unlink, target)
port = find_unused_port()
with open(target, 'w') as f:
f.write(f'''
import sys
import time
import socket
# Connect to the test process
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(('localhost', {port}))
{prologue}
# Signal that the process is ready
sock.sendall(b"ready")
print("Target process running...")
# Wait for remote script to be executed
# (the execution will happen as the following
# code is processed as soon as the recv call
# unblocks)
sock.recv(1024)
# Do a bunch of work to give the remote script time to run
x = 0
for i in range(100):
x += i
# Write confirmation back
sock.sendall(b"executed")
sock.close()
''')
# Start the target process and capture its output
cmd = [sys.executable]
if python_args:
cmd.extend(python_args)
cmd.append(target)
# Create a socket server to communicate with the target process
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('localhost', port))
server_socket.settimeout(SHORT_TIMEOUT)
server_socket.listen(1)
with subprocess.Popen(cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
env=env,
) as proc:
client_socket = None
try:
# Accept connection from target process
client_socket, _ = server_socket.accept()
server_socket.close()
response = client_socket.recv(1024)
self.assertEqual(response, b"ready")
# Try remote exec on the target process
sys.remote_exec(proc.pid, script)
# Signal script to continue
client_socket.sendall(b"continue")
# Wait for execution confirmation
response = client_socket.recv(1024)
self.assertEqual(response, b"executed")
# Return output for test verification
stdout, stderr = proc.communicate(timeout=10.0)
return proc.returncode, stdout, stderr
except PermissionError:
self.skipTest("Insufficient permissions to execute code in remote process")
finally:
if client_socket is not None:
client_socket.close()
proc.kill()
proc.terminate()
proc.wait(timeout=SHORT_TIMEOUT)
def test_remote_exec(self):
"""Test basic remote exec functionality"""
script = '''
print("Remote script executed successfully!")
'''
returncode, stdout, stderr = self._run_remote_exec_test(script)
# self.assertEqual(returncode, 0)
self.assertIn(b"Remote script executed successfully!", stdout)
self.assertEqual(stderr, b"")
def test_remote_exec_with_self_process(self):
"""Test remote exec with the target process being the same as the test process"""
code = 'import sys;print("Remote script executed successfully!", file=sys.stderr)'
file = os_helper.TESTFN + '_remote_self.py'
with open(file, 'w') as f:
f.write(code)
self.addCleanup(os_helper.unlink, file)
with mock.patch('sys.stderr', new_callable=StringIO) as mock_stderr:
with mock.patch('sys.stdout', new_callable=StringIO) as mock_stdout:
sys.remote_exec(os.getpid(), os.path.abspath(file))
print("Done")
self.assertEqual(mock_stderr.getvalue(), "Remote script executed successfully!\n")
self.assertEqual(mock_stdout.getvalue(), "Done\n")
def test_remote_exec_raises_audit_event(self):
"""Test remote exec raises an audit event"""
prologue = '''\
import sys
def audit_hook(event, arg):
print(f"Audit event: {event}, arg: {arg}")
sys.addaudithook(audit_hook)
'''
script = '''
print("Remote script executed successfully!")
'''
returncode, stdout, stderr = self._run_remote_exec_test(script, prologue=prologue)
self.assertEqual(returncode, 0)
self.assertIn(b"Remote script executed successfully!", stdout)
self.assertIn(b"Audit event: remote_debugger_script, arg: ", stdout)
self.assertEqual(stderr, b"")
def test_remote_exec_with_exception(self):
"""Test remote exec with an exception raised in the target process
The exception should be raised in the main thread of the target process
but not crash the target process.
"""
script = '''
raise Exception("Remote script exception")
'''
returncode, stdout, stderr = self._run_remote_exec_test(script)
self.assertEqual(returncode, 0)
self.assertIn(b"Remote script exception", stderr)
self.assertEqual(stdout.strip(), b"Target process running...")
def test_remote_exec_disabled_by_env(self):
"""Test remote exec is disabled when PYTHON_DISABLE_REMOTE_DEBUG is set"""
env = os.environ.copy()
env['PYTHON_DISABLE_REMOTE_DEBUG'] = '1'
with self.assertRaisesRegex(RuntimeError, "Remote debugging is not enabled in the remote process"):
self._run_remote_exec_test("print('should not run')", env=env)
def test_remote_exec_disabled_by_xoption(self):
"""Test remote exec is disabled with -Xdisable-remote-debug"""
with self.assertRaisesRegex(RuntimeError, "Remote debugging is not enabled in the remote process"):
self._run_remote_exec_test("print('should not run')", python_args=['-Xdisable-remote-debug'])
def test_remote_exec_invalid_pid(self):
"""Test remote exec with invalid process ID"""
with self.assertRaises(OSError):
sys.remote_exec(99999, "print('should not run')")
def test_remote_exec_syntax_error(self):
"""Test remote exec with syntax error in script"""
script = '''
this is invalid python code
'''
returncode, stdout, stderr = self._run_remote_exec_test(script)
self.assertEqual(returncode, 0)
self.assertIn(b"SyntaxError", stderr)
self.assertEqual(stdout.strip(), b"Target process running...")
def test_remote_exec_invalid_script_path(self):
"""Test remote exec with invalid script path"""
with self.assertRaises(OSError):
sys.remote_exec(os.getpid(), "invalid_script_path")
def test_remote_exec_in_process_without_debug_fails_envvar(self):
"""Test remote exec in a process without remote debugging enabled"""
script = os_helper.TESTFN + '_remote.py'
self.addCleanup(os_helper.unlink, script)
with open(script, 'w') as f:
f.write('print("Remote script executed successfully!")')
env = os.environ.copy()
env['PYTHON_DISABLE_REMOTE_DEBUG'] = '1'
_, out, err = assert_python_failure('-c', f'import os, sys; sys.remote_exec(os.getpid(), "{script}")', **env)
self.assertIn(b"Remote debugging is not enabled", err)
self.assertEqual(out, b"")
def test_remote_exec_in_process_without_debug_fails_xoption(self):
"""Test remote exec in a process without remote debugging enabled"""
script = os_helper.TESTFN + '_remote.py'
self.addCleanup(os_helper.unlink, script)
with open(script, 'w') as f:
f.write('print("Remote script executed successfully!")')
_, out, err = assert_python_failure('-Xdisable-remote-debug', '-c', f'import os, sys; sys.remote_exec(os.getpid(), "{script}")')
self.assertIn(b"Remote debugging is not enabled", err)
self.assertEqual(out, b"")
if __name__ == "__main__":
unittest.main()

View file

@ -506,6 +506,7 @@ PYTHON_OBJS= \
Python/suggestions.o \
Python/perf_trampoline.o \
Python/perf_jit_trampoline.o \
Python/remote_debugging.o \
Python/$(DYNLOADFILE) \
$(LIBOBJS) \
$(MACHDEP_OBJS) \

View file

@ -0,0 +1,4 @@
Implement :pep:`768` (Safe external debugger interface for CPython). Add a
new :func:`sys.remote_exec` function to the :mod:`sys` module. This function
schedules the execution of a Python file in a separate process. Patch by
Pablo Galindo, Matt Wozniski and Ivona Stojanovic.

View file

@ -260,6 +260,7 @@
<ClCompile Include="..\Python\Python-tokenize.c" />
<ClCompile Include="..\Python\pytime.c" />
<ClCompile Include="..\Python\qsbr.c" />
<ClCompile Include="..\Python\remote_debugging.c" />
<ClCompile Include="..\Python\specialize.c" />
<ClCompile Include="..\Python\structmember.c" />
<ClCompile Include="..\Python\suggestions.c" />

View file

@ -406,6 +406,9 @@
<ClCompile Include="..\Objects\sliceobject.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\Python\remote_debugging.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\Python\specialize.c">
<Filter>Source Files</Filter>
</ClCompile>

View file

@ -95,6 +95,7 @@ if "%~1"=="--experimental-jit" (set UseJIT=true) & (set UseTIER2=1) & shift & go
if "%~1"=="--experimental-jit-off" (set UseJIT=true) & (set UseTIER2=3) & shift & goto CheckOpts
if "%~1"=="--experimental-jit-interpreter" (set UseTIER2=4) & shift & goto CheckOpts
if "%~1"=="--experimental-jit-interpreter-off" (set UseTIER2=6) & shift & goto CheckOpts
if "%~1"=="--without-remote-debug" (set DisableRemoteDebug=true) & shift & goto CheckOpts
if "%~1"=="--pystats" (set PyStats=1) & shift & goto CheckOpts
if "%~1"=="--tail-call-interp" (set UseTailCallInterp=true) & shift & goto CheckOpts
rem These use the actual property names used by MSBuild. We could just let
@ -192,6 +193,7 @@ echo on
/p:UseTIER2=%UseTIER2%^
/p:PyStats=%PyStats%^
/p:UseTailCallInterp=%UseTailCallInterp%^
/p:DisableRemoteDebug=%DisableRemoteDebug%^
%1 %2 %3 %4 %5 %6 %7 %8 %9
@echo off

View file

@ -108,6 +108,7 @@
<PreprocessorDefinitions Condition="'$(UseTIER2)' != '' and '$(UseTIER2)' != '0'">_Py_TIER2=$(UseTIER2);%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions Condition="'$(UseTailCallInterp)' == 'true'">Py_TAIL_CALL_INTERP=1;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions Condition="'$(WITH_COMPUTED_GOTOS)' != ''">HAVE_COMPUTED_GOTOS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions Condition="'$(DisableRemoteDebug)' != 'true'">Py_REMOTE_DEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
</ClCompile>
<Link>
<AdditionalDependencies>version.lib;ws2_32.lib;pathcch.lib;bcrypt.lib;%(AdditionalDependencies)</AdditionalDependencies>
@ -640,6 +641,7 @@
<ClCompile Include="..\Python\pystrcmp.c" />
<ClCompile Include="..\Python\pystrhex.c" />
<ClCompile Include="..\Python\pystrtod.c" />
<ClCompile Include="..\Python\remote_debugging.c" />
<ClCompile Include="..\Python\qsbr.c" />
<ClCompile Include="..\Python\dtoa.c" />
<ClCompile Include="..\Python\Python-ast.c" />

View file

@ -1490,6 +1490,9 @@
<ClCompile Include="..\Python\pystrtod.c">
<Filter>Python</Filter>
</ClCompile>
<ClCompile Include="..\Python\remote_debugging.c">
<Filter>Python</Filter>
</ClCompile>
<ClCompile Include="..\Python\qsbr.c">
<Filter>Python</Filter>
</ClCompile>

View file

@ -1192,6 +1192,71 @@ _PyEval_DisableGIL(PyThreadState *tstate)
}
#endif
#if defined(Py_REMOTE_DEBUG) && defined(Py_SUPPORTS_REMOTE_DEBUG)
// Note that this function is inline to avoid creating a PLT entry
// that would be an easy target for a ROP gadget.
static inline void run_remote_debugger_script(const char *path)
{
if (0 != PySys_Audit("remote_debugger_script", "s", path)) {
PyErr_FormatUnraisable(
"Audit hook failed for remote debugger script %s", path);
return;
}
// Open the debugger script with the open code hook, and reopen the
// resulting file object to get a C FILE* object.
PyObject* fileobj = PyFile_OpenCode(path);
if (!fileobj) {
PyErr_FormatUnraisable("Can't open debugger script %s", path);
return;
}
int fd = PyObject_AsFileDescriptor(fileobj);
if (fd == -1) {
PyErr_FormatUnraisable("Can't find fd for debugger script %s", path);
}
else {
int dup_fd = -1;
FILE *f = NULL;
#ifdef MS_WINDOWS
dup_fd = _dup(fd);
if (dup_fd != -1) {
f = _fdopen(dup_fd, "r");
}
if (!f) {
_close(dup_fd);
}
#else
dup_fd = dup(fd);
if (dup_fd != -1) {
f = fdopen(dup_fd, "r");
}
if (!f) {
close(dup_fd);
}
#endif
if (!f) {
PyErr_SetFromErrno(PyExc_OSError);
}
else {
PyRun_AnyFileEx(f, path, 1);
}
if (PyErr_Occurred()) {
PyErr_FormatUnraisable("Error executing debugger script %s", path);
}
}
PyObject* res = PyObject_CallMethodNoArgs(fileobj, &_Py_ID(close));
if (!res) {
PyErr_FormatUnraisable("Error closing debugger script %s", path);
} else {
Py_DECREF(res);
}
Py_DECREF(fileobj);
}
#endif
/* Do periodic things, like check for signals and async I/0.
* We need to do reasonably frequently, but not too frequently.
@ -1319,5 +1384,35 @@ _Py_HandlePending(PyThreadState *tstate)
return -1;
}
}
#if defined(Py_REMOTE_DEBUG) && defined(Py_SUPPORTS_REMOTE_DEBUG)
const PyConfig *config = _PyInterpreterState_GetConfig(tstate->interp);
if (config->remote_debug == 1
&& tstate->remote_debugger_support.debugger_pending_call == 1)
{
tstate->remote_debugger_support.debugger_pending_call = 0;
// Immediately make a copy in case of a race with another debugger
// process that's trying to write to the buffer. At least this way
// we'll be internally consistent: what we audit is what we run.
const size_t pathsz
= sizeof(tstate->remote_debugger_support.debugger_script_path);
char *path = PyMem_Malloc(pathsz);
if (path) {
// And don't assume the debugger correctly null terminated it.
memcpy(
path,
tstate->remote_debugger_support.debugger_script_path,
pathsz);
path[pathsz - 1] = '\0';
if (*path) {
run_remote_debugger_script(path);
}
PyMem_Free(path);
}
}
#endif
return 0;
}

View file

@ -1519,6 +1519,104 @@ sys_is_stack_trampoline_active(PyObject *module, PyObject *Py_UNUSED(ignored))
return sys_is_stack_trampoline_active_impl(module);
}
PyDoc_STRVAR(sys_is_remote_debug_enabled__doc__,
"is_remote_debug_enabled($module, /)\n"
"--\n"
"\n"
"Return True if remote debugging is enabled, False otherwise.");
#define SYS_IS_REMOTE_DEBUG_ENABLED_METHODDEF \
{"is_remote_debug_enabled", (PyCFunction)sys_is_remote_debug_enabled, METH_NOARGS, sys_is_remote_debug_enabled__doc__},
static PyObject *
sys_is_remote_debug_enabled_impl(PyObject *module);
static PyObject *
sys_is_remote_debug_enabled(PyObject *module, PyObject *Py_UNUSED(ignored))
{
return sys_is_remote_debug_enabled_impl(module);
}
PyDoc_STRVAR(sys_remote_exec__doc__,
"remote_exec($module, /, pid, script)\n"
"--\n"
"\n"
"Executes a file containing Python code in a given remote Python process.\n"
"\n"
"This function returns immediately, and the code will be executed by the\n"
"target process\'s main thread at the next available opportunity, similarly\n"
"to how signals are handled. There is no interface to determine when the\n"
"code has been executed. The caller is responsible for making sure that\n"
"the file still exists whenever the remote process tries to read it and that\n"
"it hasn\'t been overwritten.\n"
"\n"
"The remote process must be running a CPython interpreter of the same major\n"
"and minor version as the local process. If either the local or remote\n"
"interpreter is pre-release (alpha, beta, or release candidate) then the\n"
"local and remote interpreters must be the same exact version.\n"
"\n"
"Args:\n"
" pid (int): The process ID of the target Python process.\n"
" script (str|bytes): The path to a file containing\n"
" the Python code to be executed.");
#define SYS_REMOTE_EXEC_METHODDEF \
{"remote_exec", _PyCFunction_CAST(sys_remote_exec), METH_FASTCALL|METH_KEYWORDS, sys_remote_exec__doc__},
static PyObject *
sys_remote_exec_impl(PyObject *module, int pid, PyObject *script);
static PyObject *
sys_remote_exec(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
PyObject *return_value = NULL;
#if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
#define NUM_KEYWORDS 2
static struct {
PyGC_Head _this_is_not_used;
PyObject_VAR_HEAD
Py_hash_t ob_hash;
PyObject *ob_item[NUM_KEYWORDS];
} _kwtuple = {
.ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS)
.ob_hash = -1,
.ob_item = { &_Py_ID(pid), &_Py_ID(script), },
};
#undef NUM_KEYWORDS
#define KWTUPLE (&_kwtuple.ob_base.ob_base)
#else // !Py_BUILD_CORE
# define KWTUPLE NULL
#endif // !Py_BUILD_CORE
static const char * const _keywords[] = {"pid", "script", NULL};
static _PyArg_Parser _parser = {
.keywords = _keywords,
.fname = "remote_exec",
.kwtuple = KWTUPLE,
};
#undef KWTUPLE
PyObject *argsbuf[2];
int pid;
PyObject *script;
args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser,
/*minpos*/ 2, /*maxpos*/ 2, /*minkw*/ 0, /*varpos*/ 0, argsbuf);
if (!args) {
goto exit;
}
pid = PyLong_AsInt(args[0]);
if (pid == -1 && PyErr_Occurred()) {
goto exit;
}
script = args[1];
return_value = sys_remote_exec_impl(module, pid, script);
exit:
return return_value;
}
PyDoc_STRVAR(sys__dump_tracelets__doc__,
"_dump_tracelets($module, /, outpath)\n"
"--\n"
@ -1766,4 +1864,4 @@ exit:
#ifndef SYS_GETANDROIDAPILEVEL_METHODDEF
#define SYS_GETANDROIDAPILEVEL_METHODDEF
#endif /* !defined(SYS_GETANDROIDAPILEVEL_METHODDEF) */
/*[clinic end generated code: output=75e202eec4450f50 input=a9049054013a1b77]*/
/*[clinic end generated code: output=1aca52cefbeb800f input=a9049054013a1b77]*/

View file

@ -162,6 +162,7 @@ static const PyConfigSpec PYCONFIG_SPEC[] = {
SPEC(parse_argv, BOOL, READ_ONLY, NO_SYS),
SPEC(pathconfig_warnings, BOOL, READ_ONLY, NO_SYS),
SPEC(perf_profiling, UINT, READ_ONLY, NO_SYS),
SPEC(remote_debug, BOOL, READ_ONLY, NO_SYS),
SPEC(program_name, WSTR, READ_ONLY, NO_SYS),
SPEC(run_command, WSTR_OPT, READ_ONLY, NO_SYS),
SPEC(run_filename, WSTR_OPT, READ_ONLY, NO_SYS),
@ -317,6 +318,7 @@ The following implementation-specific options are available:\n\
-X perf: support the Linux \"perf\" profiler; also PYTHONPERFSUPPORT=1\n\
-X perf_jit: support the Linux \"perf\" profiler with DWARF support;\n\
also PYTHON_PERF_JIT_SUPPORT=1\n\
-X disable-remote-debug: disable remote debugging; also PYTHON_DISABLE_REMOTE_DEBUG\n\
"
#ifdef Py_DEBUG
"-X presite=MOD: import this module before site; also PYTHON_PRESITE\n"
@ -994,6 +996,7 @@ _PyConfig_InitCompatConfig(PyConfig *config)
config->faulthandler = -1;
config->tracemalloc = -1;
config->perf_profiling = -1;
config->remote_debug = -1;
config->module_search_paths_set = 0;
config->parse_argv = 0;
config->site_import = -1;
@ -1986,6 +1989,28 @@ config_init_perf_profiling(PyConfig *config)
}
static PyStatus
config_init_remote_debug(PyConfig *config)
{
#ifndef Py_REMOTE_DEBUG
config->remote_debug = 0;
#else
int active = 1;
const char *env = Py_GETENV("PYTHON_DISABLE_REMOTE_DEBUG");
if (env) {
active = 0;
}
const wchar_t *xoption = config_get_xoption(config, L"disable-remote-debug");
if (xoption) {
active = 0;
}
config->remote_debug = active;
#endif
return _PyStatus_OK();
}
static PyStatus
config_init_tracemalloc(PyConfig *config)
{
@ -2170,6 +2195,13 @@ config_read_complex_options(PyConfig *config)
}
}
if (config->remote_debug < 0) {
status = config_init_remote_debug(config);
if (_PyStatus_EXCEPTION(status)) {
return status;
}
}
if (config->int_max_str_digits < 0) {
status = config_init_int_max_str_digits(config);
if (_PyStatus_EXCEPTION(status)) {
@ -2531,6 +2563,9 @@ config_read(PyConfig *config, int compute_path_config)
if (config->perf_profiling < 0) {
config->perf_profiling = 0;
}
if (config->remote_debug < 0) {
config->remote_debug = -1;
}
if (config->use_hash_seed < 0) {
config->use_hash_seed = 0;
config->hash_seed = 0;

984
Python/remote_debugging.c Normal file
View file

@ -0,0 +1,984 @@
#define _GNU_SOURCE
#include "pyconfig.h"
#include "Python.h"
#include "internal/pycore_runtime.h"
#include "internal/pycore_ceval.h"
#ifdef __linux__
# include <elf.h>
# include <sys/uio.h>
# if INTPTR_MAX == INT64_MAX
# define Elf_Ehdr Elf64_Ehdr
# define Elf_Shdr Elf64_Shdr
# define Elf_Phdr Elf64_Phdr
# else
# define Elf_Ehdr Elf32_Ehdr
# define Elf_Shdr Elf32_Shdr
# define Elf_Phdr Elf32_Phdr
# endif
# include <sys/mman.h>
#endif
#if defined(__APPLE__)
# include <TargetConditionals.h>
// Older macOS SDKs do not define TARGET_OS_OSX
# if !defined(TARGET_OS_OSX)
# define TARGET_OS_OSX 1
# endif
# if TARGET_OS_OSX
# include <libproc.h>
# include <mach-o/fat.h>
# include <mach-o/loader.h>
# include <mach-o/nlist.h>
# include <mach/mach.h>
# include <mach/mach_vm.h>
# include <mach/machine.h>
# include <sys/mman.h>
# include <sys/proc.h>
# include <sys/sysctl.h>
# endif
#endif
#ifdef MS_WINDOWS
// Windows includes and definitions
#include <windows.h>
#include <psapi.h>
#include <tlhelp32.h>
#endif
#include <errno.h>
#include <fcntl.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifndef MS_WINDOWS
#include <sys/param.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#endif
#ifndef HAVE_PROCESS_VM_READV
# define HAVE_PROCESS_VM_READV 0
#endif
// Define a platform-independent process handle structure
typedef struct {
pid_t pid;
#ifdef MS_WINDOWS
HANDLE hProcess;
#endif
} proc_handle_t;
// Initialize the process handle
static int
init_proc_handle(proc_handle_t *handle, pid_t pid) {
handle->pid = pid;
#ifdef MS_WINDOWS
handle->hProcess = OpenProcess(
PROCESS_VM_READ | PROCESS_VM_WRITE | PROCESS_VM_OPERATION | PROCESS_QUERY_INFORMATION,
FALSE, pid);
if (handle->hProcess == NULL) {
PyErr_SetFromWindowsErr(0);
return -1;
}
#endif
return 0;
}
// Clean up the process handle
static void
cleanup_proc_handle(proc_handle_t *handle) {
#ifdef MS_WINDOWS
if (handle->hProcess != NULL) {
CloseHandle(handle->hProcess);
handle->hProcess = NULL;
}
#endif
handle->pid = 0;
}
#if defined(Py_REMOTE_DEBUG) && defined(Py_SUPPORTS_REMOTE_DEBUG)
#if defined(__APPLE__) && TARGET_OS_OSX
static uintptr_t
return_section_address(
const char* section,
mach_port_t proc_ref,
uintptr_t base,
void* map
) {
struct mach_header_64* hdr = (struct mach_header_64*)map;
int ncmds = hdr->ncmds;
int cmd_cnt = 0;
struct segment_command_64* cmd = map + sizeof(struct mach_header_64);
mach_vm_size_t size = 0;
mach_msg_type_number_t count = sizeof(vm_region_basic_info_data_64_t);
mach_vm_address_t address = (mach_vm_address_t)base;
vm_region_basic_info_data_64_t r_info;
mach_port_t object_name;
uintptr_t vmaddr = 0;
for (int i = 0; cmd_cnt < 2 && i < ncmds; i++) {
if (cmd->cmd == LC_SEGMENT_64 && strcmp(cmd->segname, "__TEXT") == 0) {
vmaddr = cmd->vmaddr;
}
if (cmd->cmd == LC_SEGMENT_64 && strcmp(cmd->segname, "__DATA") == 0) {
while (cmd->filesize != size) {
address += size;
kern_return_t ret = mach_vm_region(
proc_ref,
&address,
&size,
VM_REGION_BASIC_INFO_64,
(vm_region_info_t)&r_info, // cppcheck-suppress [uninitvar]
&count,
&object_name
);
if (ret != KERN_SUCCESS) {
PyErr_SetString(
PyExc_RuntimeError, "Cannot get any more VM maps.\n");
return 0;
}
}
int nsects = cmd->nsects;
struct section_64* sec = (struct section_64*)(
(void*)cmd + sizeof(struct segment_command_64)
);
for (int j = 0; j < nsects; j++) {
if (strcmp(sec[j].sectname, section) == 0) {
return base + sec[j].addr - vmaddr;
}
}
cmd_cnt++;
}
cmd = (struct segment_command_64*)((void*)cmd + cmd->cmdsize);
}
// We should not be here, but if we are there, we should say about this
PyErr_SetString(
PyExc_RuntimeError, "Cannot find section address.\n");
return 0;
}
static uintptr_t
search_section_in_file(const char* secname, char* path, uintptr_t base, mach_vm_size_t size, mach_port_t proc_ref)
{
int fd = open(path, O_RDONLY);
if (fd == -1) {
PyErr_Format(PyExc_RuntimeError, "Cannot open binary %s\n", path);
return 0;
}
struct stat fs;
if (fstat(fd, &fs) == -1) {
PyErr_Format(PyExc_RuntimeError, "Cannot get size of binary %s\n", path);
close(fd);
return 0;
}
void* map = mmap(0, fs.st_size, PROT_READ, MAP_SHARED, fd, 0);
if (map == MAP_FAILED) {
PyErr_Format(PyExc_RuntimeError, "Cannot map binary %s\n", path);
close(fd);
return 0;
}
uintptr_t result = 0;
struct mach_header_64* hdr = (struct mach_header_64*)map;
switch (hdr->magic) {
case MH_MAGIC:
case MH_CIGAM:
case FAT_MAGIC:
case FAT_CIGAM:
PyErr_SetString(PyExc_RuntimeError, "32-bit Mach-O binaries are not supported");
break;
case MH_MAGIC_64:
case MH_CIGAM_64:
result = return_section_address(secname, proc_ref, base, map);
break;
default:
PyErr_SetString(PyExc_RuntimeError, "Unknown Mach-O magic");
break;
}
munmap(map, fs.st_size);
if (close(fd) != 0) {
PyErr_SetFromErrno(PyExc_OSError);
}
return result;
}
static mach_port_t
pid_to_task(pid_t pid)
{
mach_port_t task;
kern_return_t result;
result = task_for_pid(mach_task_self(), pid, &task);
if (result != KERN_SUCCESS) {
PyErr_Format(PyExc_PermissionError, "Cannot get task for PID %d", pid);
return 0;
}
return task;
}
static uintptr_t
search_map_for_section(proc_handle_t *handle, const char* secname, const char* substr) {
mach_vm_address_t address = 0;
mach_vm_size_t size = 0;
mach_msg_type_number_t count = sizeof(vm_region_basic_info_data_64_t);
vm_region_basic_info_data_64_t region_info;
mach_port_t object_name;
mach_port_t proc_ref = pid_to_task(handle->pid);
if (proc_ref == 0) {
PyErr_SetString(PyExc_PermissionError, "Cannot get task for PID");
return 0;
}
int match_found = 0;
char map_filename[MAXPATHLEN + 1];
while (mach_vm_region(
proc_ref,
&address,
&size,
VM_REGION_BASIC_INFO_64,
(vm_region_info_t)&region_info,
&count,
&object_name) == KERN_SUCCESS)
{
if ((region_info.protection & VM_PROT_READ) == 0
|| (region_info.protection & VM_PROT_EXECUTE) == 0) {
address += size;
continue;
}
int path_len = proc_regionfilename(
handle->pid, address, map_filename, MAXPATHLEN);
if (path_len == 0) {
address += size;
continue;
}
char* filename = strrchr(map_filename, '/');
if (filename != NULL) {
filename++; // Move past the '/'
} else {
filename = map_filename; // No path, use the whole string
}
if (!match_found && strncmp(filename, substr, strlen(substr)) == 0) {
match_found = 1;
return search_section_in_file(
secname, map_filename, address, size, proc_ref);
}
address += size;
}
PyErr_SetString(PyExc_RuntimeError,
"mach_vm_region failed to find the section");
return 0;
}
#endif // (__APPLE__ && TARGET_OS_OSX)
#if defined(__linux__) && HAVE_PROCESS_VM_READV
static uintptr_t
search_elf_file_for_section(
proc_handle_t *handle,
const char* secname,
uintptr_t start_address,
const char *elf_file)
{
if (start_address == 0) {
return 0;
}
uintptr_t result = 0;
void* file_memory = NULL;
int fd = open(elf_file, O_RDONLY);
if (fd < 0) {
PyErr_SetFromErrno(PyExc_OSError);
goto exit;
}
struct stat file_stats;
if (fstat(fd, &file_stats) != 0) {
PyErr_SetFromErrno(PyExc_OSError);
goto exit;
}
file_memory = mmap(NULL, file_stats.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
if (file_memory == MAP_FAILED) {
PyErr_SetFromErrno(PyExc_OSError);
goto exit;
}
Elf_Ehdr* elf_header = (Elf_Ehdr*)file_memory;
Elf_Shdr* section_header_table = (Elf_Shdr*)(file_memory + elf_header->e_shoff);
Elf_Shdr* shstrtab_section = &section_header_table[elf_header->e_shstrndx];
char* shstrtab = (char*)(file_memory + shstrtab_section->sh_offset);
Elf_Shdr* section = NULL;
for (int i = 0; i < elf_header->e_shnum; i++) {
char* this_sec_name = shstrtab + section_header_table[i].sh_name;
// Move 1 character to account for the leading "."
this_sec_name += 1;
if (strcmp(secname, this_sec_name) == 0) {
section = &section_header_table[i];
break;
}
}
Elf_Phdr* program_header_table = (Elf_Phdr*)(file_memory + elf_header->e_phoff);
// Find the first PT_LOAD segment
Elf_Phdr* first_load_segment = NULL;
for (int i = 0; i < elf_header->e_phnum; i++) {
if (program_header_table[i].p_type == PT_LOAD) {
first_load_segment = &program_header_table[i];
break;
}
}
if (section != NULL && first_load_segment != NULL) {
uintptr_t elf_load_addr = first_load_segment->p_vaddr
- (first_load_segment->p_vaddr % first_load_segment->p_align);
result = start_address + (uintptr_t)section->sh_addr - elf_load_addr;
}
exit:
if (file_memory != NULL) {
munmap(file_memory, file_stats.st_size);
}
if (fd >= 0 && close(fd) != 0) {
PyErr_SetFromErrno(PyExc_OSError);
}
return result;
}
static uintptr_t
search_linux_map_for_section(proc_handle_t *handle, const char* secname, const char* substr)
{
char maps_file_path[64];
sprintf(maps_file_path, "/proc/%d/maps", handle->pid);
FILE* maps_file = fopen(maps_file_path, "r");
if (maps_file == NULL) {
PyErr_SetFromErrno(PyExc_OSError);
return 0;
}
size_t linelen = 0;
size_t linesz = PATH_MAX;
char *line = PyMem_Malloc(linesz);
if (!line) {
fclose(maps_file);
PyErr_NoMemory();
return 0;
}
uintptr_t retval = 0;
while (fgets(line + linelen, linesz - linelen, maps_file) != NULL) {
linelen = strlen(line);
if (line[linelen - 1] != '\n') {
// Read a partial line: realloc and keep reading where we left off.
// Note that even the last line will be terminated by a newline.
linesz *= 2;
char *biggerline = PyMem_Realloc(line, linesz);
if (!biggerline) {
PyMem_Free(line);
fclose(maps_file);
PyErr_NoMemory();
return 0;
}
line = biggerline;
continue;
}
// Read a full line: strip the newline
line[linelen - 1] = '\0';
// and prepare to read the next line into the start of the buffer.
linelen = 0;
unsigned long start = 0;
unsigned long path_pos = 0;
sscanf(line, "%lx-%*x %*s %*s %*s %*s %ln", &start, &path_pos);
if (!path_pos) {
// Line didn't match our format string. This shouldn't be
// possible, but let's be defensive and skip the line.
continue;
}
const char *path = line + path_pos;
const char *filename = strrchr(path, '/');
if (filename) {
filename++; // Move past the '/'
} else {
filename = path; // No directories, or an empty string
}
if (strstr(filename, substr)) {
retval = search_elf_file_for_section(handle, secname, start, path);
if (retval) {
break;
}
}
}
PyMem_Free(line);
fclose(maps_file);
return retval;
}
#endif // __linux__
#ifdef MS_WINDOWS
static void* analyze_pe(const wchar_t* mod_path, BYTE* remote_base, const char* secname) {
HANDLE hFile = CreateFileW(mod_path, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
if (hFile == INVALID_HANDLE_VALUE) {
PyErr_SetFromWindowsErr(0);
return NULL;
}
HANDLE hMap = CreateFileMapping(hFile, NULL, PAGE_READONLY, 0, 0, 0);
if (!hMap) {
PyErr_SetFromWindowsErr(0);
CloseHandle(hFile);
return NULL;
}
BYTE* mapView = (BYTE*)MapViewOfFile(hMap, FILE_MAP_READ, 0, 0, 0);
if (!mapView) {
PyErr_SetFromWindowsErr(0);
CloseHandle(hMap);
CloseHandle(hFile);
return NULL;
}
IMAGE_DOS_HEADER* pDOSHeader = (IMAGE_DOS_HEADER*)mapView;
if (pDOSHeader->e_magic != IMAGE_DOS_SIGNATURE) {
PyErr_SetString(PyExc_RuntimeError, "Invalid DOS signature.");
UnmapViewOfFile(mapView);
CloseHandle(hMap);
CloseHandle(hFile);
return NULL;
}
IMAGE_NT_HEADERS* pNTHeaders = (IMAGE_NT_HEADERS*)(mapView + pDOSHeader->e_lfanew);
if (pNTHeaders->Signature != IMAGE_NT_SIGNATURE) {
PyErr_SetString(PyExc_RuntimeError, "Invalid NT signature.");
UnmapViewOfFile(mapView);
CloseHandle(hMap);
CloseHandle(hFile);
return NULL;
}
IMAGE_SECTION_HEADER* pSection_header = (IMAGE_SECTION_HEADER*)(mapView + pDOSHeader->e_lfanew + sizeof(IMAGE_NT_HEADERS));
void* runtime_addr = NULL;
for (int i = 0; i < pNTHeaders->FileHeader.NumberOfSections; i++) {
const char* name = (const char*)pSection_header[i].Name;
if (strncmp(name, secname, IMAGE_SIZEOF_SHORT_NAME) == 0) {
runtime_addr = remote_base + pSection_header[i].VirtualAddress;
break;
}
}
UnmapViewOfFile(mapView);
CloseHandle(hMap);
CloseHandle(hFile);
return runtime_addr;
}
static uintptr_t
search_windows_map_for_section(proc_handle_t* handle, const char* secname, const wchar_t* substr) {
HANDLE hProcSnap;
do {
hProcSnap = CreateToolhelp32Snapshot(TH32CS_SNAPMODULE, handle->pid);
} while (hProcSnap == INVALID_HANDLE_VALUE && GetLastError() == ERROR_BAD_LENGTH);
if (hProcSnap == INVALID_HANDLE_VALUE) {
PyErr_SetString(PyExc_PermissionError, "Unable to create module snapshot. Check permissions or PID.");
return 0;
}
MODULEENTRY32W moduleEntry;
moduleEntry.dwSize = sizeof(moduleEntry);
void* runtime_addr = NULL;
for (BOOL hasModule = Module32FirstW(hProcSnap, &moduleEntry); hasModule; hasModule = Module32NextW(hProcSnap, &moduleEntry)) {
// Look for either python executable or DLL
if (wcsstr(moduleEntry.szModule, substr)) {
runtime_addr = analyze_pe(moduleEntry.szExePath, moduleEntry.modBaseAddr, secname);
if (runtime_addr != NULL) {
break;
}
}
}
CloseHandle(hProcSnap);
return (uintptr_t)runtime_addr;
}
#endif // MS_WINDOWS
// Get the PyRuntime section address for any platform
static uintptr_t
get_py_runtime(proc_handle_t* handle)
{
uintptr_t address = 0;
#ifdef MS_WINDOWS
// On Windows, search for 'python' in executable or DLL
address = search_windows_map_for_section(handle, "PyRuntime", L"python");
if (address == 0) {
// Error out: 'python' substring covers both executable and DLL
PyErr_SetString(PyExc_RuntimeError, "Failed to find the PyRuntime section in the process.");
}
#elif defined(__linux__)
// On Linux, search for 'python' in executable or DLL
address = search_linux_map_for_section(handle, "PyRuntime", "python");
if (address == 0) {
// Error out: 'python' substring covers both executable and DLL
PyErr_SetString(PyExc_RuntimeError, "Failed to find the PyRuntime section in the process.");
}
#else
// On macOS, try libpython first, then fall back to python
address = search_map_for_section(handle, "PyRuntime", "libpython");
if (address == 0) {
// TODO: Differentiate between not found and error
PyErr_Clear();
address = search_map_for_section(handle, "PyRuntime", "python");
}
#endif
return address;
}
// Platform-independent memory read function
static int
read_memory(proc_handle_t *handle, uint64_t remote_address, size_t len, void* dst)
{
#ifdef MS_WINDOWS
SIZE_T read_bytes = 0;
SIZE_T result = 0;
do {
if (!ReadProcessMemory(handle->hProcess, (LPCVOID)(remote_address + result), (char*)dst + result, len - result, &read_bytes)) {
PyErr_SetFromWindowsErr(0);
return -1;
}
result += read_bytes;
} while (result < len);
return 0;
#elif defined(__linux__) && HAVE_PROCESS_VM_READV
struct iovec local[1];
struct iovec remote[1];
Py_ssize_t result = 0;
Py_ssize_t read_bytes = 0;
do {
local[0].iov_base = (char*)dst + result;
local[0].iov_len = len - result;
remote[0].iov_base = (void*)(remote_address + result);
remote[0].iov_len = len - result;
read_bytes = process_vm_readv(handle->pid, local, 1, remote, 1, 0);
if (read_bytes < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
result += read_bytes;
} while ((size_t)read_bytes != local[0].iov_len);
return 0;
#elif defined(__APPLE__) && TARGET_OS_OSX
Py_ssize_t result = -1;
kern_return_t kr = mach_vm_read_overwrite(
pid_to_task(handle->pid),
(mach_vm_address_t)remote_address,
len,
(mach_vm_address_t)dst,
(mach_vm_size_t*)&result);
if (kr != KERN_SUCCESS) {
switch (kr) {
case KERN_PROTECTION_FAILURE:
PyErr_SetString(PyExc_PermissionError, "Not enough permissions to read memory");
break;
case KERN_INVALID_ARGUMENT:
PyErr_SetString(PyExc_PermissionError, "Invalid argument to mach_vm_read_overwrite");
break;
default:
PyErr_SetString(PyExc_RuntimeError, "Unknown error reading memory");
}
return -1;
}
return 0;
#else
Py_UNREACHABLE();
#endif
}
// Platform-independent memory write function
static int
write_memory(proc_handle_t *handle, uintptr_t remote_address, size_t len, const void* src)
{
#ifdef MS_WINDOWS
SIZE_T written = 0;
SIZE_T result = 0;
do {
if (!WriteProcessMemory(handle->hProcess, (LPVOID)(remote_address + result), (const char*)src + result, len - result, &written)) {
PyErr_SetFromWindowsErr(0);
return -1;
}
result += written;
} while (result < len);
return 0;
#elif defined(__linux__) && HAVE_PROCESS_VM_READV
struct iovec local[1];
struct iovec remote[1];
Py_ssize_t result = 0;
Py_ssize_t written = 0;
do {
local[0].iov_base = (void*)((char*)src + result);
local[0].iov_len = len - result;
remote[0].iov_base = (void*)((char*)remote_address + result);
remote[0].iov_len = len - result;
written = process_vm_writev(handle->pid, local, 1, remote, 1, 0);
if (written < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return -1;
}
result += written;
} while ((size_t)written != local[0].iov_len);
return 0;
#elif defined(__APPLE__) && TARGET_OS_OSX
kern_return_t kr = mach_vm_write(
pid_to_task(handle->pid),
(mach_vm_address_t)remote_address,
(vm_offset_t)src,
(mach_msg_type_number_t)len);
if (kr != KERN_SUCCESS) {
switch (kr) {
case KERN_PROTECTION_FAILURE:
PyErr_SetString(PyExc_PermissionError, "Not enough permissions to write memory");
break;
case KERN_INVALID_ARGUMENT:
PyErr_SetString(PyExc_PermissionError, "Invalid argument to mach_vm_write");
break;
default:
PyErr_Format(PyExc_RuntimeError, "Unknown error writing memory: %d", (int)kr);
}
return -1;
}
return 0;
#else
Py_UNREACHABLE();
#endif
}
static int
is_prerelease_version(uint64_t version)
{
return (version & 0xF0) != 0xF0;
}
static int
ensure_debug_offset_compatibility(const _Py_DebugOffsets* debug_offsets)
{
if (memcmp(debug_offsets->cookie, _Py_Debug_Cookie, sizeof(debug_offsets->cookie)) != 0) {
// The remote is probably running a Python version predating debug offsets.
PyErr_SetString(
PyExc_RuntimeError,
"Can't determine the Python version of the remote process");
return -1;
}
// Assume debug offsets could change from one pre-release version to another,
// or one minor version to another, but are stable across patch versions.
if (is_prerelease_version(Py_Version) && Py_Version != debug_offsets->version) {
PyErr_SetString(
PyExc_RuntimeError,
"Can't send commands from a pre-release Python interpreter"
" to a process running a different Python version");
return -1;
}
if (is_prerelease_version(debug_offsets->version) && Py_Version != debug_offsets->version) {
PyErr_SetString(
PyExc_RuntimeError,
"Can't send commands to a pre-release Python interpreter"
" from a process running a different Python version");
return -1;
}
unsigned int remote_major = (debug_offsets->version >> 24) & 0xFF;
unsigned int remote_minor = (debug_offsets->version >> 16) & 0xFF;
if (PY_MAJOR_VERSION != remote_major || PY_MINOR_VERSION != remote_minor) {
PyErr_Format(
PyExc_RuntimeError,
"Can't send commands from a Python %d.%d process to a Python %d.%d process",
PY_MAJOR_VERSION, PY_MINOR_VERSION, remote_major, remote_minor);
return -1;
}
// The debug offsets differ between free threaded and non-free threaded builds.
if (_Py_Debug_Free_Threaded && !debug_offsets->free_threaded) {
PyErr_SetString(
PyExc_RuntimeError,
"Cannot send commands from a free-threaded Python process"
" to a process running a non-free-threaded version");
return -1;
}
if (!_Py_Debug_Free_Threaded && debug_offsets->free_threaded) {
PyErr_SetString(
PyExc_RuntimeError,
"Cannot send commands to a free-threaded Python process"
" from a process running a non-free-threaded version");
return -1;
}
return 0;
}
static int
read_offsets(
proc_handle_t *handle,
uintptr_t *runtime_start_address,
_Py_DebugOffsets* debug_offsets
) {
*runtime_start_address = get_py_runtime(handle);
if (!*runtime_start_address) {
if (!PyErr_Occurred()) {
PyErr_SetString(
PyExc_RuntimeError, "Failed to get PyRuntime address");
}
return -1;
}
size_t size = sizeof(struct _Py_DebugOffsets);
if (0 != read_memory(handle, *runtime_start_address, size, debug_offsets)) {
return -1;
}
if (ensure_debug_offset_compatibility(debug_offsets)) {
return -1;
}
return 0;
}
static int
send_exec_to_proc_handle(proc_handle_t *handle, int tid, const char *debugger_script_path)
{
uintptr_t runtime_start_address;
struct _Py_DebugOffsets debug_offsets;
if (read_offsets(handle, &runtime_start_address, &debug_offsets)) {
return -1;
}
uintptr_t interpreter_state_list_head = (uintptr_t)debug_offsets.runtime_state.interpreters_head;
uintptr_t interpreter_state_addr;
if (0 != read_memory(
handle,
runtime_start_address + interpreter_state_list_head,
sizeof(void*),
&interpreter_state_addr))
{
return -1;
}
if (interpreter_state_addr == 0) {
PyErr_SetString(PyExc_RuntimeError, "Can't find a running interpreter in the remote process");
return -1;
}
int is_remote_debugging_enabled = 0;
if (0 != read_memory(
handle,
interpreter_state_addr + debug_offsets.debugger_support.remote_debugging_enabled,
sizeof(int),
&is_remote_debugging_enabled))
{
return -1;
}
if (is_remote_debugging_enabled != 1) {
PyErr_SetString(
PyExc_RuntimeError,
"Remote debugging is not enabled in the remote process");
return -1;
}
uintptr_t thread_state_addr;
unsigned long this_tid = 0;
if (tid != 0) {
if (0 != read_memory(
handle,
interpreter_state_addr + debug_offsets.interpreter_state.threads_head,
sizeof(void*),
&thread_state_addr))
{
return -1;
}
while (thread_state_addr != 0) {
if (0 != read_memory(
handle,
thread_state_addr + debug_offsets.thread_state.native_thread_id,
sizeof(this_tid),
&this_tid))
{
return -1;
}
if (this_tid == (unsigned long)tid) {
break;
}
if (0 != read_memory(
handle,
thread_state_addr + debug_offsets.thread_state.next,
sizeof(void*),
&thread_state_addr))
{
return -1;
}
}
if (thread_state_addr == 0) {
PyErr_SetString(
PyExc_RuntimeError,
"Can't find the specified thread in the remote process");
return -1;
}
} else {
if (0 != read_memory(
handle,
interpreter_state_addr + debug_offsets.interpreter_state.threads_main,
sizeof(void*),
&thread_state_addr))
{
return -1;
}
if (thread_state_addr == 0) {
PyErr_SetString(
PyExc_RuntimeError,
"Can't find the main thread in the remote process");
return -1;
}
}
// Ensure our path is not too long
if (debug_offsets.debugger_support.debugger_script_path_size <= strlen(debugger_script_path)) {
PyErr_SetString(PyExc_ValueError, "Debugger script path is too long");
return -1;
}
uintptr_t debugger_script_path_addr = (uintptr_t)(
thread_state_addr +
debug_offsets.debugger_support.remote_debugger_support +
debug_offsets.debugger_support.debugger_script_path);
if (0 != write_memory(
handle,
debugger_script_path_addr,
strlen(debugger_script_path) + 1,
debugger_script_path))
{
return -1;
}
int pending_call = 1;
uintptr_t debugger_pending_call_addr = (uintptr_t)(
thread_state_addr +
debug_offsets.debugger_support.remote_debugger_support +
debug_offsets.debugger_support.debugger_pending_call);
if (0 != write_memory(
handle,
debugger_pending_call_addr,
sizeof(int),
&pending_call))
{
return -1;
}
uintptr_t eval_breaker;
if (0 != read_memory(
handle,
thread_state_addr + debug_offsets.debugger_support.eval_breaker,
sizeof(uintptr_t),
&eval_breaker))
{
return -1;
}
eval_breaker |= _PY_EVAL_PLEASE_STOP_BIT;
if (0 != write_memory(
handle,
thread_state_addr + (uintptr_t)debug_offsets.debugger_support.eval_breaker,
sizeof(uintptr_t),
&eval_breaker))
{
return -1;
}
return 0;
}
#endif // defined(Py_REMOTE_DEBUG) && defined(Py_SUPPORTS_REMOTE_DEBUG)
int
_PySysRemoteDebug_SendExec(int pid, int tid, const char *debugger_script_path)
{
#if !defined(Py_SUPPORTS_REMOTE_DEBUG)
PyErr_SetString(PyExc_RuntimeError, "Remote debugging is not supported on this platform");
return -1;
#elif !defined(Py_REMOTE_DEBUG)
PyErr_SetString(PyExc_RuntimeError, "Remote debugging support has not been compiled in");
return -1;
#else
PyThreadState *tstate = _PyThreadState_GET();
const PyConfig *config = _PyInterpreterState_GetConfig(tstate->interp);
if (config->remote_debug != 1) {
PyErr_SetString(PyExc_RuntimeError, "Remote debugging is not enabled");
return -1;
}
proc_handle_t handle;
if (init_proc_handle(&handle, pid) < 0) {
return -1;
}
int rc = send_exec_to_proc_handle(&handle, tid, debugger_script_path);
cleanup_proc_handle(&handle);
return rc;
#endif
}

View file

@ -2421,6 +2421,120 @@ sys_is_stack_trampoline_active_impl(PyObject *module)
Py_RETURN_FALSE;
}
/*[clinic input]
sys.is_remote_debug_enabled
Return True if remote debugging is enabled, False otherwise.
[clinic start generated code]*/
static PyObject *
sys_is_remote_debug_enabled_impl(PyObject *module)
/*[clinic end generated code: output=7ca3d38bdd5935eb input=7335c4a2fe8cf4f3]*/
{
#ifndef Py_REMOTE_DEBUG
Py_RETURN_FALSE;
#else
const PyConfig *config = _Py_GetConfig();
return PyBool_FromLong(config->remote_debug);
#endif
}
static PyObject *
sys_remote_exec_unicode_path(PyObject *module, int pid, PyObject *script)
{
const char *debugger_script_path = PyUnicode_AsUTF8(script);
if (debugger_script_path == NULL) {
return NULL;
}
#ifdef MS_WINDOWS
// Use UTF-16 (wide char) version of the path for permission checks
wchar_t *debugger_script_path_w = PyUnicode_AsWideCharString(script, NULL);
if (debugger_script_path_w == NULL) {
return NULL;
}
// Check file attributes using wide character version (W) instead of ANSI (A)
DWORD attr = GetFileAttributesW(debugger_script_path_w);
PyMem_Free(debugger_script_path_w);
if (attr == INVALID_FILE_ATTRIBUTES) {
DWORD err = GetLastError();
if (err == ERROR_FILE_NOT_FOUND || err == ERROR_PATH_NOT_FOUND) {
PyErr_SetString(PyExc_FileNotFoundError, "Script file does not exist");
}
else if (err == ERROR_ACCESS_DENIED) {
PyErr_SetString(PyExc_PermissionError, "Script file cannot be read");
}
else {
PyErr_SetFromWindowsErr(0);
}
return NULL;
}
#else
if (access(debugger_script_path, F_OK | R_OK) != 0) {
switch (errno) {
case ENOENT:
PyErr_SetString(PyExc_FileNotFoundError, "Script file does not exist");
break;
case EACCES:
PyErr_SetString(PyExc_PermissionError, "Script file cannot be read");
break;
default:
PyErr_SetFromErrno(PyExc_OSError);
}
return NULL;
}
#endif
if (_PySysRemoteDebug_SendExec(pid, 0, debugger_script_path) < 0) {
return NULL;
}
Py_RETURN_NONE;
}
/*[clinic input]
sys.remote_exec
pid: int
script: object
Executes a file containing Python code in a given remote Python process.
This function returns immediately, and the code will be executed by the
target process's main thread at the next available opportunity, similarly
to how signals are handled. There is no interface to determine when the
code has been executed. The caller is responsible for making sure that
the file still exists whenever the remote process tries to read it and that
it hasn't been overwritten.
The remote process must be running a CPython interpreter of the same major
and minor version as the local process. If either the local or remote
interpreter is pre-release (alpha, beta, or release candidate) then the
local and remote interpreters must be the same exact version.
Args:
pid (int): The process ID of the target Python process.
script (str|bytes): The path to a file containing
the Python code to be executed.
[clinic start generated code]*/
static PyObject *
sys_remote_exec_impl(PyObject *module, int pid, PyObject *script)
/*[clinic end generated code: output=7d94c56afe4a52c0 input=39908ca2c5fe1eb0]*/
{
PyObject *ret = NULL;
PyObject *path;
if (PyUnicode_FSDecoder(script, &path)) {
ret = sys_remote_exec_unicode_path(module, pid, path);
Py_DECREF(path);
}
return ret;
}
/*[clinic input]
sys._dump_tracelets
@ -2695,6 +2809,8 @@ static PyMethodDef sys_methods[] = {
SYS_ACTIVATE_STACK_TRAMPOLINE_METHODDEF
SYS_DEACTIVATE_STACK_TRAMPOLINE_METHODDEF
SYS_IS_STACK_TRAMPOLINE_ACTIVE_METHODDEF
SYS_IS_REMOTE_DEBUG_ENABLED_METHODDEF
SYS_REMOTE_EXEC_METHODDEF
SYS_UNRAISABLEHOOK_METHODDEF
SYS_GET_INT_MAX_STR_DIGITS_METHODDEF
SYS_SET_INT_MAX_STR_DIGITS_METHODDEF

30
configure generated vendored
View file

@ -1123,6 +1123,7 @@ with_wheel_pkg_dir
with_readline
with_computed_gotos
with_tail_call_interp
with_remote_debug
with_ensurepip
with_openssl
with_openssl_rpath
@ -1932,6 +1933,7 @@ Optional Packages:
default on supported compilers)
--with-tail-call-interp enable tail-calling interpreter in evaluation loop
and rest of CPython
--with-remote-debug enable remote debugging support (default is yes)
--with-ensurepip[=install|upgrade|no]
"install" or "upgrade" using bundled pip (default is
upgrade)
@ -29302,6 +29304,34 @@ esac
fi
# Check for --with-remote-debug
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for --with-remote-debug" >&5
printf %s "checking for --with-remote-debug... " >&6; }
# Check whether --with-remote-debug was given.
if test ${with_remote_debug+y}
then :
withval=$with_remote_debug;
else case e in #(
e) with_remote_debug=yes ;;
esac
fi
if test "$with_remote_debug" = yes; then
printf "%s\n" "#define Py_REMOTE_DEBUG 1" >>confdefs.h
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: yes" >&5
printf "%s\n" "yes" >&6; }
else
printf "%s\n" "#define Py_REMOTE_DEBUG 0" >>confdefs.h
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: no" >&5
printf "%s\n" "no" >&6; }
fi
case $ac_sys_system in
AIX*)

View file

@ -7034,6 +7034,26 @@ fi
],
[AC_MSG_RESULT([no value specified])])
# Check for --with-remote-debug
AC_MSG_CHECKING([for --with-remote-debug])
AC_ARG_WITH(
[remote-debug],
[AS_HELP_STRING(
[--with-remote-debug],
[enable remote debugging support (default is yes)])],
[],
[with_remote_debug=yes])
if test "$with_remote_debug" = yes; then
AC_DEFINE([Py_REMOTE_DEBUG], [1],
[Define if you want to enable remote debugging support.])
AC_MSG_RESULT([yes])
else
AC_DEFINE([Py_REMOTE_DEBUG], [0],
[Define if you want to enable remote debugging support.])
AC_MSG_RESULT([no])
fi
case $ac_sys_system in
AIX*)

View file

@ -1718,6 +1718,9 @@
/* Define if year with century should be normalized for strftime. */
#undef Py_NORMALIZE_CENTURY
/* Define if you want to enable remote debugging support. */
#undef Py_REMOTE_DEBUG
/* Define if rl_startup_hook takes arguments */
#undef Py_RL_STARTUP_HOOK_TAKES_ARGS