gh-108724: Add PyMutex and _PyParkingLot APIs (gh-109344)

PyMutex is a one byte lock with fast, inlineable lock and unlock functions for the common uncontended case.  The design is based on WebKit's WTF::Lock.

PyMutex is built using the _PyParkingLot APIs, which provides a cross-platform futex-like API (based on WebKit's WTF::ParkingLot).  This internal API will be used for building other synchronization primitives used to implement PEP 703, such as one-time initialization and events.

This also includes tests and a mini benchmark in Tools/lockbench/lockbench.py to compare with the existing PyThread_type_lock.

Uncontended acquisition + release:
* Linux (x86-64): PyMutex: 11 ns, PyThread_type_lock: 44 ns
* macOS (arm64): PyMutex: 13 ns, PyThread_type_lock: 18 ns
* Windows (x86-64): PyMutex: 13 ns, PyThread_type_lock: 38 ns

PR Overview:

The primary purpose of this PR is to implement PyMutex, but there are a number of support pieces (described below).

* PyMutex:  A 1-byte lock that doesn't require memory allocation to initialize and is generally faster than the existing PyThread_type_lock.  The API is internal only for now.
* _PyParking_Lot:  A futex-like API based on the API of the same name in WebKit.  Used to implement PyMutex.
* _PyRawMutex:  A word sized lock used to implement _PyParking_Lot.
* PyEvent:  A one time event.  This was used a bunch in the "nogil" fork and is useful for testing the PyMutex implementation, so I've included it as part of the PR.
* pycore_llist.h:  Defines common operations on doubly-linked list.  Not strictly necessary (could do the list operations manually), but they come up frequently in the "nogil" fork. ( Similar to https://man.freebsd.org/cgi/man.cgi?queue)

---------

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
This commit is contained in:
Sam Gross 2023-09-19 11:54:29 -04:00 committed by GitHub
parent 0a31ff0050
commit 0c89056fe5
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
29 changed files with 1665 additions and 21 deletions

View file

@ -95,6 +95,7 @@
<ItemGroup>
<ClCompile Include="..\Modules\_testinternalcapi.c" />
<ClCompile Include="..\Modules\_testinternalcapi\pytime.c" />
<ClCompile Include="..\Modules\_testinternalcapi\test_lock.c" />
</ItemGroup>
<ItemGroup>
<ResourceCompile Include="..\PC\python_nt.rc" />

View file

@ -15,6 +15,9 @@
<ClCompile Include="..\Modules\_testinternalcapi\pytime.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\Modules\_testinternalcapi\test_lock.c">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ResourceCompile Include="..\PC\python_nt.rc">

View file

@ -245,6 +245,8 @@
<ClInclude Include="..\Include\internal\pycore_interp.h" />
<ClInclude Include="..\Include\internal\pycore_intrinsics.h" />
<ClInclude Include="..\Include\internal\pycore_list.h" />
<ClInclude Include="..\Include\internal\pycore_llist.h" />
<ClInclude Include="..\Include\internal\pycore_lock.h" />
<ClInclude Include="..\Include\internal\pycore_long.h" />
<ClInclude Include="..\Include\internal\pycore_modsupport.h" />
<ClInclude Include="..\Include\internal\pycore_moduleobject.h" />
@ -254,6 +256,7 @@
<ClInclude Include="..\Include\internal\pycore_obmalloc.h" />
<ClInclude Include="..\Include\internal\pycore_obmalloc_init.h" />
<ClInclude Include="..\Include\internal\pycore_optimizer.h" />
<ClInclude Include="..\Include\internal\pycore_parking_lot.h" />
<ClInclude Include="..\Include\internal\pycore_pathconfig.h" />
<ClInclude Include="..\Include\internal\pycore_pyarena.h" />
<ClInclude Include="..\Include\internal\pycore_pyerrors.h" />
@ -269,6 +272,7 @@
<ClInclude Include="..\Include\internal\pycore_runtime.h" />
<ClInclude Include="..\Include\internal\pycore_runtime_init.h" />
<ClInclude Include="..\Include\internal\pycore_runtime_init_generated.h" />
<ClInclude Include="..\Include\internal\pycore_semaphore.h" />
<ClInclude Include="..\Include\internal\pycore_setobject.h" />
<ClInclude Include="..\Include\internal\pycore_signal.h" />
<ClInclude Include="..\Include\internal\pycore_sliceobject.h" />
@ -307,6 +311,7 @@
<ClInclude Include="..\Include\osmodule.h" />
<ClInclude Include="..\Include\patchlevel.h" />
<ClInclude Include="..\Include\py_curses.h" />
<ClInclude Include="..\Include\pyatomic.h" />
<ClInclude Include="..\Include\pybuffer.h" />
<ClInclude Include="..\Include\pycapsule.h" />
<ClInclude Include="..\Include\pyerrors.h" />
@ -552,12 +557,14 @@
<ClCompile Include="..\Python\intrinsics.c" />
<ClCompile Include="..\Python\instrumentation.c" />
<ClCompile Include="..\Python\legacy_tracing.c" />
<ClCompile Include="..\Python\lock.c" />
<ClCompile Include="..\Python\marshal.c" />
<ClCompile Include="..\Python\modsupport.c" />
<ClCompile Include="..\Python\mysnprintf.c" />
<ClCompile Include="..\Python\mystrtoul.c" />
<ClCompile Include="..\Python\optimizer.c" />
<ClCompile Include="..\Python\optimizer_analysis.c" />
<ClCompile Include="..\Python\parking_lot.c" />
<ClCompile Include="..\Python\pathconfig.c" />
<ClCompile Include="..\Python\perf_trampoline.c" />
<ClCompile Include="..\Python\preconfig.c" />

View file

@ -144,6 +144,9 @@
<ClInclude Include="..\Include\py_curses.h">
<Filter>Include</Filter>
</ClInclude>
<ClInclude Include="..\Include\pyatomic.h">
<Filter>Include</Filter>
</ClInclude>
<ClInclude Include="..\Include\pybuffer.h">
<Filter>Include</Filter>
</ClInclude>
@ -645,6 +648,12 @@
<ClInclude Include="..\Include\internal\pycore_list.h">
<Filter>Include\internal</Filter>
</ClInclude>
<ClInclude Include="..\Include\internal\pycore_llist.h">
<Filter>Include\internal</Filter>
</ClInclude>
<ClInclude Include="..\Include\internal\pycore_lock.h">
<Filter>Include\internal</Filter>
</ClInclude>
<ClInclude Include="..\Include\internal\pycore_long.h">
<Filter>Include\internal</Filter>
</ClInclude>
@ -672,6 +681,9 @@
<ClInclude Include="..\Include\internal\pycore_optimizer.h">
<Filter>Include\internal</Filter>
</ClInclude>
<ClInclude Include="..\Include\internal\pycore_parking_lot.h">
<Filter>Include\internal</Filter>
</ClInclude>
<ClInclude Include="..\Include\internal\pycore_pathconfig.h">
<Filter>Include\internal</Filter>
</ClInclude>
@ -717,6 +729,9 @@
<ClInclude Include="..\Include\internal\pycore_runtime_init_generated.h">
<Filter>Include\internal</Filter>
</ClInclude>
<ClInclude Include="..\Include\internal\pycore_semaphore.h">
<Filter>Include\internal</Filter>
</ClInclude>
<ClInclude Include="..\Include\internal\pycore_setobject.h">
<Filter>Include\internal</Filter>
</ClInclude>
@ -1241,6 +1256,9 @@
<ClCompile Include="..\Python\legacy_tracing.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\Python\lock.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\Python\marshal.c">
<Filter>Python</Filter>
</ClCompile>
@ -1259,6 +1277,9 @@
<ClCompile Include="..\Python\optimizer_analysis.c">
<Filter>Python</Filter>
</ClCompile>
<ClCompile Include="..\Python\parking_lot.c">
<Filter>Python</Filter>
</ClCompile>
<ClCompile Include="..\Python\pathconfig.c">
<Filter>Python</Filter>
</ClCompile>