mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-08-01 23:53:15 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

gh-97781: Apply changes from importlib_metadata 5. (GH-97785)

Browse source 
* gh-97781: Apply changes from importlib_metadata 5.

* Apply changes from upstream

* Apply changes from upstream.
This commit is contained in:
Jason R. Coombs 2022-10-06 15:25:24 -04:00 • committed by GitHub
parent 2b5f1360ea
commit 8af04cdef2
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
5 changed files with 88 additions and 302 deletions
Download patch file Download diff file Expand all files Collapse all files

108
Doc/library/importlib.metadata.rst
View file

@ -13,21 +13,39 @@
**Source code:** :source:`Lib/importlib/metadata/__init__.py` **Source code:** :source:`Lib/importlib/metadata/__init__.py`
``importlib.metadata`` is a library that provides access to installed ``importlib_metadata`` is a library that provides access to
package metadata, such as its entry points or its the metadata of an installed `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_,
top-level name. Built in part on Python's import system, this library such as its entry points
or its top-level names (`Import Package <https://packaging.python.org/en/latest/glossary/#term-Import-Package>`_\s, modules, if any).
Built in part on Python's import system, this library
intends to replace similar functionality in the `entry point intends to replace similar functionality in the `entry point
API`_ and `metadata API`_ of ``pkg_resources``. Along with API`_ and `metadata API`_ of ``pkg_resources``. Along with
:mod:`importlib.resources`, :mod:`importlib.resources`,
this package can eliminate the need to use the older and less efficient this package can eliminate the need to use the older and less efficient
``pkg_resources`` package. ``pkg_resources`` package.
By "installed package" we generally mean a third-party package installed into ``importlib_metadata`` operates on third-party *distribution packages*
Python's ``site-packages`` directory via tools such as `pip installed into Python's ``site-packages`` directory via tools such as
<https://pypi.org/project/pip/>`_. Specifically, `pip <https://pypi.org/project/pip/>`_.
it means a package with either a discoverable ``dist-info`` or ``egg-info`` Specifically, it works with distributions with discoverable
directory, and metadata defined by :pep:`566` or its older specifications. ``dist-info`` or ``egg-info`` directories,
By default, package metadata can live on the file system or in zip archives on and metadata defined by the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_.
.. important::
These are *not* necessarily equivalent to or correspond 1:1 with
the top-level *import package* names
that can be imported inside Python code.
One *distribution package* can contain multiple *import packages*
(and single modules),
and one top-level *import package*
may map to multiple *distribution packages*
if it is a namespace package.
You can use :ref:`package_distributions() <package-distributions>`
to get a mapping between them.
By default, distribution metadata can live on the file system
or in zip archives on
:data:`sys.path`. Through an extension mechanism, the metadata can live almost :data:`sys.path`. Through an extension mechanism, the metadata can live almost
anywhere. anywhere.
@ -37,12 +55,19 @@ anywhere.
https://importlib-metadata.readthedocs.io/ https://importlib-metadata.readthedocs.io/
The documentation for ``importlib_metadata``, which supplies a The documentation for ``importlib_metadata``, which supplies a
backport of ``importlib.metadata``. backport of ``importlib.metadata``.
This includes an `API reference
<https://importlib-metadata.readthedocs.io/en/latest/api.html>`__
for this module's classes and functions,
as well as a `migration guide
<https://importlib-metadata.readthedocs.io/en/latest/migration.html>`__
for existing users of ``pkg_resources``.
Overview Overview
======== ========
Let's say you wanted to get the version string for a package you've installed Let's say you wanted to get the version string for a
`Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ you've installed
using ``pip``. We start by creating a virtual environment and installing using ``pip``. We start by creating a virtual environment and installing
something into it: something into it:
@ -151,11 +176,10 @@ for more information on entry points, their definition, and usage.
The "selectable" entry points were introduced in ``importlib_metadata`` The "selectable" entry points were introduced in ``importlib_metadata``
3.6 and Python 3.10. Prior to those changes, ``entry_points`` accepted 3.6 and Python 3.10. Prior to those changes, ``entry_points`` accepted
no parameters and always returned a dictionary of entry points, keyed no parameters and always returned a dictionary of entry points, keyed
by group. For compatibility, if no parameters are passed to entry_points, by group. With ``importlib_metadata`` 5.0 and Python 3.12,
a ``SelectableGroups`` object is returned, implementing that dict ``entry_points`` always returns an ``EntryPoints`` object. See
interface. In the future, calling ``entry_points`` with no parameters `backports.entry_points_selectable <https://pypi.org/project/backports.entry_points_selectable>`_
will return an ``EntryPoints`` object. Users should rely on the selection for compatibility options.
interface to retrieve entry points by group.
.. _metadata: .. _metadata:
@ -163,7 +187,8 @@ interface to retrieve entry points by group.
Distribution metadata Distribution metadata
--------------------- ---------------------
Every distribution includes some metadata, which you can extract using the Every `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ includes some metadata,
which you can extract using the
``metadata()`` function:: ``metadata()`` function::
>>> wheel_metadata = metadata('wheel') # doctest: +SKIP >>> wheel_metadata = metadata('wheel') # doctest: +SKIP
@ -201,7 +226,8 @@ all the metadata in a JSON-compatible form per :PEP:`566`::
Distribution versions Distribution versions
--------------------- ---------------------
The ``version()`` function is the quickest way to get a distribution's version The ``version()`` function is the quickest way to get a
`Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_'s version
number, as a string:: number, as a string::
>>> version('wheel') # doctest: +SKIP >>> version('wheel') # doctest: +SKIP
@ -214,7 +240,8 @@ Distribution files
------------------ ------------------
You can also get the full set of files contained within a distribution. The You can also get the full set of files contained within a distribution. The
``files()`` function takes a distribution package name and returns all of the ``files()`` function takes a `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ name
and returns all of the
files installed by this distribution. Each file object returned is a files installed by this distribution. Each file object returned is a
``PackagePath``, a :class:`pathlib.PurePath` derived object with additional ``dist``, ``PackagePath``, a :class:`pathlib.PurePath` derived object with additional ``dist``,
``size``, and ``hash`` properties as indicated by the metadata. For example:: ``size``, and ``hash`` properties as indicated by the metadata. For example::
@ -259,19 +286,24 @@ distribution is not known to have the metadata present.
Distribution requirements Distribution requirements
------------------------- -------------------------
To get the full set of requirements for a distribution, use the ``requires()`` To get the full set of requirements for a `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_,
use the ``requires()``
function:: function::
>>> requires('wheel') # doctest: +SKIP >>> requires('wheel') # doctest: +SKIP
["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"] ["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"]
Package distributions .. _package-distributions:
--------------------- .. _import-distribution-package-mapping:
A convenience method to resolve the distribution or Mapping import to distribution packages
distributions (in the case of a namespace package) for top-level ---------------------------------------
Python packages or modules::
A convenience method to resolve the `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_
name (or names, in the case of a namespace package)
that provide each importable top-level
Python module or `Import Package <https://packaging.python.org/en/latest/glossary/#term-Import-Package>`_::
>>> packages_distributions() >>> packages_distributions()
{'importlib_metadata': ['importlib-metadata'], 'yaml': ['PyYAML'], 'jaraco': ['jaraco.classes', 'jaraco.functools'], ...} {'importlib_metadata': ['importlib-metadata'], 'yaml': ['PyYAML'], 'jaraco': ['jaraco.classes', 'jaraco.functools'], ...}
@ -285,7 +317,8 @@ Distributions
While the above API is the most common and convenient usage, you can get all While the above API is the most common and convenient usage, you can get all
of that information from the ``Distribution`` class. A ``Distribution`` is an of that information from the ``Distribution`` class. A ``Distribution`` is an
abstract object that represents the metadata for a Python package. You can abstract object that represents the metadata for
a Python `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_. You can
get the ``Distribution`` instance:: get the ``Distribution`` instance::
>>> from importlib.metadata import distribution # doctest: +SKIP >>> from importlib.metadata import distribution # doctest: +SKIP
@ -305,14 +338,16 @@ instance::
>>> dist.metadata['License'] # doctest: +SKIP >>> dist.metadata['License'] # doctest: +SKIP
'MIT' 'MIT'
The full set of available metadata is not described here. See :pep:`566` The full set of available metadata is not described here.
for additional details. See the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_ for additional details.
Distribution Discovery Distribution Discovery
====================== ======================
By default, this package provides built-in support for discovery of metadata for file system and zip file packages. This metadata finder search defaults to ``sys.path``, but varies slightly in how it interprets those values from how other import machinery does. In particular: By default, this package provides built-in support for discovery of metadata
for file system and zip file `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_\s.
This metadata finder search defaults to ``sys.path``, but varies slightly in how it interprets those values from how other import machinery does. In particular:
- ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``. - ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``.
- ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports. - ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports.
@ -321,15 +356,18 @@ By default, this package provides built-in support for discovery of metadata for
Extending the search algorithm Extending the search algorithm
============================== ==============================
Because package metadata is not available through :data:`sys.path` searches, or Because `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ metadata
package loaders directly, the metadata for a package is found through import is not available through :data:`sys.path` searches, or
system :ref:`finders <finders-and-loaders>`. To find a distribution package's metadata, package loaders directly,
the metadata for a distribution is found through import
system `finders`_. To find a distribution package's metadata,
``importlib.metadata`` queries the list of :term:`meta path finders <meta path finder>` on ``importlib.metadata`` queries the list of :term:`meta path finders <meta path finder>` on
:data:`sys.meta_path`. :data:`sys.meta_path`.
The default ``PathFinder`` for Python includes a hook that calls into By default ``importlib_metadata`` installs a finder for distribution packages
``importlib.metadata.MetadataPathFinder`` for finding distributions found on the file system.
loaded from typical file-system-based paths. This finder doesn't actually find any *distributions*,
but it can find their metadata.
The abstract class :py:class:`importlib.abc.MetaPathFinder` defines the The abstract class :py:class:`importlib.abc.MetaPathFinder` defines the
interface expected of finders by Python's import system. interface expected of finders by Python's import system.
@ -358,4 +396,4 @@ a custom finder, return instances of this derived ``Distribution`` in the
.. _`entry point API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points .. _`entry point API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
.. _`metadata API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api .. _`metadata API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api
.. _`importlib_resources`: https://importlib-resources.readthedocs.io/en/latest/index.html .. _`finders`: https://docs.python.org/3/reference/import.html#finders-and-loaders

211
Lib/importlib/metadata/__init__.py
View file

@ -24,7 +24,7 @@ from contextlib import suppress
from importlib import import_module from importlib import import_module
from importlib.abc import MetaPathFinder from importlib.abc import MetaPathFinder
from itertools import starmap from itertools import starmap
from typing import List, Mapping, Optional, Union from typing import List, Mapping, Optional
__all__ = [ __all__ = [
@ -134,6 +134,7 @@ class DeprecatedTuple:
1 1
""" """
# Do not remove prior to 2023-05-01 or Python 3.13
_warn = functools.partial( _warn = functools.partial(
warnings.warn, warnings.warn,
"EntryPoint tuple interface is deprecated. Access members by name.", "EntryPoint tuple interface is deprecated. Access members by name.",
@ -184,6 +185,10 @@ class EntryPoint(DeprecatedTuple):
following the attr, and following any extras. following the attr, and following any extras.
""" """
name: str
value: str
group: str
dist: Optional['Distribution'] = None dist: Optional['Distribution'] = None
def __init__(self, name, value, group): def __init__(self, name, value, group):
@ -218,17 +223,6 @@ class EntryPoint(DeprecatedTuple):
vars(self).update(dist=dist) vars(self).update(dist=dist)
return self return self
def __iter__(self):
"""
Supply iter so one may construct dicts of EntryPoints by name.
"""
msg = (
"Construction of dict of EntryPoints is deprecated in "
"favor of EntryPoints."
)
warnings.warn(msg, DeprecationWarning)
return iter((self.name, self))
def matches(self, **params): def matches(self, **params):
""" """
EntryPoint matches the given parameters. EntryPoint matches the given parameters.
@ -274,77 +268,7 @@ class EntryPoint(DeprecatedTuple):
return hash(self._key()) return hash(self._key())
class DeprecatedList(list): class EntryPoints(tuple):
"""
Allow an otherwise immutable object to implement mutability
for compatibility.
>>> recwarn = getfixture('recwarn')
>>> dl = DeprecatedList(range(3))
>>> dl[0] = 1
>>> dl.append(3)
>>> del dl[3]
>>> dl.reverse()
>>> dl.sort()
>>> dl.extend([4])
>>> dl.pop(-1)
4
>>> dl.remove(1)
>>> dl += [5]
>>> dl + [6]
[1, 2, 5, 6]
>>> dl + (6,)
[1, 2, 5, 6]
>>> dl.insert(0, 0)
>>> dl
[0, 1, 2, 5]
>>> dl == [0, 1, 2, 5]
True
>>> dl == (0, 1, 2, 5)
True
>>> len(recwarn)
1
"""
__slots__ = ()
_warn = functools.partial(
warnings.warn,
"EntryPoints list interface is deprecated. Cast to list if needed.",
DeprecationWarning,
stacklevel=2,
)
def _wrap_deprecated_method(method_name: str): # type: ignore
def wrapped(self, *args, **kwargs):
self._warn()
return getattr(super(), method_name)(*args, **kwargs)
return method_name, wrapped
locals().update(
map(
_wrap_deprecated_method,
'__setitem__ __delitem__ append reverse extend pop remove '
'__iadd__ insert sort'.split(),
)
)
def __add__(self, other):
if not isinstance(other, tuple):
self._warn()
other = tuple(other)
return self.__class__(tuple(self) + other)
def __eq__(self, other):
if not isinstance(other, tuple):
self._warn()
other = tuple(other)
return tuple(self).__eq__(other)
class EntryPoints(DeprecatedList):
""" """
An immutable collection of selectable EntryPoint objects. An immutable collection of selectable EntryPoint objects.
""" """
@ -355,14 +279,6 @@ class EntryPoints(DeprecatedList):
""" """
Get the EntryPoint in self matching name. Get the EntryPoint in self matching name.
""" """
if isinstance(name, int):
warnings.warn(
"Accessing entry points by index is deprecated. "
"Cast to tuple if needed.",
DeprecationWarning,
stacklevel=2,
)
return super().__getitem__(name)
try: try:
return next(iter(self.select(name=name))) return next(iter(self.select(name=name)))
except StopIteration: except StopIteration:
@ -386,10 +302,6 @@ class EntryPoints(DeprecatedList):
def groups(self): def groups(self):
""" """
Return the set of all groups of all entry points. Return the set of all groups of all entry points.
For coverage while SelectableGroups is present.
>>> EntryPoints().groups
set()
""" """
return {ep.group for ep in self} return {ep.group for ep in self}
@ -405,101 +317,6 @@ class EntryPoints(DeprecatedList):
) )
class Deprecated:
"""
Compatibility add-in for mapping to indicate that
mapping behavior is deprecated.
>>> recwarn = getfixture('recwarn')
>>> class DeprecatedDict(Deprecated, dict): pass
>>> dd = DeprecatedDict(foo='bar')
>>> dd.get('baz', None)
>>> dd['foo']
'bar'
>>> list(dd)
['foo']
>>> list(dd.keys())
['foo']
>>> 'foo' in dd
True
>>> list(dd.values())
['bar']
>>> len(recwarn)
1
"""
_warn = functools.partial(
warnings.warn,
"SelectableGroups dict interface is deprecated. Use select.",
DeprecationWarning,
stacklevel=2,
)
def __getitem__(self, name):
self._warn()
return super().__getitem__(name)
def get(self, name, default=None):
self._warn()
return super().get(name, default)
def __iter__(self):
self._warn()
return super().__iter__()
def __contains__(self, *args):
self._warn()
return super().__contains__(*args)
def keys(self):
self._warn()
return super().keys()
def values(self):
self._warn()
return super().values()
class SelectableGroups(Deprecated, dict):
"""
A backward- and forward-compatible result from
entry_points that fully implements the dict interface.
"""
@classmethod
def load(cls, eps):
by_group = operator.attrgetter('group')
ordered = sorted(eps, key=by_group)
grouped = itertools.groupby(ordered, by_group)
return cls((group, EntryPoints(eps)) for group, eps in grouped)
@property
def _all(self):
"""
Reconstruct a list of all entrypoints from the groups.
"""
groups = super(Deprecated, self).values()
return EntryPoints(itertools.chain.from_iterable(groups))
@property
def groups(self):
return self._all.groups
@property
def names(self):
"""
for coverage:
>>> SelectableGroups().names
set()
"""
return self._all.names
def select(self, **params):
if not params:
return self
return self._all.select(**params)
class PackagePath(pathlib.PurePosixPath): class PackagePath(pathlib.PurePosixPath):
"""A reference to a path in a package""" """A reference to a path in a package"""
@ -1013,27 +830,19 @@ Wrapper for ``distributions`` to return unique distributions by name.
""" """
def entry_points(**params) -> Union[EntryPoints, SelectableGroups]: def entry_points(**params) -> EntryPoints:
"""Return EntryPoint objects for all installed packages. """Return EntryPoint objects for all installed packages.
Pass selection parameters (group or name) to filter the Pass selection parameters (group or name) to filter the
result to entry points matching those properties (see result to entry points matching those properties (see
EntryPoints.select()). EntryPoints.select()).
For compatibility, returns ``SelectableGroups`` object unless :return: EntryPoints for all installed packages.
selection parameters are supplied. In the future, this function
will return ``EntryPoints`` instead of ``SelectableGroups``
even when no selection parameters are supplied.
For maximum future compatibility, pass selection parameters
or invoke ``.select`` with parameters on the result.
:return: EntryPoints or SelectableGroups for all installed packages.
""" """
eps = itertools.chain.from_iterable( eps = itertools.chain.from_iterable(
dist.entry_points for dist in _unique(distributions()) dist.entry_points for dist in _unique(distributions())
) )
return SelectableGroups.load(eps).select(**params) return EntryPoints(eps).select(**params)
def files(distribution_name): def files(distribution_name):

10
Lib/test/test_importlib/test_main.py
View file

@ -1,8 +1,6 @@
import re import re
import json
import pickle import pickle
import unittest import unittest
import warnings
import importlib.metadata import importlib.metadata
try: try:
@ -260,14 +258,6 @@ class TestEntryPoints(unittest.TestCase):
"""EntryPoints should be hashable""" """EntryPoints should be hashable"""
hash(self.ep) hash(self.ep)
def test_json_dump(self):
"""
json should not expect to be able to dump an EntryPoint
"""
with self.assertRaises(Exception):
with warnings.catch_warnings(record=True):
json.dumps(self.ep)
def test_module(self): def test_module(self):
assert self.ep.module == 'value' assert self.ep.module == 'value'

56
Lib/test/test_importlib/test_metadata_api.py
View file

@ -124,62 +124,6 @@ class APITests(
def test_entry_points_missing_group(self): def test_entry_points_missing_group(self):
assert entry_points(group='missing') == () assert entry_points(group='missing') == ()
def test_entry_points_dict_construction(self):
"""
Prior versions of entry_points() returned simple lists and
allowed casting those lists into maps by name using ``dict()``.
Capture this now deprecated use-case.
"""
with suppress_known_deprecation() as caught:
eps = dict(entry_points(group='entries'))
assert 'main' in eps
assert eps['main'] == entry_points(group='entries')['main']
# check warning
expected = next(iter(caught))
assert expected.category is DeprecationWarning
assert "Construction of dict of EntryPoints is deprecated" in str(expected)
def test_entry_points_by_index(self):
"""
Prior versions of Distribution.entry_points would return a
tuple that allowed access by index.
Capture this now deprecated use-case
See python/importlib_metadata#300 and bpo-44246.
"""
eps = distribution('distinfo-pkg').entry_points
with suppress_known_deprecation() as caught:
eps[0]
# check warning
expected = next(iter(caught))
assert expected.category is DeprecationWarning
assert "Accessing entry points by index is deprecated" in str(expected)
def test_entry_points_groups_getitem(self):
"""
Prior versions of entry_points() returned a dict. Ensure
that callers using '.__getitem__()' are supported but warned to
migrate.
"""
with suppress_known_deprecation():
entry_points()['entries'] == entry_points(group='entries')
with self.assertRaises(KeyError):
entry_points()['missing']
def test_entry_points_groups_get(self):
"""
Prior versions of entry_points() returned a dict. Ensure
that callers using '.get()' are supported but warned to
migrate.
"""
with suppress_known_deprecation():
entry_points().get('missing', 'default') == 'default'
entry_points().get('entries', 'default') == entry_points()['entries']
entry_points().get('missing', ()) == ()
def test_entry_points_allows_no_attributes(self): def test_entry_points_allows_no_attributes(self):
ep = entry_points().select(group='entries', name='main') ep = entry_points().select(group='entries', name='main')
with self.assertRaises(AttributeError): with self.assertRaises(AttributeError):

5
Misc/NEWS.d/next/Library/2022-10-03-13-25-19.gh-issue-97781.gCLLef.rst Normal file
View file

@ -0,0 +1,5 @@
Removed deprecated interfaces in ``importlib.metadata`` (entry points
accessed as dictionary, implicit dictionary construction of sequence of
``EntryPoint`` objects, mutablility of ``EntryPoints`` result, access of
entry point by index). ``entry_points`` now has a simpler, more
straightforward API (returning ``EntryPoints``).