gh-104773: PEP 594: Remove the xdrlib module (#104900)

pickle documentation no longer mentions the XDR format.
This commit is contained in:
Victor Stinner 2023-05-25 00:40:30 +02:00 committed by GitHub
parent 17e1fe0f9b
commit 087c1a6539
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
13 changed files with 8 additions and 615 deletions

View file

@ -260,10 +260,6 @@ Examples::
Module :mod:`struct`
Packing and unpacking of heterogeneous binary data.
Module :mod:`xdrlib`
Packing and unpacking of External Data Representation (XDR) data as used in some
remote procedure call systems.
`NumPy <https://numpy.org/>`_
The NumPy package defines another array type.

View file

@ -125,7 +125,7 @@ Data stream format
The data format used by :mod:`pickle` is Python-specific. This has the
advantage that there are no restrictions imposed by external standards such as
JSON or XDR (which can't represent pointer sharing); however it means that
JSON (which can't represent pointer sharing); however it means that
non-Python programs may not be able to reconstruct pickled Python objects.
By default, the :mod:`pickle` data format uses a relatively compact binary

View file

@ -18,4 +18,3 @@ backwards compatibility. They have been superseded by other modules.
msilib.rst
optparse.rst
uu.rst
xdrlib.rst

View file

@ -1,283 +0,0 @@
:mod:`xdrlib` --- Encode and decode XDR data
============================================
.. module:: xdrlib
:synopsis: Encoders and decoders for the External Data Representation (XDR).
:deprecated:
**Source code:** :source:`Lib/xdrlib.py`
.. index::
single: XDR
single: External Data Representation
.. deprecated-removed:: 3.11 3.13
The :mod:`xdrlib` module is deprecated
(see :pep:`PEP 594 <594#xdrlib>` for details).
--------------
The :mod:`xdrlib` module supports the External Data Representation Standard as
described in :rfc:`1014`, written by Sun Microsystems, Inc. June 1987. It
supports most of the data types described in the RFC.
The :mod:`xdrlib` module defines two classes, one for packing variables into XDR
representation, and another for unpacking from XDR representation. There are
also two exception classes.
.. class:: Packer()
:class:`Packer` is the class for packing data into XDR representation. The
:class:`Packer` class is instantiated with no arguments.
.. class:: Unpacker(data)
``Unpacker`` is the complementary class which unpacks XDR data values from a
string buffer. The input buffer is given as *data*.
.. seealso::
:rfc:`1014` - XDR: External Data Representation Standard
This RFC defined the encoding of data which was XDR at the time this module was
originally written. It has apparently been obsoleted by :rfc:`1832`.
:rfc:`1832` - XDR: External Data Representation Standard
Newer RFC that provides a revised definition of XDR.
.. _xdr-packer-objects:
Packer Objects
--------------
:class:`Packer` instances have the following methods:
.. method:: Packer.get_buffer()
Returns the current pack buffer as a string.
.. method:: Packer.reset()
Resets the pack buffer to the empty string.
In general, you can pack any of the most common XDR data types by calling the
appropriate ``pack_type()`` method. Each method takes a single argument, the
value to pack. The following simple data type packing methods are supported:
:meth:`pack_uint`, :meth:`pack_int`, :meth:`pack_enum`, :meth:`pack_bool`,
:meth:`pack_uhyper`, and :meth:`pack_hyper`.
.. method:: Packer.pack_float(value)
Packs the single-precision floating point number *value*.
.. method:: Packer.pack_double(value)
Packs the double-precision floating point number *value*.
The following methods support packing strings, bytes, and opaque data:
.. method:: Packer.pack_fstring(n, s)
Packs a fixed length string, *s*. *n* is the length of the string but it is
*not* packed into the data buffer. The string is padded with null bytes if
necessary to guaranteed 4 byte alignment.
.. method:: Packer.pack_fopaque(n, data)
Packs a fixed length opaque data stream, similarly to :meth:`pack_fstring`.
.. method:: Packer.pack_string(s)
Packs a variable length string, *s*. The length of the string is first packed
as an unsigned integer, then the string data is packed with
:meth:`pack_fstring`.
.. method:: Packer.pack_opaque(data)
Packs a variable length opaque data string, similarly to :meth:`pack_string`.
.. method:: Packer.pack_bytes(bytes)
Packs a variable length byte stream, similarly to :meth:`pack_string`.
The following methods support packing arrays and lists:
.. method:: Packer.pack_list(list, pack_item)
Packs a *list* of homogeneous items. This method is useful for lists with an
indeterminate size; i.e. the size is not available until the entire list has
been walked. For each item in the list, an unsigned integer ``1`` is packed
first, followed by the data value from the list. *pack_item* is the function
that is called to pack the individual item. At the end of the list, an unsigned
integer ``0`` is packed.
For example, to pack a list of integers, the code might appear like this::
import xdrlib
p = xdrlib.Packer()
p.pack_list([1, 2, 3], p.pack_int)
.. method:: Packer.pack_farray(n, array, pack_item)
Packs a fixed length list (*array*) of homogeneous items. *n* is the length of
the list; it is *not* packed into the buffer, but a :exc:`ValueError` exception
is raised if ``len(array)`` is not equal to *n*. As above, *pack_item* is the
function used to pack each element.
.. method:: Packer.pack_array(list, pack_item)
Packs a variable length *list* of homogeneous items. First, the length of the
list is packed as an unsigned integer, then each element is packed as in
:meth:`pack_farray` above.
.. _xdr-unpacker-objects:
Unpacker Objects
----------------
The :class:`Unpacker` class offers the following methods:
.. method:: Unpacker.reset(data)
Resets the string buffer with the given *data*.
.. method:: Unpacker.get_position()
Returns the current unpack position in the data buffer.
.. method:: Unpacker.set_position(position)
Sets the data buffer unpack position to *position*. You should be careful about
using :meth:`get_position` and :meth:`set_position`.
.. method:: Unpacker.get_buffer()
Returns the current unpack data buffer as a string.
.. method:: Unpacker.done()
Indicates unpack completion. Raises an :exc:`Error` exception if all of the
data has not been unpacked.
In addition, every data type that can be packed with a :class:`Packer`, can be
unpacked with an :class:`Unpacker`. Unpacking methods are of the form
``unpack_type()``, and take no arguments. They return the unpacked object.
.. method:: Unpacker.unpack_float()
Unpacks a single-precision floating point number.
.. method:: Unpacker.unpack_double()
Unpacks a double-precision floating point number, similarly to
:meth:`unpack_float`.
In addition, the following methods unpack strings, bytes, and opaque data:
.. method:: Unpacker.unpack_fstring(n)
Unpacks and returns a fixed length string. *n* is the number of characters
expected. Padding with null bytes to guaranteed 4 byte alignment is assumed.
.. method:: Unpacker.unpack_fopaque(n)
Unpacks and returns a fixed length opaque data stream, similarly to
:meth:`unpack_fstring`.
.. method:: Unpacker.unpack_string()
Unpacks and returns a variable length string. The length of the string is first
unpacked as an unsigned integer, then the string data is unpacked with
:meth:`unpack_fstring`.
.. method:: Unpacker.unpack_opaque()
Unpacks and returns a variable length opaque data string, similarly to
:meth:`unpack_string`.
.. method:: Unpacker.unpack_bytes()
Unpacks and returns a variable length byte stream, similarly to
:meth:`unpack_string`.
The following methods support unpacking arrays and lists:
.. method:: Unpacker.unpack_list(unpack_item)
Unpacks and returns a list of homogeneous items. The list is unpacked one
element at a time by first unpacking an unsigned integer flag. If the flag is
``1``, then the item is unpacked and appended to the list. A flag of ``0``
indicates the end of the list. *unpack_item* is the function that is called to
unpack the items.
.. method:: Unpacker.unpack_farray(n, unpack_item)
Unpacks and returns (as a list) a fixed length array of homogeneous items. *n*
is number of list elements to expect in the buffer. As above, *unpack_item* is
the function used to unpack each element.
.. method:: Unpacker.unpack_array(unpack_item)
Unpacks and returns a variable length *list* of homogeneous items. First, the
length of the list is unpacked as an unsigned integer, then each element is
unpacked as in :meth:`unpack_farray` above.
.. _xdr-exceptions:
Exceptions
----------
Exceptions in this module are coded as class instances:
.. exception:: Error
The base exception class. :exc:`Error` has a single public attribute
:attr:`msg` containing the description of the error.
.. exception:: ConversionError
Class derived from :exc:`Error`. Contains no additional instance variables.
Here is an example of how you would catch one of these exceptions::
import xdrlib
p = xdrlib.Packer()
try:
p.pack_double(8.01)
except xdrlib.ConversionError as instance:
print('packing the double failed:', instance.msg)

View file

@ -237,7 +237,6 @@ Doc/library/webbrowser.rst
Doc/library/winreg.rst
Doc/library/winsound.rst
Doc/library/wsgiref.rst
Doc/library/xdrlib.rst
Doc/library/xml.dom.minidom.rst
Doc/library/xml.dom.pulldom.rst
Doc/library/xml.dom.rst

View file

@ -1735,7 +1735,7 @@ Modules
+---------------------+---------------------+---------------------+---------------------+---------------------+
| :mod:`audioop` | :mod:`crypt` | :mod:`!nis` | :mod:`!sndhdr` | :mod:`uu` |
+---------------------+---------------------+---------------------+---------------------+---------------------+
| :mod:`!cgi` | :mod:`imghdr` | :mod:`!nntplib` | :mod:`!spwd` | :mod:`xdrlib` |
| :mod:`!cgi` | :mod:`imghdr` | :mod:`!nntplib` | :mod:`!spwd` | :mod:`!xdrlib` |
+---------------------+---------------------+---------------------+---------------------+---------------------+
| :mod:`!cgitb` | :mod:`!mailcap` | :mod:`!ossaudiodev` | :mod:`!sunau` | |
+---------------------+---------------------+---------------------+---------------------+---------------------+

View file

@ -904,7 +904,7 @@ Modules (see :pep:`594`):
* :mod:`!sunau`
* :mod:`!telnetlib`
* :mod:`uu`
* :mod:`xdrlib`
* :mod:`!xdrlib`
APIs:

View file

@ -195,6 +195,9 @@ Removed
* :pep:`594`: Remove the :mod:`!nis` module, deprecated in Python 3.11.
(Contributed by Victor Stinner in :gh:`104773`.)
* :pep:`594`: Remove the :mod:`!xdrlib` module, deprecated in Python 3.11.
(Contributed by Victor Stinner in :gh:`104773`.)
Porting to Python 3.13
======================

View file

@ -1,79 +0,0 @@
import unittest
from test.support import warnings_helper
xdrlib = warnings_helper.import_deprecated("xdrlib")
class XDRTest(unittest.TestCase):
def test_xdr(self):
p = xdrlib.Packer()
s = b'hello world'
a = [b'what', b'is', b'hapnin', b'doctor']
p.pack_int(42)
p.pack_int(-17)
p.pack_uint(9)
p.pack_bool(True)
p.pack_bool(False)
p.pack_uhyper(45)
p.pack_float(1.9)
p.pack_double(1.9)
p.pack_string(s)
p.pack_list(range(5), p.pack_uint)
p.pack_array(a, p.pack_string)
# now verify
data = p.get_buffer()
up = xdrlib.Unpacker(data)
self.assertEqual(up.get_position(), 0)
self.assertEqual(up.unpack_int(), 42)
self.assertEqual(up.unpack_int(), -17)
self.assertEqual(up.unpack_uint(), 9)
self.assertTrue(up.unpack_bool() is True)
# remember position
pos = up.get_position()
self.assertTrue(up.unpack_bool() is False)
# rewind and unpack again
up.set_position(pos)
self.assertTrue(up.unpack_bool() is False)
self.assertEqual(up.unpack_uhyper(), 45)
self.assertAlmostEqual(up.unpack_float(), 1.9)
self.assertAlmostEqual(up.unpack_double(), 1.9)
self.assertEqual(up.unpack_string(), s)
self.assertEqual(up.unpack_list(up.unpack_uint), list(range(5)))
self.assertEqual(up.unpack_array(up.unpack_string), a)
up.done()
self.assertRaises(EOFError, up.unpack_uint)
class ConversionErrorTest(unittest.TestCase):
def setUp(self):
self.packer = xdrlib.Packer()
def assertRaisesConversion(self, *args):
self.assertRaises(xdrlib.ConversionError, *args)
def test_pack_int(self):
self.assertRaisesConversion(self.packer.pack_int, 'string')
def test_pack_uint(self):
self.assertRaisesConversion(self.packer.pack_uint, 'string')
def test_float(self):
self.assertRaisesConversion(self.packer.pack_float, 'string')
def test_double(self):
self.assertRaisesConversion(self.packer.pack_double, 'string')
def test_uhyper(self):
self.assertRaisesConversion(self.packer.pack_uhyper, 'string')
if __name__ == "__main__":
unittest.main()

View file

@ -1,242 +0,0 @@
"""Implements (a subset of) Sun XDR -- eXternal Data Representation.
See: RFC 1014
"""
import struct
from io import BytesIO
from functools import wraps
import warnings
warnings._deprecated(__name__, remove=(3, 13))
__all__ = ["Error", "Packer", "Unpacker", "ConversionError"]
# exceptions
class Error(Exception):
"""Exception class for this module. Use:
except xdrlib.Error as var:
# var has the Error instance for the exception
Public ivars:
msg -- contains the message
"""
def __init__(self, msg):
self.msg = msg
def __repr__(self):
return repr(self.msg)
def __str__(self):
return str(self.msg)
class ConversionError(Error):
pass
def raise_conversion_error(function):
""" Wrap any raised struct.errors in a ConversionError. """
@wraps(function)
def result(self, value):
try:
return function(self, value)
except struct.error as e:
raise ConversionError(e.args[0]) from None
return result
class Packer:
"""Pack various data representations into a buffer."""
def __init__(self):
self.reset()
def reset(self):
self.__buf = BytesIO()
def get_buffer(self):
return self.__buf.getvalue()
# backwards compatibility
get_buf = get_buffer
@raise_conversion_error
def pack_uint(self, x):
self.__buf.write(struct.pack('>L', x))
@raise_conversion_error
def pack_int(self, x):
self.__buf.write(struct.pack('>l', x))
pack_enum = pack_int
def pack_bool(self, x):
if x: self.__buf.write(b'\0\0\0\1')
else: self.__buf.write(b'\0\0\0\0')
def pack_uhyper(self, x):
try:
self.pack_uint(x>>32 & 0xffffffff)
except (TypeError, struct.error) as e:
raise ConversionError(e.args[0]) from None
try:
self.pack_uint(x & 0xffffffff)
except (TypeError, struct.error) as e:
raise ConversionError(e.args[0]) from None
pack_hyper = pack_uhyper
@raise_conversion_error
def pack_float(self, x):
self.__buf.write(struct.pack('>f', x))
@raise_conversion_error
def pack_double(self, x):
self.__buf.write(struct.pack('>d', x))
def pack_fstring(self, n, s):
if n < 0:
raise ValueError('fstring size must be nonnegative')
data = s[:n]
n = ((n+3)//4)*4
data = data + (n - len(data)) * b'\0'
self.__buf.write(data)
pack_fopaque = pack_fstring
def pack_string(self, s):
n = len(s)
self.pack_uint(n)
self.pack_fstring(n, s)
pack_opaque = pack_string
pack_bytes = pack_string
def pack_list(self, list, pack_item):
for item in list:
self.pack_uint(1)
pack_item(item)
self.pack_uint(0)
def pack_farray(self, n, list, pack_item):
if len(list) != n:
raise ValueError('wrong array size')
for item in list:
pack_item(item)
def pack_array(self, list, pack_item):
n = len(list)
self.pack_uint(n)
self.pack_farray(n, list, pack_item)
class Unpacker:
"""Unpacks various data representations from the given buffer."""
def __init__(self, data):
self.reset(data)
def reset(self, data):
self.__buf = data
self.__pos = 0
def get_position(self):
return self.__pos
def set_position(self, position):
self.__pos = position
def get_buffer(self):
return self.__buf
def done(self):
if self.__pos < len(self.__buf):
raise Error('unextracted data remains')
def unpack_uint(self):
i = self.__pos
self.__pos = j = i+4
data = self.__buf[i:j]
if len(data) < 4:
raise EOFError
return struct.unpack('>L', data)[0]
def unpack_int(self):
i = self.__pos
self.__pos = j = i+4
data = self.__buf[i:j]
if len(data) < 4:
raise EOFError
return struct.unpack('>l', data)[0]
unpack_enum = unpack_int
def unpack_bool(self):
return bool(self.unpack_int())
def unpack_uhyper(self):
hi = self.unpack_uint()
lo = self.unpack_uint()
return int(hi)<<32 | lo
def unpack_hyper(self):
x = self.unpack_uhyper()
if x >= 0x8000000000000000:
x = x - 0x10000000000000000
return x
def unpack_float(self):
i = self.__pos
self.__pos = j = i+4
data = self.__buf[i:j]
if len(data) < 4:
raise EOFError
return struct.unpack('>f', data)[0]
def unpack_double(self):
i = self.__pos
self.__pos = j = i+8
data = self.__buf[i:j]
if len(data) < 8:
raise EOFError
return struct.unpack('>d', data)[0]
def unpack_fstring(self, n):
if n < 0:
raise ValueError('fstring size must be nonnegative')
i = self.__pos
j = i + (n+3)//4*4
if j > len(self.__buf):
raise EOFError
self.__pos = j
return self.__buf[i:i+n]
unpack_fopaque = unpack_fstring
def unpack_string(self):
n = self.unpack_uint()
return self.unpack_fstring(n)
unpack_opaque = unpack_string
unpack_bytes = unpack_string
def unpack_list(self, unpack_item):
list = []
while (x := self.unpack_uint()) != 0:
if x != 1:
raise ConversionError('0 or 1 expected, got %r' % (x,))
item = unpack_item()
list.append(item)
return list
def unpack_farray(self, n, unpack_item):
list = []
for i in range(n):
list.append(unpack_item())
return list
def unpack_array(self, unpack_item):
n = self.unpack_uint()
return self.unpack_farray(n, unpack_item)

View file

@ -0,0 +1,2 @@
:pep:`594`: Remove the :mod:`!xdrlib` module, deprecated in Python 3.11.
Patch by Victor Stinner.

View file

@ -282,7 +282,6 @@ static const char* _Py_stdlib_module_names[] = {
"winreg",
"winsound",
"wsgiref",
"xdrlib",
"xml",
"xmlrpc",
"zipapp",

View file

@ -43,7 +43,6 @@ OMIT_FILES = (
"venv/",
# deprecated
"uu.py",
"xdrlib.py",
# other platforms
"_aix_support.py",
"_osx_support.py",