diff --git a/Doc/distutils/index.rst b/Doc/distutils/index.rst index 1cd5f3c5dc6..1a6f04c87af 100644 --- a/Doc/distutils/index.rst +++ b/Doc/distutils/index.rst @@ -12,6 +12,16 @@ the module developer's point of view, describing how to use the Distutils to make Python modules and extensions easily available to a wider audience with very little overhead for build/release/install mechanics. +.. note:: + + This guide only covers the basic tools for building and distributing + extensions that are provided as part of this version of Python. Third + party tools offer easier to use and more secure alternatives. Refer to the + `quick recommendations section + `__ + in the Python Packaging User Guide for more information. + + .. toctree:: :maxdepth: 2 :numbered: diff --git a/Doc/extending/index.rst b/Doc/extending/index.rst index 61129439958..44a7f92c007 100644 --- a/Doc/extending/index.rst +++ b/Doc/extending/index.rst @@ -21,6 +21,15 @@ Python) that give the language its wide application range. For a detailed description of the whole Python/C API, see the separate :ref:`c-api-index`. +.. note:: + + This guide only covers the basic tools for creating extensions provided + as part of this version of CPython. Third party tools may offer simpler + alternatives. Refer to the `binary extensions section + `__ + in the Python Packaging User Guide for more information. + + .. toctree:: :maxdepth: 2 :numbered: diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 887f0b435af..42e55d6d6fe 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1607,26 +1607,34 @@ Modules How do I create a .pyc file? ---------------------------- -When a module is imported for the first time (or when the source is more recent -than the current compiled file) a ``.pyc`` file containing the compiled code -should be created in the same directory as the ``.py`` file. +When a module is imported for the first time (or when the source file has +changed since the current compiled file was created) a ``.pyc`` file containing +the compiled code should be created in a ``__pycache__`` subdirectory of the +directory containing the ``.py`` file. The ``.pyc`` file will have a +filename that starts with the same name as the ``.py`` file, and ends with +``.pyc``, with a middle component that depends on the particular ``python`` +binary that created it. (See :pep:`3147` for details.) -One reason that a ``.pyc`` file may not be created is permissions problems with -the directory. This can happen, for example, if you develop as one user but run -as another, such as if you are testing with a web server. Creation of a .pyc -file is automatic if you're importing a module and Python has the ability -(permissions, free space, etc...) to write the compiled module back to the -directory. +One reason that a ``.pyc`` file may not be created is a permissions problem +with the directory containing the source file, meaning that the ``__pycache__`` +subdirectory cannot be created. This can happen, for example, if you develop as +one user but run as another, such as if you are testing with a web server. + +Unless the :envvar:`PYTHONDONTWRITEBYTECODE` environment variable is set, +creation of a .pyc file is automatic if you're importing a module and Python +has the ability (permissions, free space, etc...) to create a ``__pycache__`` +subdirectory and write the compiled module to that subdirectory. Running Python on a top level script is not considered an import and no ``.pyc`` will be created. For example, if you have a top-level module -``foo.py`` that imports another module ``xyz.py``, when you run ``foo``, -``xyz.pyc`` will be created since ``xyz`` is imported, but no ``foo.pyc`` file -will be created since ``foo.py`` isn't being imported. +``foo.py`` that imports another module ``xyz.py``, when you run ``foo`` (by +typing ``python foo.py`` as a shell command), a ``.pyc`` will be created for +``xyz`` because ``xyz`` is imported, but no ``.pyc`` file will be created for +``foo`` since ``foo.py`` isn't being imported. -If you need to create ``foo.pyc`` -- that is, to create a ``.pyc`` file for a module -that is not imported -- you can, using the :mod:`py_compile` and -:mod:`compileall` modules. +If you need to create a ``.pyc`` file for ``foo`` -- that is, to create a +``.pyc`` file for a module that is not imported -- you can, using the +:mod:`py_compile` and :mod:`compileall` modules. The :mod:`py_compile` module can manually compile any module. One way is to use the ``compile()`` function in that module interactively:: @@ -1634,8 +1642,9 @@ the ``compile()`` function in that module interactively:: >>> import py_compile >>> py_compile.compile('foo.py') # doctest: +SKIP -This will write the ``.pyc`` to the same location as ``foo.py`` (or you can -override that with the optional parameter ``cfile``). +This will write the ``.pyc`` to a ``__pycache__`` subdirectory in the same +location as ``foo.py`` (or you can override that with the optional parameter +``cfile``). You can also automatically compile all files in a directory or directories using the :mod:`compileall` module. You can do it from the shell prompt by running diff --git a/Doc/install/index.rst b/Doc/install/index.rst index 738aaa152fc..779e923d945 100644 --- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -20,12 +20,20 @@ Finally, it might be useful to include all the material from my "Care and Feeding of a Python Installation" talk in here somewhere. Yow! -.. topic:: Abstract +This document describes the Python Distribution Utilities ("Distutils") from the +end-user's point-of-view, describing how to extend the capabilities of a +standard Python installation by building and installing third-party Python +modules and extensions. - This document describes the Python Distribution Utilities ("Distutils") from the - end-user's point-of-view, describing how to extend the capabilities of a - standard Python installation by building and installing third-party Python - modules and extensions. + +.. note:: + + This guide only covers the basic tools for installing extensions that are + provided as part of this version of Python. Third party tools offer easier + to use and more secure alternatives. Refer to the + `quick recommendations section + `__ + in the Python Packaging User Guide for more information. .. _inst-intro: diff --git a/Doc/library/bz2.rst b/Doc/library/bz2.rst index 95a5abb75db..b79bccdb31e 100644 --- a/Doc/library/bz2.rst +++ b/Doc/library/bz2.rst @@ -91,6 +91,11 @@ All of the classes in this module may safely be accessed from multiple threads. byte of data will be returned (unless at EOF). The exact number of bytes returned is unspecified. + .. note:: While calling :meth:`peek` does not change the file position of + the :class:`BZ2File`, it may change the position of the underlying file + object (e.g. if the :class:`BZ2File` was constructed by passing a file + object for *filename*). + .. versionadded:: 3.3 .. versionchanged:: 3.1 diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst index ec16f6ae88b..354deed921a 100644 --- a/Doc/library/gzip.rst +++ b/Doc/library/gzip.rst @@ -113,6 +113,11 @@ The module defines the following items: the call. The number of bytes returned may be more or less than requested. + .. note:: While calling :meth:`peek` does not change the file position of + the :class:`GzipFile`, it may change the position of the underlying + file object (e.g. if the :class:`GzipFile` was constructed with the + *fileobj* parameter). + .. versionadded:: 3.2 .. versionchanged:: 3.1 diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index efd027b3162..92339dcd85c 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -82,10 +82,13 @@ Functions derived from :func:`importlib.__import__`, including requiring the package from which an import is occurring to have been previously imported (i.e., *package* must already be imported). The most important difference - is that :func:`import_module` returns the most nested package or module - that was imported (e.g. ``pkg.mod``), while :func:`__import__` returns the + is that :func:`import_module` returns the specified package or module + (e.g. ``pkg.mod``), while :func:`__import__` returns the top-level package or module (e.g. ``pkg``). + .. versionchanged:: 3.3 + Parent packages are automatically imported. + .. function:: find_loader(name, path=None) Find the loader for a module, optionally within the specified *path*. If the diff --git a/Doc/library/lzma.rst b/Doc/library/lzma.rst index da4657551e2..07b69eb772a 100644 --- a/Doc/library/lzma.rst +++ b/Doc/library/lzma.rst @@ -98,6 +98,11 @@ Reading and writing compressed files byte of data will be returned, unless EOF has been reached. The exact number of bytes returned is unspecified (the *size* argument is ignored). + .. note:: While calling :meth:`peek` does not change the file position of + the :class:`LZMAFile`, it may change the position of the underlying + file object (e.g. if the :class:`LZMAFile` was constructed by passing a + file object for *filename*). + Compressing and decompressing data in memory -------------------------------------------- diff --git a/Doc/library/mailbox.rst b/Doc/library/mailbox.rst index d668a6eacb3..0478ed1087b 100644 --- a/Doc/library/mailbox.rst +++ b/Doc/library/mailbox.rst @@ -1550,7 +1550,7 @@ programs, mail loss due to interruption of the program, or premature termination due to malformed messages in the mailbox:: import mailbox - import email.Errors + import email.errors list_names = ('python-list', 'python-dev', 'python-bugs') @@ -1560,7 +1560,7 @@ due to malformed messages in the mailbox:: for key in inbox.iterkeys(): try: message = inbox[key] - except email.Errors.MessageParseError: + except email.errors.MessageParseError: continue # The message is malformed. Just leave it. for name in list_names: diff --git a/Doc/license.rst b/Doc/license.rst index 3e93a264390..0947349c3b4 100644 --- a/Doc/license.rst +++ b/Doc/license.rst @@ -845,3 +845,36 @@ used for the build:: Jean-loup Gailly Mark Adler jloup@gzip.org madler@alumni.caltech.edu + +libmpdec +-------- + +The :mod:`_decimal` Module is built using an included copy of the libmpdec +library unless the build is configured ``--with-system-libmpdec``:: + + Copyright (c) 2008-2016 Stefan Krah. All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + 1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + 2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS "AS IS" AND + ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + SUCH DAMAGE. + + diff --git a/Include/import.h b/Include/import.h index fdc27336c35..73c86ee400f 100644 --- a/Include/import.h +++ b/Include/import.h @@ -86,15 +86,15 @@ PyAPI_FUNC(int) _PyImport_ReleaseLock(void); PyAPI_FUNC(void) _PyImport_ReInitLock(void); -PyAPI_FUNC(PyObject *)_PyImport_FindBuiltin( +PyAPI_FUNC(PyObject *) _PyImport_FindBuiltin( const char *name /* UTF-8 encoded string */ ); -PyAPI_FUNC(PyObject *)_PyImport_FindExtensionObject(PyObject *, PyObject *); -PyAPI_FUNC(int)_PyImport_FixupBuiltin( +PyAPI_FUNC(PyObject *) _PyImport_FindExtensionObject(PyObject *, PyObject *); +PyAPI_FUNC(int) _PyImport_FixupBuiltin( PyObject *mod, char *name /* UTF-8 encoded string */ ); -PyAPI_FUNC(int)_PyImport_FixupExtensionObject(PyObject*, PyObject *, PyObject *); +PyAPI_FUNC(int) _PyImport_FixupExtensionObject(PyObject*, PyObject *, PyObject *); struct _inittab { char *name; /* ASCII encoded string */ diff --git a/Include/longobject.h b/Include/longobject.h index fbe738cd2c8..1c1c5de8e1e 100644 --- a/Include/longobject.h +++ b/Include/longobject.h @@ -152,6 +152,12 @@ PyAPI_FUNC(int) _PyLong_AsByteArray(PyLongObject* v, unsigned char* bytes, size_t n, int little_endian, int is_signed); +/* _PyLong_FromNbInt: Convert the given object to a PyLongObject + using the nb_int slot, if available. Raise TypeError if either the + nb_int slot is not available or the result of the call to nb_int + returns something not of type int. +*/ +PyAPI_FUNC(PyLongObject *)_PyLong_FromNbInt(PyObject *); /* _PyLong_Format: Convert the long to a string object with given base, appending a base prefix of 0[box] if base is 2, 8 or 16. */ diff --git a/Lib/aifc.py b/Lib/aifc.py index 86a5edc8c98..db0924a2b7f 100644 --- a/Lib/aifc.py +++ b/Lib/aifc.py @@ -771,7 +771,10 @@ class Aifc_write: self._datalength = (self._datalength + 3) // 4 if self._datalength & 1: self._datalength = self._datalength + 1 - self._form_length_pos = self._file.tell() + try: + self._form_length_pos = self._file.tell() + except (AttributeError, OSError): + self._form_length_pos = None commlength = self._write_form_length(self._datalength) if self._aifc: self._file.write(b'AIFC') @@ -783,7 +786,8 @@ class Aifc_write: self._file.write(b'COMM') _write_ulong(self._file, commlength) _write_short(self._file, self._nchannels) - self._nframes_pos = self._file.tell() + if self._form_length_pos is not None: + self._nframes_pos = self._file.tell() _write_ulong(self._file, self._nframes) if self._comptype in (b'ULAW', b'ulaw', b'ALAW', b'alaw', b'G722'): _write_short(self._file, 8) @@ -794,7 +798,8 @@ class Aifc_write: self._file.write(self._comptype) _write_string(self._file, self._compname) self._file.write(b'SSND') - self._ssnd_length_pos = self._file.tell() + if self._form_length_pos is not None: + self._ssnd_length_pos = self._file.tell() _write_ulong(self._file, self._datalength + 8) _write_ulong(self._file, 0) _write_ulong(self._file, 0) diff --git a/Lib/email/charset.py b/Lib/email/charset.py index 892bab54a48..e999472355e 100644 --- a/Lib/email/charset.py +++ b/Lib/email/charset.py @@ -386,7 +386,8 @@ class Charset: string using the ascii codec produces the correct string version of the content. """ - # 7bit/8bit encodings return the string unchanged (module conversions) + if not string: + return string if self.body_encoding is BASE64: if isinstance(string, str): string = string.encode(self.output_charset) @@ -398,13 +399,9 @@ class Charset: # character set, then, we must turn it into pseudo bytes via the # latin1 charset, which will encode any byte as a single code point # between 0 and 255, which is what body_encode is expecting. - # - # Note that this clause doesn't handle the case of a _payload that - # is already bytes. It never did, and the semantics of _payload - # being bytes has never been nailed down, so fixing that is a - # longer term TODO. if isinstance(string, str): - string = string.encode(self.output_charset).decode('latin1') + string = string.encode(self.output_charset) + string = string.decode('latin1') return email.quoprimime.body_encode(string) else: if isinstance(string, str): diff --git a/Lib/email/message.py b/Lib/email/message.py index f43a3809bec..63b51f6007d 100644 --- a/Lib/email/message.py +++ b/Lib/email/message.py @@ -275,9 +275,19 @@ class Message: Optional charset sets the message's default character set. See set_charset() for details. """ - if isinstance(payload, bytes): - payload = payload.decode('ascii', 'surrogateescape') - self._payload = payload + if hasattr(payload, 'encode'): + if charset is None: + # We should check for ASCII-only here, but we can't do that + # for backward compatibility reasons. Fixed in 3.4. + self._payload = payload + return + if not isinstance(charset, Charset): + charset = Charset(charset) + payload = payload.encode(charset.output_charset) + if hasattr(payload, 'decode'): + self._payload = payload.decode('ascii', 'surrogateescape') + else: + self._payload = payload if charset is not None: self.set_charset(charset) @@ -316,7 +326,15 @@ class Message: try: cte(self) except TypeError: - self._payload = charset.body_encode(self._payload) + # This if is for backward compatibility and will be removed + # in 3.4 when the ascii check is added to set_payload. + payload = self._payload + if payload: + try: + payload = payload.encode('ascii', 'surrogateescape') + except UnicodeError: + payload = payload.encode(charset.output_charset) + self._payload = charset.body_encode(payload) self.add_header('Content-Transfer-Encoding', cte) def get_charset(self): diff --git a/Lib/idlelib/PyShell.py b/Lib/idlelib/PyShell.py old mode 100644 new mode 100755 index b2a1f58c137..6674f4ecdda --- a/Lib/idlelib/PyShell.py +++ b/Lib/idlelib/PyShell.py @@ -1331,8 +1331,11 @@ class PseudoOutputFile(PseudoFile): def write(self, s): if self.closed: raise ValueError("write to closed file") - if not isinstance(s, str): - raise TypeError('must be str, not ' + type(s).__name__) + if type(s) is not str: + if not isinstance(s, str): + raise TypeError('must be str, not ' + type(s).__name__) + # See issue #19481 + s = str.__str__(s) return self.shell.write(s, self.tags) @@ -1531,20 +1534,22 @@ def main(): args.remove(filename) if not args: flist.new() + if enable_shell: shell = flist.open_shell() if not shell: return # couldn't open shell - if macosxSupport.runningAsOSXApp() and flist.dict: # On OSX: when the user has double-clicked on a file that causes # IDLE to be launched the shell window will open just in front of # the file she wants to see. Lower the interpreter window when # there are open files. shell.top.lower() + else: + shell = flist.pyshell - shell = flist.pyshell - # handle remaining options: + # Handle remaining options. If any of these are set, enable_shell + # was set also, so shell must be true to reach here. if debug: shell.open_debugger() if startup: @@ -1552,7 +1557,7 @@ def main(): os.environ.get("PYTHONSTARTUP") if filename and os.path.isfile(filename): shell.interp.execfile(filename) - if shell and cmd or script: + if cmd or script: shell.interp.runcommand("""if 1: import sys as _sys _sys.argv = %r @@ -1563,13 +1568,14 @@ def main(): elif script: shell.interp.prepend_syspath(script) shell.interp.execfile(script) - - # Check for problematic OS X Tk versions and print a warning message - # in the IDLE shell window; this is less intrusive than always opening - # a separate window. - tkversionwarning = macosxSupport.tkVersionWarning(root) - if tkversionwarning: - shell.interp.runcommand(''.join(("print('", tkversionwarning, "')"))) + elif shell: + # If there is a shell window and no cmd or script in progress, + # check for problematic OS X Tk versions and print a warning + # message in the IDLE shell window; this is less intrusive + # than always opening a separate window. + tkversionwarning = macosxSupport.tkVersionWarning(root) + if tkversionwarning: + shell.interp.runcommand("print('%s')" % tkversionwarning) while flist.inversedict: # keep IDLE running while files are open. root.mainloop() diff --git a/Lib/json/__init__.py b/Lib/json/__init__.py index 48a4f8f863e..6d9b30d92e6 100644 --- a/Lib/json/__init__.py +++ b/Lib/json/__init__.py @@ -3,11 +3,8 @@ JavaScript syntax (ECMA-262 3rd edition) used as a lightweight data interchange format. :mod:`json` exposes an API familiar to users of the standard library -:mod:`marshal` and :mod:`pickle` modules. It is the externally maintained -version of the :mod:`json` library contained in Python 2.6, but maintains -compatibility with Python 2.4 and Python 2.5 and (currently) has -significant performance advantages, even without using the optional C -extension for speedups. +:mod:`marshal` and :mod:`pickle` modules. It is derived from a +version of the externally maintained simplejson library. Encoding basic Python object hierarchies:: diff --git a/Lib/lib2to3/fixes/fix_import.py b/Lib/lib2to3/fixes/fix_import.py index e978fce43fc..734ca294699 100644 --- a/Lib/lib2to3/fixes/fix_import.py +++ b/Lib/lib2to3/fixes/fix_import.py @@ -32,7 +32,7 @@ def traverse_imports(names): elif node.type == syms.dotted_as_names: pending.extend(node.children[::-2]) else: - raise AssertionError("unkown node type") + raise AssertionError("unknown node type") class FixImport(fixer_base.BaseFix): diff --git a/Lib/platform.py b/Lib/platform.py index ec5df85d6f4..030ef2a3a4b 100755 --- a/Lib/platform.py +++ b/Lib/platform.py @@ -129,6 +129,10 @@ except AttributeError: # Standard Unix uses /dev/null DEV_NULL = '/dev/null' +# Directory to search for configuration information on Unix. +# Constant used by test_platform to test linux_distribution(). +_UNIXCONFDIR = '/etc' + ### Platform specific APIs _libc_search = re.compile(b'(__libc_init)' @@ -315,7 +319,7 @@ def linux_distribution(distname='', version='', id='', """ try: - etc = os.listdir('/etc') + etc = os.listdir(_UNIXCONFDIR) except os.error: # Probably not a Unix system return distname,version,id @@ -331,7 +335,8 @@ def linux_distribution(distname='', version='', id='', return _dist_try_harder(distname,version,id) # Read the first line - with open('/etc/'+file, 'r') as f: + with open(os.path.join(_UNIXCONFDIR, file), 'r', + encoding='utf-8', errors='surrogateescape') as f: firstline = f.readline() _distname, _version, _id = _parse_release_file(firstline) diff --git a/Lib/subprocess.py b/Lib/subprocess.py index c3a278836c1..78e4fcfc596 100644 --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -1621,6 +1621,9 @@ class Popen(object): self._save_input(input) + if self._input: + input_view = memoryview(self._input) + while self._fd2file: timeout = self._remaining_time(endtime) if timeout is not None and timeout < 0: @@ -1638,8 +1641,8 @@ class Popen(object): for fd, mode in ready: if mode & select.POLLOUT: - chunk = self._input[self._input_offset : - self._input_offset + _PIPE_BUF] + chunk = input_view[self._input_offset : + self._input_offset + _PIPE_BUF] try: self._input_offset += os.write(fd, chunk) except OSError as e: @@ -1651,7 +1654,7 @@ class Popen(object): if self._input_offset >= len(self._input): close_unregister_and_remove(fd) elif mode & select_POLLIN_POLLPRI: - data = os.read(fd, 4096) + data = os.read(fd, 32768) if not data: close_unregister_and_remove(fd) self._fd2output[fd].append(data) diff --git a/Lib/telnetlib.py b/Lib/telnetlib.py index 14ca1b19041..d49d4f41aff 100644 --- a/Lib/telnetlib.py +++ b/Lib/telnetlib.py @@ -315,7 +315,8 @@ class Telnet: poller.register(self, poll_in_or_priority_flags) while i < 0 and not self.eof: try: - ready = poller.poll(call_timeout) + ready = poller.poll(None if timeout is None + else 1000 * call_timeout) except select.error as e: if e.errno == errno.EINTR: if timeout is not None: @@ -683,7 +684,8 @@ class Telnet: poller.register(self, poll_in_or_priority_flags) while not m and not self.eof: try: - ready = poller.poll(call_timeout) + ready = poller.poll(None if timeout is None + else 1000 * call_timeout) except select.error as e: if e.errno == errno.EINTR: if timeout is not None: diff --git a/Lib/test/audiotests.py b/Lib/test/audiotests.py index 008281f9af2..6ff5bb81bba 100644 --- a/Lib/test/audiotests.py +++ b/Lib/test/audiotests.py @@ -5,6 +5,13 @@ import io import pickle import sys +class UnseekableIO(io.FileIO): + def tell(self): + raise io.UnsupportedOperation + + def seek(self, *args, **kwargs): + raise io.UnsupportedOperation + def byteswap2(data): a = array.array('h') a.frombytes(data) @@ -129,6 +136,61 @@ class AudioWriteTests(AudioTests): self.assertEqual(testfile.read(13), b'ababagalamaga') self.check_file(testfile, self.nframes, self.frames) + def test_unseekable_read(self): + f = self.create_file(TESTFN) + f.setnframes(self.nframes) + f.writeframes(self.frames) + f.close() + + with UnseekableIO(TESTFN, 'rb') as testfile: + self.check_file(testfile, self.nframes, self.frames) + + def test_unseekable_write(self): + with UnseekableIO(TESTFN, 'wb') as testfile: + f = self.create_file(testfile) + f.setnframes(self.nframes) + f.writeframes(self.frames) + f.close() + + self.check_file(TESTFN, self.nframes, self.frames) + + def test_unseekable_incompleted_write(self): + with UnseekableIO(TESTFN, 'wb') as testfile: + testfile.write(b'ababagalamaga') + f = self.create_file(testfile) + f.setnframes(self.nframes + 1) + try: + f.writeframes(self.frames) + except OSError: + pass + try: + f.close() + except OSError: + pass + + with open(TESTFN, 'rb') as testfile: + self.assertEqual(testfile.read(13), b'ababagalamaga') + self.check_file(testfile, self.nframes + 1, self.frames) + + def test_unseekable_overflowed_write(self): + with UnseekableIO(TESTFN, 'wb') as testfile: + testfile.write(b'ababagalamaga') + f = self.create_file(testfile) + f.setnframes(self.nframes - 1) + try: + f.writeframes(self.frames) + except OSError: + pass + try: + f.close() + except OSError: + pass + + with open(TESTFN, 'rb') as testfile: + self.assertEqual(testfile.read(13), b'ababagalamaga') + framesize = self.nchannels * self.sampwidth + self.check_file(testfile, self.nframes - 1, self.frames[:-framesize]) + class AudioTestsWithSourceFile(AudioTests): diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 1343f345a82..ab554762265 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -2025,6 +2025,7 @@ class TestDateTime(TestDate): class TestSubclassDateTime(TestDateTime): theclass = SubclassDatetime # Override tests not designed for subclass + @unittest.skip('not appropriate for subclasses') def test_roundtrip(self): pass diff --git a/Lib/test/multibytecodec_support.py b/Lib/test/multibytecodec_support.py index 26bac7be108..24003239675 100644 --- a/Lib/test/multibytecodec_support.py +++ b/Lib/test/multibytecodec_support.py @@ -73,7 +73,7 @@ class TestBase: def test_xmlcharrefreplace(self): if self.has_iso10646: - return + self.skipTest('encoding contains full ISO 10646 map') s = "\u0b13\u0b23\u0b60 nd eggs" self.assertEqual( @@ -83,7 +83,7 @@ class TestBase: def test_customreplace_encode(self): if self.has_iso10646: - return + self.skipTest('encoding contains full ISO 10646 map') from html.entities import codepoint2name diff --git a/Lib/test/string_tests.py b/Lib/test/string_tests.py index 1d70385d055..1b5db9406b0 100644 --- a/Lib/test/string_tests.py +++ b/Lib/test/string_tests.py @@ -663,10 +663,10 @@ class BaseTest: self.checkraises(TypeError, 'hello', 'replace', 42, 'h') self.checkraises(TypeError, 'hello', 'replace', 'h', 42) + @unittest.skipIf(sys.maxsize > (1 << 32) or struct.calcsize('P') != 4, + 'only applies to 32-bit platforms') def test_replace_overflow(self): # Check for overflow checking on 32 bit machines - if sys.maxsize != 2147483647 or struct.calcsize("P") > 4: - return A2_16 = "A" * (2**16) self.checkraises(OverflowError, A2_16, "replace", "", A2_16) self.checkraises(OverflowError, A2_16, "replace", "A", A2_16) diff --git a/Lib/test/test_abstract_numbers.py b/Lib/test/test_abstract_numbers.py index 253e6f082c3..2e06f0d16fd 100644 --- a/Lib/test/test_abstract_numbers.py +++ b/Lib/test/test_abstract_numbers.py @@ -4,7 +4,6 @@ import math import operator import unittest from numbers import Complex, Real, Rational, Integral -from test import support class TestNumbers(unittest.TestCase): def test_int(self): @@ -40,9 +39,6 @@ class TestNumbers(unittest.TestCase): self.assertRaises(TypeError, float, c1) self.assertRaises(TypeError, int, c1) -def test_main(): - support.run_unittest(TestNumbers) - if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_aifc.py b/Lib/test/test_aifc.py index b77354bb070..10260e3269f 100644 --- a/Lib/test/test_aifc.py +++ b/Lib/test/test_aifc.py @@ -8,10 +8,14 @@ import struct import aifc -class AifcPCM8Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): +class AifcTest(audiotests.AudioWriteTests, + audiotests.AudioTestsWithSourceFile): module = aifc + close_fd = True + test_unseekable_read = None + + +class AifcPCM8Test(AifcTest, unittest.TestCase): sndfilename = 'pluck-pcm8.aiff' sndfilenframes = 3307 nchannels = 2 @@ -26,13 +30,9 @@ class AifcPCM8Test(audiotests.AudioWriteTests, 11FA 3EFB BCFC 66FF CF04 4309 C10E 5112 EE17 8216 7F14 8012 \ 490E 520D EF0F CE0F E40C 630A 080A 2B0B 510E 8B11 B60E 440A \ """) - close_fd = True -class AifcPCM16Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = aifc +class AifcPCM16Test(AifcTest, unittest.TestCase): sndfilename = 'pluck-pcm16.aiff' sndfilenframes = 3307 nchannels = 2 @@ -49,13 +49,9 @@ class AifcPCM16Test(audiotests.AudioWriteTests, EEE21753 82071665 7FFF1443 8004128F 49A20EAF 52BB0DBA EFB40F60 CE3C0FBF \ E4B30CEC 63430A5C 08C80A20 2BBB0B08 514A0E43 8BCF1139 B6F60EEB 44120A5E \ """) - close_fd = True -class AifcPCM24Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = aifc +class AifcPCM24Test(AifcTest, unittest.TestCase): sndfilename = 'pluck-pcm24.aiff' sndfilenframes = 3307 nchannels = 2 @@ -78,13 +74,9 @@ class AifcPCM24Test(audiotests.AudioWriteTests, E4B49C0CEA2D 6344A80A5A7C 08C8FE0A1FFE 2BB9860B0A0E \ 51486F0E44E1 8BCC64113B05 B6F4EC0EEB36 4413170A5B48 \ """) - close_fd = True -class AifcPCM32Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = aifc +class AifcPCM32Test(AifcTest, unittest.TestCase): sndfilename = 'pluck-pcm32.aiff' sndfilenframes = 3307 nchannels = 2 @@ -107,13 +99,9 @@ class AifcPCM32Test(audiotests.AudioWriteTests, E4B49CC00CEA2D90 6344A8800A5A7CA0 08C8FE800A1FFEE0 2BB986C00B0A0E00 \ 51486F800E44E190 8BCC6480113B0580 B6F4EC000EEB3630 441317800A5B48A0 \ """) - close_fd = True -class AifcULAWTest(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = aifc +class AifcULAWTest(AifcTest, unittest.TestCase): sndfilename = 'pluck-ulaw.aifc' sndfilenframes = 3307 nchannels = 2 @@ -132,13 +120,9 @@ class AifcULAWTest(audiotests.AudioWriteTests, """) if sys.byteorder != 'big': frames = audiotests.byteswap2(frames) - close_fd = True -class AifcALAWTest(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = aifc +class AifcALAWTest(AifcTest, unittest.TestCase): sndfilename = 'pluck-alaw.aifc' sndfilenframes = 3307 nchannels = 2 @@ -157,7 +141,6 @@ class AifcALAWTest(audiotests.AudioWriteTests, """) if sys.byteorder != 'big': frames = audiotests.byteswap2(frames) - close_fd = True class AifcMiscTest(audiotests.AudioTests, unittest.TestCase): diff --git a/Lib/test/test_array.py b/Lib/test/test_array.py index cccb705b0d2..e1362b20e5a 100755 --- a/Lib/test/test_array.py +++ b/Lib/test/test_array.py @@ -946,7 +946,7 @@ class BaseTest: try: import gc except ImportError: - return + self.skipTest('gc module not available') a = array.array(self.typecode) l = [iter(a)] l.append(l) diff --git a/Lib/test/test_bytes.py b/Lib/test/test_bytes.py index 3520e837a17..3c091411849 100644 --- a/Lib/test/test_bytes.py +++ b/Lib/test/test_bytes.py @@ -729,6 +729,12 @@ class BytesTest(BaseBytesTest, unittest.TestCase): self.assertEqual(PyBytes_FromFormat(b's:%s', c_char_p(b'cstr')), b's:cstr') + # Issue #19969 + self.assertRaises(OverflowError, + PyBytes_FromFormat, b'%c', c_int(-1)) + self.assertRaises(OverflowError, + PyBytes_FromFormat, b'%c', c_int(256)) + class ByteArrayTest(BaseBytesTest, unittest.TestCase): type2test = bytearray diff --git a/Lib/test/test_codecencodings_iso2022.py b/Lib/test/test_codecencodings_iso2022.py index e4c1839b426..e4b2f306d5d 100644 --- a/Lib/test/test_codecencodings_iso2022.py +++ b/Lib/test/test_codecencodings_iso2022.py @@ -36,6 +36,7 @@ class Test_ISO2022_KR(multibytecodec_support.TestBase, unittest.TestCase): # iso2022_kr.txt cannot be used to test "chunk coding": the escape # sequence is only written on the first line + @unittest.skip('iso2022_kr.txt cannot be used to test "chunk coding"') def test_chunkcoding(self): pass diff --git a/Lib/test/test_configparser.py b/Lib/test/test_configparser.py index 29e7ed99ae9..78bd315344c 100644 --- a/Lib/test/test_configparser.py +++ b/Lib/test/test_configparser.py @@ -707,8 +707,7 @@ boolean {0[0]} NO def test_read_returns_file_list(self): if self.delimiters[0] != '=': - # skip reading the file if we're using an incompatible format - return + self.skipTest('incompatible format') file1 = support.findfile("cfgparser.1") # check when we pass a mix of readable and non-readable files: cf = self.newconfig() diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py index 54a3615e9cd..40313475fb9 100644 --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -290,7 +290,6 @@ class IBMTestCases(unittest.TestCase): global skip_expected if skip_expected: raise unittest.SkipTest - return with open(file) as f: for line in f: line = line.replace('\r\n', '').replace('\n', '') @@ -301,7 +300,6 @@ class IBMTestCases(unittest.TestCase): #Exception raised where there shouldn't have been one. self.fail('Exception "'+exception.__class__.__name__ + '" raised on line '+line) - return def eval_line(self, s): if s.find(' -> ') >= 0 and s[:2] != '--' and not s.startswith(' --'): @@ -461,7 +459,6 @@ class IBMTestCases(unittest.TestCase): self.assertEqual(myexceptions, theirexceptions, 'Incorrect flags set in ' + s + ' -- got ' + str(myexceptions)) - return def getexceptions(self): return [e for e in Signals[self.decimal] if self.context.flags[e]] @@ -1073,7 +1070,7 @@ class FormatTest(unittest.TestCase): try: from locale import CHAR_MAX except ImportError: - return + self.skipTest('locale.CHAR_MAX not available') def make_grouping(lst): return ''.join([chr(x) for x in lst]) if self.decimal == C else lst @@ -1164,8 +1161,12 @@ class FormatTest(unittest.TestCase): decimal_point = locale.localeconv()['decimal_point'] thousands_sep = locale.localeconv()['thousands_sep'] - if decimal_point != '\u066b' or thousands_sep != '\u066c': - return + if decimal_point != '\u066b': + self.skipTest('inappropriate decimal point separator' + '({!a} not {!a})'.format(decimal_point, '\u066b')) + if thousands_sep != '\u066c': + self.skipTest('inappropriate thousands separator' + '({!a} not {!a})'.format(thousands_sep, '\u066c')) self.assertEqual(format(Decimal('100000000.123'), 'n'), '100\u066c000\u066c000\u066b123') @@ -1515,7 +1516,6 @@ def thfunc1(cls): cls.assertTrue(c1.flags[Inexact]) for sig in Overflow, Underflow, DivisionByZero, InvalidOperation: cls.assertFalse(c1.flags[sig]) - return def thfunc2(cls): Decimal = cls.decimal.Decimal @@ -1560,7 +1560,6 @@ def thfunc2(cls): cls.assertTrue(thiscontext.flags[Inexact]) for sig in Overflow, Underflow, DivisionByZero, InvalidOperation: cls.assertFalse(thiscontext.flags[sig]) - return class ThreadingTest(unittest.TestCase): '''Unit tests for thread local contexts in Decimal.''' @@ -1602,7 +1601,6 @@ class ThreadingTest(unittest.TestCase): DefaultContext.prec = save_prec DefaultContext.Emax = save_emax DefaultContext.Emin = save_emin - return @unittest.skipUnless(threading, 'threading required') class CThreadingTest(ThreadingTest): @@ -4525,7 +4523,6 @@ class PyWhitebox(unittest.TestCase): self.assertEqual(d1._sign, b1._sign) self.assertEqual(d1._int, b1._int) self.assertEqual(d1._exp, b1._exp) - return Decimal(d1) self.assertEqual(d1._sign, b1._sign) @@ -5271,7 +5268,7 @@ class CWhitebox(unittest.TestCase): try: from locale import CHAR_MAX except ImportError: - return + self.skipTest('locale.CHAR_MAX not available') def make_grouping(lst): return ''.join([chr(x) for x in lst]) diff --git a/Lib/test/test_devpoll.py b/Lib/test/test_devpoll.py index bef4e1846b2..c7bacf89a21 100644 --- a/Lib/test/test_devpoll.py +++ b/Lib/test/test_devpoll.py @@ -87,6 +87,17 @@ class DevPollTests(unittest.TestCase): self.assertRaises(OverflowError, pollster.poll, 1 << 63) self.assertRaises(OverflowError, pollster.poll, 1 << 64) + def test_events_mask_overflow(self): + pollster = select.devpoll() + w, r = os.pipe() + pollster.register(w) + # Issue #17919 + self.assertRaises(OverflowError, pollster.register, 0, -1) + self.assertRaises(OverflowError, pollster.register, 0, USHRT_MAX + 1) + self.assertRaises(OverflowError, pollster.modify, 1, -1) + self.assertRaises(OverflowError, pollster.modify, 1, USHRT_MAX + 1) + + def test_main(): run_unittest(DevPollTests) diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py index d1355bbb960..9edf60202ea 100644 --- a/Lib/test/test_dis.py +++ b/Lib/test/test_dis.py @@ -217,16 +217,18 @@ class DisTests(unittest.TestCase): def test_bug_708901(self): self.do_disassembly_test(bug708901, dis_bug708901) + # Test has been disabled due to change in the way + # list comps are handled. The byte code now includes + # a memory address and a file location, so they change from + # run to run. + @unittest.skip('disabled due to a change in the way list comps are handled') def test_bug_1333982(self): # XXX: re-enable this test! # This one is checking bytecodes generated for an `assert` statement, # so fails if the tests are run with -O. Skip this test then. - pass # Test has been disabled due to change in the way - # list comps are handled. The byte code now includes - # a memory address and a file location, so they change from - # run to run. - # if __debug__: - # self.do_disassembly_test(bug1333982, dis_bug1333982) + + if __debug__: + self.do_disassembly_test(bug1333982, dis_bug1333982) def test_big_linenos(self): def func(count): diff --git a/Lib/test/test_docxmlrpc.py b/Lib/test/test_docxmlrpc.py index 7086d9a6a12..cb6366c7d5c 100644 --- a/Lib/test/test_docxmlrpc.py +++ b/Lib/test/test_docxmlrpc.py @@ -202,10 +202,12 @@ class DocXMLRPCHTTPGETServer(unittest.TestCase): """ Test that annotations works as expected """ self.client.request("GET", "/") response = self.client.getresponse() + docstring = (b'' if sys.flags.optimize >= 2 else + b'
Use function annotations.
') self.assertIn( (b'
annotation' - b'(x: int)
Use function annotations.' - b'
\n
' + b'(x: int)
' + docstring + b'
\n' + b'
' b'method_annotation(x: bytes)
'), response.read()) diff --git a/Lib/test/test_email/test_email.py b/Lib/test/test_email/test_email.py index cd4f757c1f1..6af62788a63 100644 --- a/Lib/test/test_email/test_email.py +++ b/Lib/test/test_email/test_email.py @@ -92,6 +92,38 @@ class TestMessageAPI(TestEmailBase): msg.set_payload('This is a string payload', charset) self.assertEqual(msg.get_charset().input_charset, 'iso-8859-1') + def test_set_payload_with_8bit_data_and_charset(self): + data = b'\xd0\x90\xd0\x91\xd0\x92' + charset = Charset('utf-8') + msg = Message() + msg.set_payload(data, charset) + self.assertEqual(msg['content-transfer-encoding'], 'base64') + self.assertEqual(msg.get_payload(decode=True), data) + self.assertEqual(msg.get_payload(), '0JDQkdCS\n') + + def test_set_payload_with_non_ascii_and_charset_body_encoding_none(self): + data = b'\xd0\x90\xd0\x91\xd0\x92' + charset = Charset('utf-8') + charset.body_encoding = None # Disable base64 encoding + msg = Message() + msg.set_payload(data.decode('utf-8'), charset) + self.assertEqual(msg['content-transfer-encoding'], '8bit') + self.assertEqual(msg.get_payload(decode=True), data) + + def test_set_payload_with_8bit_data_and_charset_body_encoding_none(self): + data = b'\xd0\x90\xd0\x91\xd0\x92' + charset = Charset('utf-8') + charset.body_encoding = None # Disable base64 encoding + msg = Message() + msg.set_payload(data, charset) + self.assertEqual(msg['content-transfer-encoding'], '8bit') + self.assertEqual(msg.get_payload(decode=True), data) + + def test_set_payload_to_list(self): + msg = Message() + msg.set_payload([]) + self.assertEqual(msg.get_payload(), []) + def test_get_charsets(self): eq = self.assertEqual diff --git a/Lib/test/test_fileio.py b/Lib/test/test_fileio.py index f91ccaa4278..a6853a147ef 100644 --- a/Lib/test/test_fileio.py +++ b/Lib/test/test_fileio.py @@ -341,8 +341,7 @@ class OtherFileTests(unittest.TestCase): try: fn = TESTFN.encode("ascii") except UnicodeEncodeError: - # Skip test - return + self.skipTest('could not encode %r to ascii' % TESTFN) f = _FileIO(fn, "w") try: f.write(b"abc") diff --git a/Lib/test/test_float.py b/Lib/test/test_float.py index 502292f6154..96d907a6719 100644 --- a/Lib/test/test_float.py +++ b/Lib/test/test_float.py @@ -70,7 +70,7 @@ class GeneralFloatCases(unittest.TestCase): # it still has to accept the normal python syntax import locale if not locale.localeconv()['decimal_point'] == ',': - return + self.skipTest('decimal_point is not ","') self.assertEqual(float(" 3.14 "), 3.14) self.assertEqual(float("+3.14 "), 3.14) diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py index db1e9348dd4..d1ce2a9c8ef 100644 --- a/Lib/test/test_functools.py +++ b/Lib/test/test_functools.py @@ -45,8 +45,6 @@ class TestPartial(unittest.TestCase): self.assertEqual(p.args, (1, 2)) self.assertEqual(p.keywords, dict(a=10, b=20)) # attributes should not be writable - if not isinstance(self.thetype, type): - return self.assertRaises(AttributeError, setattr, p, 'func', map) self.assertRaises(AttributeError, setattr, p, 'args', (1, 2)) self.assertRaises(AttributeError, setattr, p, 'keywords', dict(a=1, b=2)) @@ -210,11 +208,13 @@ class TestPythonPartial(TestPartial): thetype = PythonPartial # the python version hasn't a nice repr - def test_repr(self): pass + test_repr = None # the python version isn't picklable - def test_pickle(self): pass - def test_setstate_refcount(self): pass + test_pickle = test_setstate_refcount = None + + # the python version isn't a type + test_attributes = None class TestUpdateWrapper(unittest.TestCase): diff --git a/Lib/test/test_getargs2.py b/Lib/test/test_getargs2.py index 48ca94ee381..715b5c6a32e 100644 --- a/Lib/test/test_getargs2.py +++ b/Lib/test/test_getargs2.py @@ -50,12 +50,33 @@ class Int: def __int__(self): return 99 +class IntSubclass(int): + def __int__(self): + return 99 + +class BadInt: + def __int__(self): + return 1.0 + +class BadInt2: + def __int__(self): + return True + +class BadInt3(int): + def __int__(self): + return True + + class Unsigned_TestCase(unittest.TestCase): def test_b(self): from _testcapi import getargs_b # b returns 'unsigned char', and does range checking (0 ... UCHAR_MAX) self.assertRaises(TypeError, getargs_b, 3.14) self.assertEqual(99, getargs_b(Int())) + self.assertEqual(0, getargs_b(IntSubclass())) + self.assertRaises(TypeError, getargs_b, BadInt()) + self.assertEqual(1, getargs_b(BadInt2())) + self.assertEqual(0, getargs_b(BadInt3())) self.assertRaises(OverflowError, getargs_b, -1) self.assertEqual(0, getargs_b(0)) @@ -70,6 +91,10 @@ class Unsigned_TestCase(unittest.TestCase): # B returns 'unsigned char', no range checking self.assertRaises(TypeError, getargs_B, 3.14) self.assertEqual(99, getargs_B(Int())) + self.assertEqual(0, getargs_B(IntSubclass())) + self.assertRaises(TypeError, getargs_B, BadInt()) + self.assertEqual(1, getargs_B(BadInt2())) + self.assertEqual(0, getargs_B(BadInt3())) self.assertEqual(UCHAR_MAX, getargs_B(-1)) self.assertEqual(0, getargs_B(0)) @@ -84,6 +109,10 @@ class Unsigned_TestCase(unittest.TestCase): # H returns 'unsigned short', no range checking self.assertRaises(TypeError, getargs_H, 3.14) self.assertEqual(99, getargs_H(Int())) + self.assertEqual(0, getargs_H(IntSubclass())) + self.assertRaises(TypeError, getargs_H, BadInt()) + self.assertEqual(1, getargs_H(BadInt2())) + self.assertEqual(0, getargs_H(BadInt3())) self.assertEqual(USHRT_MAX, getargs_H(-1)) self.assertEqual(0, getargs_H(0)) @@ -99,6 +128,10 @@ class Unsigned_TestCase(unittest.TestCase): # I returns 'unsigned int', no range checking self.assertRaises(TypeError, getargs_I, 3.14) self.assertEqual(99, getargs_I(Int())) + self.assertEqual(0, getargs_I(IntSubclass())) + self.assertRaises(TypeError, getargs_I, BadInt()) + self.assertEqual(1, getargs_I(BadInt2())) + self.assertEqual(0, getargs_I(BadInt3())) self.assertEqual(UINT_MAX, getargs_I(-1)) self.assertEqual(0, getargs_I(0)) @@ -115,6 +148,10 @@ class Unsigned_TestCase(unittest.TestCase): # it does not accept float, or instances with __int__ self.assertRaises(TypeError, getargs_k, 3.14) self.assertRaises(TypeError, getargs_k, Int()) + self.assertEqual(0, getargs_k(IntSubclass())) + self.assertRaises(TypeError, getargs_k, BadInt()) + self.assertRaises(TypeError, getargs_k, BadInt2()) + self.assertEqual(0, getargs_k(BadInt3())) self.assertEqual(ULONG_MAX, getargs_k(-1)) self.assertEqual(0, getargs_k(0)) @@ -131,6 +168,10 @@ class Signed_TestCase(unittest.TestCase): # h returns 'short', and does range checking (SHRT_MIN ... SHRT_MAX) self.assertRaises(TypeError, getargs_h, 3.14) self.assertEqual(99, getargs_h(Int())) + self.assertEqual(0, getargs_h(IntSubclass())) + self.assertRaises(TypeError, getargs_h, BadInt()) + self.assertEqual(1, getargs_h(BadInt2())) + self.assertEqual(0, getargs_h(BadInt3())) self.assertRaises(OverflowError, getargs_h, SHRT_MIN-1) self.assertEqual(SHRT_MIN, getargs_h(SHRT_MIN)) @@ -145,6 +186,10 @@ class Signed_TestCase(unittest.TestCase): # i returns 'int', and does range checking (INT_MIN ... INT_MAX) self.assertRaises(TypeError, getargs_i, 3.14) self.assertEqual(99, getargs_i(Int())) + self.assertEqual(0, getargs_i(IntSubclass())) + self.assertRaises(TypeError, getargs_i, BadInt()) + self.assertEqual(1, getargs_i(BadInt2())) + self.assertEqual(0, getargs_i(BadInt3())) self.assertRaises(OverflowError, getargs_i, INT_MIN-1) self.assertEqual(INT_MIN, getargs_i(INT_MIN)) @@ -159,6 +204,10 @@ class Signed_TestCase(unittest.TestCase): # l returns 'long', and does range checking (LONG_MIN ... LONG_MAX) self.assertRaises(TypeError, getargs_l, 3.14) self.assertEqual(99, getargs_l(Int())) + self.assertEqual(0, getargs_l(IntSubclass())) + self.assertRaises(TypeError, getargs_l, BadInt()) + self.assertEqual(1, getargs_l(BadInt2())) + self.assertEqual(0, getargs_l(BadInt3())) self.assertRaises(OverflowError, getargs_l, LONG_MIN-1) self.assertEqual(LONG_MIN, getargs_l(LONG_MIN)) @@ -174,6 +223,10 @@ class Signed_TestCase(unittest.TestCase): # (PY_SSIZE_T_MIN ... PY_SSIZE_T_MAX) self.assertRaises(TypeError, getargs_n, 3.14) self.assertRaises(TypeError, getargs_n, Int()) + self.assertEqual(0, getargs_n(IntSubclass())) + self.assertRaises(TypeError, getargs_n, BadInt()) + self.assertRaises(TypeError, getargs_n, BadInt2()) + self.assertEqual(0, getargs_n(BadInt3())) self.assertRaises(OverflowError, getargs_n, PY_SSIZE_T_MIN-1) self.assertEqual(PY_SSIZE_T_MIN, getargs_n(PY_SSIZE_T_MIN)) @@ -192,6 +245,10 @@ class LongLong_TestCase(unittest.TestCase): self.assertRaises(TypeError, getargs_L, 3.14) self.assertRaises(TypeError, getargs_L, "Hello") self.assertEqual(99, getargs_L(Int())) + self.assertEqual(0, getargs_L(IntSubclass())) + self.assertRaises(TypeError, getargs_L, BadInt()) + self.assertEqual(1, getargs_L(BadInt2())) + self.assertEqual(0, getargs_L(BadInt3())) self.assertRaises(OverflowError, getargs_L, LLONG_MIN-1) self.assertEqual(LLONG_MIN, getargs_L(LLONG_MIN)) @@ -206,6 +263,11 @@ class LongLong_TestCase(unittest.TestCase): # K return 'unsigned long long', no range checking self.assertRaises(TypeError, getargs_K, 3.14) self.assertRaises(TypeError, getargs_K, Int()) + self.assertEqual(0, getargs_K(IntSubclass())) + self.assertRaises(TypeError, getargs_K, BadInt()) + self.assertRaises(TypeError, getargs_K, BadInt2()) + self.assertEqual(0, getargs_K(BadInt3())) + self.assertEqual(ULLONG_MAX, getargs_K(ULLONG_MAX)) self.assertEqual(0, getargs_K(0)) self.assertEqual(0, getargs_K(ULLONG_MAX+1)) diff --git a/Lib/test/test_grp.py b/Lib/test/test_grp.py index 04a8af6ac9b..749041c2810 100644 --- a/Lib/test/test_grp.py +++ b/Lib/test/test_grp.py @@ -26,8 +26,10 @@ class GroupDatabaseTestCase(unittest.TestCase): for e in entries: self.check_value(e) + def test_values_extended(self): + entries = grp.getgrall() if len(entries) > 1000: # Huge group file (NIS?) -- skip the rest - return + self.skipTest('huge group file, extended test skipped') for e in entries: e2 = grp.getgrgid(e.gr_gid) diff --git a/Lib/test/test_imp.py b/Lib/test/test_imp.py index b56efe3bea1..71c220f03e6 100644 --- a/Lib/test/test_imp.py +++ b/Lib/test/test_imp.py @@ -245,7 +245,7 @@ class ImportTests(unittest.TestCase): if found[0] is not None: found[0].close() if found[2][2] != imp.C_EXTENSION: - return + self.skipTest("found module doesn't appear to be a C extension") imp.load_module(name, None, *found[1:]) def test_multiple_calls_to_get_data(self): diff --git a/Lib/test/test_index.py b/Lib/test/test_index.py index 66eedaaed72..87857119e1d 100644 --- a/Lib/test/test_index.py +++ b/Lib/test/test_index.py @@ -9,7 +9,7 @@ class newstyle: class TrapInt(int): def __index__(self): - return self + return int(self) class BaseTestCase(unittest.TestCase): def setUp(self): @@ -55,6 +55,39 @@ class BaseTestCase(unittest.TestCase): self.assertRaises(TypeError, slice(self.o).indices, 0) self.assertRaises(TypeError, slice(self.n).indices, 0) + def test_int_subclass_with_index(self): + # __index__ should be used when computing indices, even for int + # subclasses. See issue #17576. + class MyInt(int): + def __index__(self): + return int(self) + 1 + + my_int = MyInt(7) + direct_index = my_int.__index__() + operator_index = operator.index(my_int) + self.assertEqual(direct_index, 8) + self.assertEqual(operator_index, 7) + # Both results should be of exact type int. + self.assertIs(type(direct_index), int) + #self.assertIs(type(operator_index), int) + + def test_index_returns_int_subclass(self): + class BadInt: + def __index__(self): + return True + + class BadInt2(int): + def __index__(self): + return True + + bad_int = BadInt() + n = operator.index(bad_int) + self.assertEqual(n, 1) + + bad_int = BadInt2() + n = operator.index(bad_int) + self.assertEqual(n, 0) + class SeqTestCase: # This test case isn't run directly. It just defines common tests diff --git a/Lib/test/test_int.py b/Lib/test/test_int.py index c198bcc7401..c489510537e 100644 --- a/Lib/test/test_int.py +++ b/Lib/test/test_int.py @@ -263,32 +263,7 @@ class IntTestCases(unittest.TestCase): def __int__(self): return 42 - class Foo1(object): - def __int__(self): - return 42 - - class Foo2(int): - def __int__(self): - return 42 - - class Foo3(int): - def __int__(self): - return self - - class Foo4(int): - def __int__(self): - return 42 - - class Foo5(int): - def __int__(self): - return 42. - self.assertEqual(int(Foo0()), 42) - self.assertEqual(int(Foo1()), 42) - self.assertEqual(int(Foo2()), 42) - self.assertEqual(int(Foo3()), 0) - self.assertEqual(int(Foo4()), 42) - self.assertRaises(TypeError, int, Foo5()) class Classic: pass @@ -351,6 +326,54 @@ class IntTestCases(unittest.TestCase): with self.assertRaises(TypeError): int(TruncReturnsBadInt()) + def test_int_subclass_with_int(self): + class MyInt(int): + def __int__(self): + return 42 + + class BadInt(int): + def __int__(self): + return 42.0 + + my_int = MyInt(7) + self.assertEqual(my_int, 7) + self.assertEqual(int(my_int), 42) + + self.assertRaises(TypeError, int, BadInt()) + + def test_int_returns_int_subclass(self): + class BadInt: + def __int__(self): + return True + + class BadInt2(int): + def __int__(self): + return True + + class TruncReturnsBadInt: + def __trunc__(self): + return BadInt() + + class TruncReturnsIntSubclass: + def __trunc__(self): + return True + + bad_int = BadInt() + n = int(bad_int) + self.assertEqual(n, 1) + + bad_int = BadInt2() + n = int(bad_int) + self.assertEqual(n, 1) + + bad_int = TruncReturnsBadInt() + n = int(bad_int) + self.assertEqual(n, 1) + + good_int = TruncReturnsIntSubclass() + n = int(good_int) + self.assertEqual(n, 1) + def test_error_message(self): def check(s, base=None): with self.assertRaises(ValueError, diff --git a/Lib/test/test_io.py b/Lib/test/test_io.py index 9b8920211e3..e843cacae15 100644 --- a/Lib/test/test_io.py +++ b/Lib/test/test_io.py @@ -421,14 +421,9 @@ class IOTest(unittest.TestCase): # a long time to build the >2GB file and takes >2GB of disk space # therefore the resource must be enabled to run this test. if sys.platform[:3] == 'win' or sys.platform == 'darwin': - if not support.is_resource_enabled("largefile"): - print("\nTesting large file ops skipped on %s." % sys.platform, - file=sys.stderr) - print("It requires %d bytes and a long time." % self.LARGE, - file=sys.stderr) - print("Use 'regrtest.py -u largefile test_io' to run it.", - file=sys.stderr) - return + support.requires( + 'largefile', + 'test requires %s bytes and a long time to run' % self.LARGE) with self.open(support.TESTFN, "w+b", 0) as f: self.large_file_ops(f) with self.open(support.TESTFN, "w+b") as f: @@ -698,6 +693,7 @@ class CommonBufferedTests: self.assertEqual(42, bufio.fileno()) + @unittest.skip('test having existential crisis') def test_no_fileno(self): # XXX will we always have fileno() function? If so, kill # this test. Else, write it. diff --git a/Lib/test/test_memoryview.py b/Lib/test/test_memoryview.py index ee6b15ac148..bf0eaad2acf 100644 --- a/Lib/test/test_memoryview.py +++ b/Lib/test/test_memoryview.py @@ -57,7 +57,7 @@ class AbstractMemoryTests: def test_setitem_readonly(self): if not self.ro_type: - return + self.skipTest("no read-only type to test") b = self.ro_type(self._source) oldrefcount = sys.getrefcount(b) m = self._view(b) @@ -71,7 +71,7 @@ class AbstractMemoryTests: def test_setitem_writable(self): if not self.rw_type: - return + self.skipTest("no writable type to test") tp = self.rw_type b = self.rw_type(self._source) oldrefcount = sys.getrefcount(b) @@ -189,13 +189,13 @@ class AbstractMemoryTests: def test_attributes_readonly(self): if not self.ro_type: - return + self.skipTest("no read-only type to test") m = self.check_attributes_with_type(self.ro_type) self.assertEqual(m.readonly, True) def test_attributes_writable(self): if not self.rw_type: - return + self.skipTest("no writable type to test") m = self.check_attributes_with_type(self.rw_type) self.assertEqual(m.readonly, False) @@ -301,7 +301,7 @@ class AbstractMemoryTests: # buffer as writable causing a segfault if using mmap tp = self.ro_type if tp is None: - return + self.skipTest("no read-only type to test") b = tp(self._source) m = self._view(b) i = io.BytesIO(b'ZZZZ') @@ -370,12 +370,12 @@ class BaseArrayMemoryTests(AbstractMemoryTests): itemsize = array.array('i').itemsize format = 'i' + @unittest.skip('XXX test should be adapted for non-byte buffers') def test_getbuffer(self): - # XXX Test should be adapted for non-byte buffers pass + @unittest.skip('XXX NotImplementedError: tolist() only supports byte views') def test_tolist(self): - # XXX NotImplementedError: tolist() only supports byte views pass diff --git a/Lib/test/test_multiprocessing.py b/Lib/test/test_multiprocessing.py index 27db029b3c6..d20a57698d2 100644 --- a/Lib/test/test_multiprocessing.py +++ b/Lib/test/test_multiprocessing.py @@ -195,7 +195,7 @@ class _TestProcess(BaseTestCase): def test_current(self): if self.TYPE == 'threads': - return + self.skipTest('test not appropriate for {}'.format(self.TYPE)) current = self.current_process() authkey = current.authkey @@ -209,7 +209,7 @@ class _TestProcess(BaseTestCase): def test_daemon_argument(self): if self.TYPE == "threads": - return + self.skipTest('test not appropriate for {}'.format(self.TYPE)) # By default uses the current process's daemon flag. proc0 = self.Process(target=self._test) @@ -274,7 +274,7 @@ class _TestProcess(BaseTestCase): def test_terminate(self): if self.TYPE == 'threads': - return + self.skipTest('test not appropriate for {}'.format(self.TYPE)) p = self.Process(target=self._test_terminate) p.daemon = True @@ -378,7 +378,7 @@ class _TestProcess(BaseTestCase): def test_sentinel(self): if self.TYPE == "threads": - return + self.skipTest('test not appropriate for {}'.format(self.TYPE)) event = self.Event() p = self.Process(target=self._test_sentinel, args=(event,)) with self.assertRaises(ValueError): @@ -434,7 +434,7 @@ class _TestSubclassingProcess(BaseTestCase): def test_stderr_flush(self): # sys.stderr is flushed at process shutdown (issue #13812) if self.TYPE == "threads": - return + self.skipTest('test not appropriate for {}'.format(self.TYPE)) testfn = test.support.TESTFN self.addCleanup(test.support.unlink, testfn) @@ -462,7 +462,7 @@ class _TestSubclassingProcess(BaseTestCase): def test_sys_exit(self): # See Issue 13854 if self.TYPE == 'threads': - return + self.skipTest('test not appropriate for {}'.format(self.TYPE)) testfn = test.support.TESTFN self.addCleanup(test.support.unlink, testfn) @@ -671,7 +671,7 @@ class _TestQueue(BaseTestCase): try: self.assertEqual(q.qsize(), 0) except NotImplementedError: - return + self.skipTest('qsize method not implemented') q.put(1) self.assertEqual(q.qsize(), 1) q.put(5) @@ -779,7 +779,7 @@ class _TestSemaphore(BaseTestCase): def test_timeout(self): if self.TYPE != 'processes': - return + self.skipTest('test not appropriate for {}'.format(self.TYPE)) sem = self.Semaphore(0) acquire = TimingWrapper(sem.acquire) @@ -1399,7 +1399,7 @@ class _TestBarrier(BaseTestCase): def test_thousand(self): if self.TYPE == 'manager': - return + self.skipTest('test not appropriate for {}'.format(self.TYPE)) passes = 1000 lock = self.Lock() conn, child_conn = self.Pipe(False) @@ -1694,7 +1694,7 @@ class _TestPool(BaseTestCase): def test_map_unplicklable(self): # Issue #19425 -- failure to pickle should not cause a hang if self.TYPE == 'threads': - return + self.skipTest('test not appropriate for {}'.format(self.TYPE)) class A(object): def __reduce__(self): raise RuntimeError('cannot pickle') @@ -2188,7 +2188,7 @@ class _TestConnection(BaseTestCase): def test_sendbytes(self): if self.TYPE != 'processes': - return + self.skipTest('test not appropriate for {}'.format(self.TYPE)) msg = latin('abcdefghijklmnopqrstuvwxyz') a, b = self.Pipe() diff --git a/Lib/test/test_nis.py b/Lib/test/test_nis.py index 830c24d7813..a3a3c260e37 100644 --- a/Lib/test/test_nis.py +++ b/Lib/test/test_nis.py @@ -12,11 +12,7 @@ class NisTests(unittest.TestCase): maps = nis.maps() except nis.error as msg: # NIS is probably not active, so this test isn't useful - if support.verbose: - print("Test Skipped:", msg) - # Can't raise SkipTest as regrtest only recognizes the exception - # import time. - return + self.skipTest(str(msg)) try: # On some systems, this map is only accessible to the # super user diff --git a/Lib/test/test_ntpath.py b/Lib/test/test_ntpath.py index f8098767cea..309d59aba39 100644 --- a/Lib/test/test_ntpath.py +++ b/Lib/test/test_ntpath.py @@ -218,7 +218,7 @@ class TestNtpath(unittest.TestCase): import nt tester('ntpath.abspath("C:\\")', "C:\\") except ImportError: - pass + self.skipTest('nt module not available') def test_relpath(self): currentdir = os.path.split(os.getcwd())[-1] diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index 601c6b2e97c..b5bfe9433bb 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -263,7 +263,7 @@ class StatAttributeTests(unittest.TestCase): except OSError as e: # On AtheOS, glibc always returns ENOSYS if e.errno == errno.ENOSYS: - return + self.skipTest('glibc always returns ENOSYS on AtheOS') # Make sure direct access works self.assertEqual(result.f_bfree, result[3]) @@ -480,7 +480,7 @@ class StatAttributeTests(unittest.TestCase): os.stat(r"c:\pagefile.sys") except WindowsError as e: if e.errno == 2: # file does not exist; cannot run test - return + self.skipTest(r'c:\pagefile.sys does not exist') self.fail("Could not stat pagefile.sys") @unittest.skipUnless(sys.platform == "win32", "Win32 specific tests") diff --git a/Lib/test/test_platform.py b/Lib/test/test_platform.py index 0dcfe0504a5..b3de43b7abb 100644 --- a/Lib/test/test_platform.py +++ b/Lib/test/test_platform.py @@ -1,7 +1,9 @@ +from unittest import mock import os import platform import subprocess import sys +import tempfile import unittest import warnings @@ -295,6 +297,19 @@ class PlatformTest(unittest.TestCase): returncode = ret >> 8 self.assertEqual(returncode, len(data)) + def test_linux_distribution_encoding(self): + # Issue #17429 + with tempfile.TemporaryDirectory() as tempdir: + filename = os.path.join(tempdir, 'fedora-release') + with open(filename, 'w', encoding='utf-8') as f: + f.write('Fedora release 19 (Schr\xf6dinger\u2019s Cat)\n') + + with mock.patch('platform._UNIXCONFDIR', tempdir): + distname, version, distid = platform.linux_distribution() + + self.assertEqual(distname, 'Fedora') + self.assertEqual(version, '19') + self.assertEqual(distid, 'Schr\xf6dinger\u2019s Cat') def test_main(): support.run_unittest( diff --git a/Lib/test/test_poll.py b/Lib/test/test_poll.py index f98a280e9a5..3c86ef639f6 100644 --- a/Lib/test/test_poll.py +++ b/Lib/test/test_poll.py @@ -3,7 +3,7 @@ import os import random import select -import _testcapi +from _testcapi import USHRT_MAX, INT_MAX, UINT_MAX try: import threading except ImportError: @@ -159,14 +159,13 @@ class PollTests(unittest.TestCase): if x != 5: self.fail('Overflow must have occurred') - pollster = select.poll() - # Issue 15989 - self.assertRaises(OverflowError, pollster.register, 0, - _testcapi.SHRT_MAX + 1) - self.assertRaises(OverflowError, pollster.register, 0, - _testcapi.USHRT_MAX + 1) - self.assertRaises(OverflowError, pollster.poll, _testcapi.INT_MAX + 1) - self.assertRaises(OverflowError, pollster.poll, _testcapi.UINT_MAX + 1) + # Issues #15989, #17919 + self.assertRaises(OverflowError, pollster.register, 0, -1) + self.assertRaises(OverflowError, pollster.register, 0, USHRT_MAX + 1) + self.assertRaises(OverflowError, pollster.modify, 1, -1) + self.assertRaises(OverflowError, pollster.modify, 1, USHRT_MAX + 1) + self.assertRaises(OverflowError, pollster.poll, INT_MAX + 1) + self.assertRaises(OverflowError, pollster.poll, UINT_MAX + 1) @unittest.skipUnless(threading, 'Threading required for this test.') @reap_threads diff --git a/Lib/test/test_pwd.py b/Lib/test/test_pwd.py index aa8f69f69bf..37a1bcb28b8 100644 --- a/Lib/test/test_pwd.py +++ b/Lib/test/test_pwd.py @@ -8,8 +8,6 @@ class PwdTest(unittest.TestCase): def test_values(self): entries = pwd.getpwall() - entriesbyname = {} - entriesbyuid = {} for e in entries: self.assertEqual(len(e), 7) @@ -32,13 +30,20 @@ class PwdTest(unittest.TestCase): # for one uid # self.assertEqual(pwd.getpwuid(e.pw_uid), e) # instead of this collect all entries for one uid - # and check afterwards + # and check afterwards (done in test_values_extended) + + def test_values_extended(self): + entries = pwd.getpwall() + entriesbyname = {} + entriesbyuid = {} + + if len(entries) > 1000: # Huge passwd file (NIS?) -- skip this test + self.skipTest('passwd file is huge; extended test skipped') + + for e in entries: entriesbyname.setdefault(e.pw_name, []).append(e) entriesbyuid.setdefault(e.pw_uid, []).append(e) - if len(entries) > 1000: # Huge passwd file (NIS?) -- skip the rest - return - # check whether the entry returned by getpwuid() # for each uid is among those from getpwall() for this uid for e in entries: diff --git a/Lib/test/test_reprlib.py b/Lib/test/test_reprlib.py index 589ecdd4daf..c233b1d67be 100644 --- a/Lib/test/test_reprlib.py +++ b/Lib/test/test_reprlib.py @@ -167,8 +167,15 @@ class ReprTests(unittest.TestCase): eq(r([[[[[[[{}]]]]]]]), "[[[[[[[...]]]]]]]") def test_cell(self): - # XXX Hmm? How to get at a cell object? - pass + def get_cell(): + x = 42 + def inner(): + return x + return inner + x = get_cell().__closure__[0] + self.assertRegex(repr(x), r'') + self.assertRegex(r(x), r'') def test_descriptors(self): eq = self.assertEqual @@ -272,6 +279,7 @@ class foo(object): eq(repr(foo.foo), "" % foo.__name__) + @unittest.skip('need a suitable object') def test_object(self): # XXX Test the repr of a type with a really long tp_name but with no # tp_repr. WIBNI we had ::Inline? :) @@ -319,6 +327,7 @@ class aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa '= 0, "Error resolving host to ip.") try: hname, aliases, ipaddrs = socket.gethostbyaddr(ip) except socket.error: # Probably a similar problem as above; skip this test - return + self.skipTest('name lookup failure') all_host_names = [hostname, hname] + aliases fqhn = socket.getfqdn(ip) if not fqhn in all_host_names: @@ -932,9 +932,9 @@ class GeneralModuleTests(unittest.TestCase): try: from socket import inet_pton, AF_INET6, has_ipv6 if not has_ipv6: - return + self.skipTest('IPv6 not available') except ImportError: - return + self.skipTest('could not import needed symbols from socket') f = lambda a: inet_pton(AF_INET6, a) assertInvalid = lambda a: self.assertRaises( (socket.error, ValueError), f, a @@ -1010,9 +1010,9 @@ class GeneralModuleTests(unittest.TestCase): try: from socket import inet_ntop, AF_INET6, has_ipv6 if not has_ipv6: - return + self.skipTest('IPv6 not available') except ImportError: - return + self.skipTest('could not import needed symbols from socket') f = lambda a: inet_ntop(AF_INET6, a) assertInvalid = lambda a: self.assertRaises( (socket.error, ValueError), f, a @@ -1045,7 +1045,7 @@ class GeneralModuleTests(unittest.TestCase): my_ip_addr = socket.gethostbyname(socket.gethostname()) except socket.error: # Probably name lookup wasn't set up right; skip this test - return + self.skipTest('name lookup failure') self.assertIn(name[0], ("0.0.0.0", my_ip_addr), '%s invalid' % name[0]) self.assertEqual(name[1], port) diff --git a/Lib/test/test_strptime.py b/Lib/test/test_strptime.py index d5bc39d0dd3..13a72d450d8 100644 --- a/Lib/test/test_strptime.py +++ b/Lib/test/test_strptime.py @@ -323,7 +323,7 @@ class StrptimeTests(unittest.TestCase): # when time.tzname[0] == time.tzname[1] and time.daylight tz_name = time.tzname[0] if tz_name.upper() in ("UTC", "GMT"): - return + self.skipTest('need non-UTC/GMT timezone') try: original_tzname = time.tzname original_daylight = time.daylight @@ -536,7 +536,7 @@ class CacheTests(unittest.TestCase): try: locale.setlocale(locale.LC_TIME, ('en_US', 'UTF8')) except locale.Error: - return + self.skipTest('test needs en_US.UTF8 locale') try: _strptime._strptime_time('10', '%d') # Get id of current cache object. @@ -553,7 +553,7 @@ class CacheTests(unittest.TestCase): # If this is the case just suppress the exception and fall-through # to the resetting to the original locale. except locale.Error: - pass + self.skipTest('test needs de_DE.UTF8 locale') # Make sure we don't trample on the locale setting once we leave the # test. finally: diff --git a/Lib/test/test_sunau.py b/Lib/test/test_sunau.py index 767314f5b27..1655317c07e 100644 --- a/Lib/test/test_sunau.py +++ b/Lib/test/test_sunau.py @@ -5,10 +5,12 @@ import sys import sunau -class SunauPCM8Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): +class SunauTest(audiotests.AudioWriteTests, + audiotests.AudioTestsWithSourceFile): module = sunau + + +class SunauPCM8Test(SunauTest, unittest.TestCase): sndfilename = 'pluck-pcm8.au' sndfilenframes = 3307 nchannels = 2 @@ -25,10 +27,7 @@ class SunauPCM8Test(audiotests.AudioWriteTests, """) -class SunauPCM16Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = sunau +class SunauPCM16Test(SunauTest, unittest.TestCase): sndfilename = 'pluck-pcm16.au' sndfilenframes = 3307 nchannels = 2 @@ -47,10 +46,7 @@ class SunauPCM16Test(audiotests.AudioWriteTests, """) -class SunauPCM32Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = sunau +class SunauPCM32Test(SunauTest, unittest.TestCase): sndfilename = 'pluck-pcm32.au' sndfilenframes = 3307 nchannels = 2 @@ -75,10 +71,7 @@ class SunauPCM32Test(audiotests.AudioWriteTests, """) -class SunauULAWTest(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = sunau +class SunauULAWTest(SunauTest, unittest.TestCase): sndfilename = 'pluck-ulaw.au' sndfilenframes = 3307 nchannels = 2 diff --git a/Lib/test/test_tempfile.py b/Lib/test/test_tempfile.py index 4ff56b8bf66..9c4c158b22b 100644 --- a/Lib/test/test_tempfile.py +++ b/Lib/test/test_tempfile.py @@ -324,10 +324,9 @@ class TestMkstempInner(BaseTestCase): finally: os.rmdir(dir) + @unittest.skipUnless(has_stat, 'os.stat not available') def test_file_mode(self): # _mkstemp_inner creates files with the proper mode - if not has_stat: - return # ugh, can't use SkipTest. file = self.do_create() mode = stat.S_IMODE(os.stat(file.name).st_mode) @@ -339,10 +338,9 @@ class TestMkstempInner(BaseTestCase): expected = user * (1 + 8 + 64) self.assertEqual(mode, expected) + @unittest.skipUnless(has_spawnl, 'os.spawnl not available') def test_noinherit(self): # _mkstemp_inner file handles are not inherited by child processes - if not has_spawnl: - return # ugh, can't use SkipTest. if support.verbose: v="v" @@ -377,10 +375,9 @@ class TestMkstempInner(BaseTestCase): "child process caught fatal signal %d" % -retval) self.assertFalse(retval > 0, "child process reports failure %d"%retval) + @unittest.skipUnless(has_textmode, "text mode not available") def test_textmode(self): # _mkstemp_inner can create files in text mode - if not has_textmode: - return # ugh, can't use SkipTest. # A text file is truncated at the first Ctrl+Z byte f = self.do_create(bin=0) @@ -556,10 +553,9 @@ class TestMkdtemp(BaseTestCase): finally: os.rmdir(dir) + @unittest.skipUnless(has_stat, 'os.stat not available') def test_mode(self): # mkdtemp creates directories with the proper mode - if not has_stat: - return # ugh, can't use SkipTest. dir = self.do_create() try: diff --git a/Lib/test/test_thread.py b/Lib/test/test_thread.py index a191e157bc3..54372813b9d 100644 --- a/Lib/test/test_thread.py +++ b/Lib/test/test_thread.py @@ -68,39 +68,35 @@ class ThreadRunningTests(BasicThreadTest): thread.stack_size(0) self.assertEqual(thread.stack_size(), 0, "stack_size not reset to default") - if os.name not in ("nt", "os2", "posix"): - return - - tss_supported = True + @unittest.skipIf(os.name not in ("nt", "os2", "posix"), 'test meant for nt, os2, and posix') + def test_nt_and_posix_stack_size(self): try: thread.stack_size(4096) except ValueError: verbose_print("caught expected ValueError setting " "stack_size(4096)") except thread.error: - tss_supported = False - verbose_print("platform does not support changing thread stack " - "size") + self.skipTest("platform does not support changing thread stack " + "size") - if tss_supported: - fail_msg = "stack_size(%d) failed - should succeed" - for tss in (262144, 0x100000, 0): - thread.stack_size(tss) - self.assertEqual(thread.stack_size(), tss, fail_msg % tss) - verbose_print("successfully set stack_size(%d)" % tss) + fail_msg = "stack_size(%d) failed - should succeed" + for tss in (262144, 0x100000, 0): + thread.stack_size(tss) + self.assertEqual(thread.stack_size(), tss, fail_msg % tss) + verbose_print("successfully set stack_size(%d)" % tss) - for tss in (262144, 0x100000): - verbose_print("trying stack_size = (%d)" % tss) - self.next_ident = 0 - self.created = 0 - for i in range(NUMTASKS): - self.newtask() + for tss in (262144, 0x100000): + verbose_print("trying stack_size = (%d)" % tss) + self.next_ident = 0 + self.created = 0 + for i in range(NUMTASKS): + self.newtask() - verbose_print("waiting for all tasks to complete") - self.done_mutex.acquire() - verbose_print("all tasks done") + verbose_print("waiting for all tasks to complete") + self.done_mutex.acquire() + verbose_print("all tasks done") - thread.stack_size(0) + thread.stack_size(0) def test__count(self): # Test the _count() function. diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index 0ebeb39cbdc..fee48a39102 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -17,6 +17,10 @@ import weakref import os from test.script_helper import assert_python_ok, assert_python_failure import subprocess +try: + import _testcapi +except ImportError: + _testcapi = None from test import lock_tests @@ -769,6 +773,45 @@ class ThreadJoinOnShutdown(BaseTestCase): for t in threads: t.join() + @unittest.skipIf(_testcapi is None, "need _testcapi module") + def test_frame_tstate_tracing(self): + # Issue #14432: Crash when a generator is created in a C thread that is + # destroyed while the generator is still used. The issue was that a + # generator contains a frame, and the frame kept a reference to the + # Python state of the destroyed C thread. The crash occurs when a trace + # function is setup. + + def noop_trace(frame, event, arg): + # no operation + return noop_trace + + def generator(): + while 1: + yield "genereator" + + def callback(): + if callback.gen is None: + callback.gen = generator() + return next(callback.gen) + callback.gen = None + + old_trace = sys.gettrace() + sys.settrace(noop_trace) + try: + # Install a trace function + threading.settrace(noop_trace) + + # Create a generator in a C thread which exits after the call + _testcapi.call_in_temporary_c_thread(callback) + + # Call the generator in a different Python thread, check that the + # generator didn't keep a reference to the destroyed thread state + for test in range(3): + # The trace function is still called here + callback() + finally: + sys.settrace(old_trace) + class ThreadingExceptionTests(BaseTestCase): # A RuntimeError should be raised if Thread.start() is called diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py index d8d6029ea5a..2997077bcbb 100644 --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -463,8 +463,7 @@ class TestLocale(unittest.TestCase): try: tmp = locale.setlocale(locale.LC_ALL, "fr_FR") except locale.Error: - # skip this test - return + self.skipTest('could not set locale.LC_ALL to fr_FR') # This should not cause an exception time.strftime("%B", (2009,2,1,0,0,0,0,0,0)) diff --git a/Lib/test/test_unicode.py b/Lib/test/test_unicode.py index 502d39368d6..f6b2a3fd719 100644 --- a/Lib/test/test_unicode.py +++ b/Lib/test/test_unicode.py @@ -1952,12 +1952,12 @@ class UnicodeTest(string_tests.CommonTest, self.assertEqual(repr('\U00010000'), "'%c'" % (0x10000,)) # printable self.assertEqual(repr('\U00014000'), "'\\U00014000'") # nonprintable + # This test only affects 32-bit platforms because expandtabs can only take + # an int as the max value, not a 64-bit C long. If expandtabs is changed + # to take a 64-bit long, this test should apply to all platforms. + @unittest.skipIf(sys.maxsize > (1 << 32) or struct.calcsize('P') != 4, + 'only applies to 32-bit platforms') def test_expandtabs_overflows_gracefully(self): - # This test only affects 32-bit platforms because expandtabs can only take - # an int as the max value, not a 64-bit C long. If expandtabs is changed - # to take a 64-bit long, this test should apply to all platforms. - if sys.maxsize > (1 << 32) or struct.calcsize('P') != 4: - return self.assertRaises(OverflowError, 't\tt\t'.expandtabs, sys.maxsize) @support.cpython_only diff --git a/Lib/test/test_urllibnet.py b/Lib/test/test_urllibnet.py index 20efca6adbf..9aa16b4c9ce 100644 --- a/Lib/test/test_urllibnet.py +++ b/Lib/test/test_urllibnet.py @@ -98,11 +98,10 @@ class urlopenNetworkTests(unittest.TestCase): open_url.close() self.assertEqual(code, 404) + # On Windows, socket handles are not file descriptors; this + # test can't pass on Windows. + @unittest.skipIf(sys.platform in ('win32',), 'not appropriate for Windows') def test_fileno(self): - if sys.platform in ('win32',): - # On Windows, socket handles are not file descriptors; this - # test can't pass on Windows. - return # Make sure fd returned by fileno is valid. with self.urlopen("http://www.python.org/", timeout=None) as open_url: fd = open_url.fileno() diff --git a/Lib/test/test_warnings.py b/Lib/test/test_warnings.py index 9f6d775c590..6ce0b23474b 100644 --- a/Lib/test/test_warnings.py +++ b/Lib/test/test_warnings.py @@ -682,7 +682,7 @@ class CatchWarningTests(BaseTest): # Explicit tests for the test.support convenience wrapper wmod = self.module if wmod is not sys.modules['warnings']: - return + self.skipTest('module to test is not loaded warnings module') with support.check_warnings(quiet=False) as w: self.assertEqual(w.warnings, []) wmod.simplefilter("always") diff --git a/Lib/test/test_wave.py b/Lib/test/test_wave.py index 5be12519d9e..f64201f9fc6 100644 --- a/Lib/test/test_wave.py +++ b/Lib/test/test_wave.py @@ -5,10 +5,15 @@ import sys import wave -class WavePCM8Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): +class WaveTest(audiotests.AudioWriteTests, + audiotests.AudioTestsWithSourceFile): module = wave + test_unseekable_write = None + test_unseekable_overflowed_write = None + test_unseekable_incompleted_write = None + + +class WavePCM8Test(WaveTest, unittest.TestCase): sndfilename = 'pluck-pcm8.wav' sndfilenframes = 3307 nchannels = 2 @@ -25,10 +30,7 @@ class WavePCM8Test(audiotests.AudioWriteTests, """) -class WavePCM16Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = wave +class WavePCM16Test(WaveTest, unittest.TestCase): sndfilename = 'pluck-pcm16.wav' sndfilenframes = 3307 nchannels = 2 @@ -55,10 +57,7 @@ class WavePCM16Test(audiotests.AudioWriteTests, -class WavePCM24Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = wave +class WavePCM24Test(WaveTest, unittest.TestCase): sndfilename = 'pluck-pcm24.wav' sndfilenframes = 3307 nchannels = 2 @@ -85,10 +84,7 @@ class WavePCM24Test(audiotests.AudioWriteTests, frames = audiotests.byteswap3(frames) -class WavePCM32Test(audiotests.AudioWriteTests, - audiotests.AudioTestsWithSourceFile, - unittest.TestCase): - module = wave +class WavePCM32Test(WaveTest, unittest.TestCase): sndfilename = 'pluck-pcm32.wav' sndfilenframes = 3307 nchannels = 2 diff --git a/Lib/test/test_xmlrpc_net.py b/Lib/test/test_xmlrpc_net.py index dfb5f9aa3d9..4c6f1ec4e95 100644 --- a/Lib/test/test_xmlrpc_net.py +++ b/Lib/test/test_xmlrpc_net.py @@ -20,7 +20,6 @@ class CurrentTimeTest(unittest.TestCase): t0 = server.currentTime.getCurrentTime() except socket.error as e: self.skipTest("network error: %s" % e) - return # Perform a minimal sanity check on the result, just to be sure # the request means what we think it means. @@ -44,7 +43,6 @@ class CurrentTimeTest(unittest.TestCase): builders = server.getAllBuilders() except socket.error as e: self.skipTest("network error: %s" % e) - return self.addCleanup(lambda: server('close')()) # Perform a minimal sanity check on the result, just to be sure diff --git a/Lib/test/test_zipimport.py b/Lib/test/test_zipimport.py index f7cb8b977ff..37603b9bcde 100644 --- a/Lib/test/test_zipimport.py +++ b/Lib/test/test_zipimport.py @@ -111,7 +111,7 @@ class UncompressedZipImportTestCase(ImportHooksBaseTestCase): # so we'll simply skip it then. Bug #765456. # if "zlib" in sys.builtin_module_names: - return + self.skipTest('zlib is a builtin module') if "zlib" in sys.modules: del sys.modules["zlib"] files = {"zlib.py": (NOW, test_src)} diff --git a/Misc/ACKS b/Misc/ACKS index 2c957e3c1f2..486e2affdee 100644 --- a/Misc/ACKS +++ b/Misc/ACKS @@ -689,6 +689,7 @@ Steven Kryskalla Andrew Kuchling Dave Kuhlman Jon Kuhn +Toshio Kuratomi Vladimir Kushnir Erno Kuusela Ross Lagerwall diff --git a/Misc/NEWS b/Misc/NEWS index 623e1fe6b97..60e741f8dd1 100644 --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,17 @@ What's New in Python 3.3.4 release candidate 1? Core and Builtins ----------------- +- Issue #19969: PyBytes_FromFormatV() now raises an OverflowError if "%c" + argument is not in range [0; 255]. + +- Issue #14432: Generator now clears the borrowed reference to the thread + state. Fix a crash when a generator is created in a C thread that is + destroyed while the generator is still used. The issue was that a generator + contains a frame, and the frame kept a reference to the Python state of the + destroyed C thread. The crash occurs when a trace function is setup. + +- Issue #19932: Fix typo in import.h, missing whitespaces in function prototypes. + - Issue #19729: In str.format(), fix recursive expansion in format spec. - Issue #19638: Fix possible crash / undefined behaviour from huge (more than 2 @@ -18,6 +29,34 @@ Core and Builtins Library ------- +- Issue #19623: Fixed writing to unseekable files in the aifc module. + +- Issue #17919: select.poll.register() again works with poll.POLLNVAL on AIX. + Fixed integer overflow in the eventmask parameter. + +- Issue #19063: if a Charset's body_encoding was set to None, the email + package would generate a message claiming the Content-Transfer-Encoding + was 7bit, and produce garbage output for the content. This now works. + A couple of other set_payload mishandlings of non-ASCII are also fixed. + +- Issue #17200: telnetlib's read_until and expect timeout was broken by the + fix to Issue #14635 in Python 3.3.0 to be interpreted as milliseconds + instead of seconds when the platform supports select.poll (ie: everywhere). + It is now treated as seconds once again. + +- Issue #17429: platform.linux_distribution() now decodes files from the UTF-8 + encoding with the surrogateescape error handler, instead of decoding from the + locale encoding in strict mode. It fixes the function on Fedora 19 which is + probably the first major distribution release with a non-ASCII name. Patch + written by Toshio Kuratomi. + +- Issue #19929: Call os.read with 32768 within subprocess.Popen.communicate + rather than 4096 for efficiency. A microbenchmark shows Linux and OS X + both using ~50% less cpu time this way. + +- Issue #19506: Use a memoryview to avoid a data copy when piping data + to stdin within subprocess.Popen.communicate. 5-10% less cpu usage. + - Issue #19839: Fix regression in bz2 module's handling of non-bzip2 data at EOF, and analogous bug in lzma module. @@ -91,9 +130,26 @@ Library - Issue #19286: Directories in ``package_data`` are no longer added to the filelist, preventing failure outlined in the ticket. +IDLE +---- + +- Issue #19481: print() of string subclass instance in IDLE no longer hangs. + +- Issue #18270: Prevent possible IDLE AttributeError on OS X when no initial + shell window is present. + Tests ----- +- Issue #19828: Fixed test_site when the whole suite is run with -S. + +- Issue #19928: Implemented a test for repr() of cell objects. + +- Issue #19535: Fixed test_docxmlrpc when python is run with -OO. + +- Issue #19926: Removed unneeded test_main from test_abstract_numbers. + Patch by Vajrasky Kok. + - Issue #19595: Re-enabled a long-disabled test in test_winsound. - Issue #19588: Fixed tests in test_random that were silently skipped most @@ -106,13 +162,16 @@ Tests import, converting from test_main to unittest.main, and running the _testcapi module tests within a unittest TestCase. -- Issue #18702: All skipped tests now reported as skipped. +- Issue #18702, 19572: All skipped tests now reported as skipped. - Issue #19085: Added basic tests for all tkinter widget options. Documentation ------------- +- Issue #19963: Document that importlib.import_module() no longer requires + importing parent packages separately. + - Issue #18840: Introduce the json module in the tutorial, and deemphasize the pickle module. diff --git a/Modules/_decimal/libmpdec/basearith.c b/Modules/_decimal/libmpdec/basearith.c index dd21a7a885f..35de6b82849 100644 --- a/Modules/_decimal/libmpdec/basearith.c +++ b/Modules/_decimal/libmpdec/basearith.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/basearith.h b/Modules/_decimal/libmpdec/basearith.h index 114cef38603..976358a110e 100644 --- a/Modules/_decimal/libmpdec/basearith.h +++ b/Modules/_decimal/libmpdec/basearith.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/bits.h b/Modules/_decimal/libmpdec/bits.h index 949ec944ca5..b5eaa24976a 100644 --- a/Modules/_decimal/libmpdec/bits.h +++ b/Modules/_decimal/libmpdec/bits.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/constants.c b/Modules/_decimal/libmpdec/constants.c index 92f5891b56f..2c2d5ea4810 100644 --- a/Modules/_decimal/libmpdec/constants.c +++ b/Modules/_decimal/libmpdec/constants.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/constants.h b/Modules/_decimal/libmpdec/constants.h index 13df9202123..c0febfc8772 100644 --- a/Modules/_decimal/libmpdec/constants.h +++ b/Modules/_decimal/libmpdec/constants.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/context.c b/Modules/_decimal/libmpdec/context.c index 159f88c339e..24c7b890c1d 100644 --- a/Modules/_decimal/libmpdec/context.c +++ b/Modules/_decimal/libmpdec/context.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/convolute.c b/Modules/_decimal/libmpdec/convolute.c index b5fe131b07c..4c62e8bd3ab 100644 --- a/Modules/_decimal/libmpdec/convolute.c +++ b/Modules/_decimal/libmpdec/convolute.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/convolute.h b/Modules/_decimal/libmpdec/convolute.h index c35ed461d3b..f30a177a684 100644 --- a/Modules/_decimal/libmpdec/convolute.h +++ b/Modules/_decimal/libmpdec/convolute.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/crt.c b/Modules/_decimal/libmpdec/crt.c index c71c4ee8f80..4a1e80a2322 100644 --- a/Modules/_decimal/libmpdec/crt.c +++ b/Modules/_decimal/libmpdec/crt.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/crt.h b/Modules/_decimal/libmpdec/crt.h index 8909232231a..f61e7729363 100644 --- a/Modules/_decimal/libmpdec/crt.h +++ b/Modules/_decimal/libmpdec/crt.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/difradix2.c b/Modules/_decimal/libmpdec/difradix2.c index 4ebb0b54b0a..06e5ab5e222 100644 --- a/Modules/_decimal/libmpdec/difradix2.c +++ b/Modules/_decimal/libmpdec/difradix2.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/difradix2.h b/Modules/_decimal/libmpdec/difradix2.h index 81aa598543f..5e22bcf324f 100644 --- a/Modules/_decimal/libmpdec/difradix2.h +++ b/Modules/_decimal/libmpdec/difradix2.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/fnt.c b/Modules/_decimal/libmpdec/fnt.c index 93116539b9f..7e924c85242 100644 --- a/Modules/_decimal/libmpdec/fnt.c +++ b/Modules/_decimal/libmpdec/fnt.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/fnt.h b/Modules/_decimal/libmpdec/fnt.h index 1f302cccbfa..fa2154a798d 100644 --- a/Modules/_decimal/libmpdec/fnt.h +++ b/Modules/_decimal/libmpdec/fnt.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/fourstep.c b/Modules/_decimal/libmpdec/fourstep.c index aa32c0d5cf2..21d3e7485df 100644 --- a/Modules/_decimal/libmpdec/fourstep.c +++ b/Modules/_decimal/libmpdec/fourstep.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/fourstep.h b/Modules/_decimal/libmpdec/fourstep.h index 593f27d489f..80dcd4be3d5 100644 --- a/Modules/_decimal/libmpdec/fourstep.h +++ b/Modules/_decimal/libmpdec/fourstep.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/io.c b/Modules/_decimal/libmpdec/io.c index 36470ca0062..b67324da941 100644 --- a/Modules/_decimal/libmpdec/io.c +++ b/Modules/_decimal/libmpdec/io.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/io.h b/Modules/_decimal/libmpdec/io.h index 3dfce732aa4..de5486a00ca 100644 --- a/Modules/_decimal/libmpdec/io.h +++ b/Modules/_decimal/libmpdec/io.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/literature/fnt.py b/Modules/_decimal/libmpdec/literature/fnt.py index bf937459f52..6363536da64 100644 --- a/Modules/_decimal/libmpdec/literature/fnt.py +++ b/Modules/_decimal/libmpdec/literature/fnt.py @@ -1,5 +1,5 @@ # -# Copyright (c) 2008-2012 Stefan Krah. All rights reserved. +# Copyright (c) 2008-2016 Stefan Krah. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/literature/mulmod-64.txt b/Modules/_decimal/libmpdec/literature/mulmod-64.txt index 93bf22e9fed..029b8de3d7c 100644 --- a/Modules/_decimal/libmpdec/literature/mulmod-64.txt +++ b/Modules/_decimal/libmpdec/literature/mulmod-64.txt @@ -59,7 +59,7 @@ The reduction step b) preserves congruence: Maximum numbers of step b): --------------------------- -# To avoid unneccessary formalism, define: +# To avoid unnecessary formalism, define: def R(hi, lo, z): return divmod(hi * z - hi + lo, 2**64) diff --git a/Modules/_decimal/libmpdec/literature/umodarith.lisp b/Modules/_decimal/libmpdec/literature/umodarith.lisp index 008e9f4507f..99d71c373d1 100644 --- a/Modules/_decimal/libmpdec/literature/umodarith.lisp +++ b/Modules/_decimal/libmpdec/literature/umodarith.lisp @@ -1,5 +1,5 @@ ; -; Copyright (c) 2008-2012 Stefan Krah. All rights reserved. +; Copyright (c) 2008-2016 Stefan Krah. All rights reserved. ; ; Redistribution and use in source and binary forms, with or without ; modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/memory.c b/Modules/_decimal/libmpdec/memory.c index bf6350f9049..0f41fe50645 100644 --- a/Modules/_decimal/libmpdec/memory.c +++ b/Modules/_decimal/libmpdec/memory.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/memory.h b/Modules/_decimal/libmpdec/memory.h index 7e73c130593..9c98d1a4000 100644 --- a/Modules/_decimal/libmpdec/memory.h +++ b/Modules/_decimal/libmpdec/memory.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/mpdecimal.c b/Modules/_decimal/libmpdec/mpdecimal.c index 5c8797ab520..05fd04dce6a 100644 --- a/Modules/_decimal/libmpdec/mpdecimal.c +++ b/Modules/_decimal/libmpdec/mpdecimal.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -3202,9 +3202,9 @@ mpd_qabs(mpd_t *result, const mpd_t *a, const mpd_context_t *ctx, } static inline void -_mpd_ptrswap(mpd_t **a, mpd_t **b) +_mpd_ptrswap(const mpd_t **a, const mpd_t **b) { - mpd_t *t = *a; + const mpd_t *t = *a; *a = *b; *b = t; } @@ -3232,7 +3232,7 @@ static void _mpd_qaddsub(mpd_t *result, const mpd_t *a, const mpd_t *b, uint8_t sign_b, const mpd_context_t *ctx, uint32_t *status) { - mpd_t *big, *small; + const mpd_t *big, *small; MPD_NEW_STATIC(big_aligned,0,0,0,0); MPD_NEW_CONST(tiny,0,0,1,1,1,1); mpd_uint_t carry; @@ -3242,7 +3242,7 @@ _mpd_qaddsub(mpd_t *result, const mpd_t *a, const mpd_t *b, uint8_t sign_b, /* compare exponents */ - big = (mpd_t *)a; small = (mpd_t *)b; + big = a; small = b; if (big->exp != small->exp) { if (small->exp > big->exp) { _mpd_ptrswap(&big, &small); @@ -4421,7 +4421,7 @@ mpd_qfma(mpd_t *result, const mpd_t *a, const mpd_t *b, const mpd_t *c, const mpd_context_t *ctx, uint32_t *status) { uint32_t workstatus = 0; - mpd_t *cc = (mpd_t *)c; + const mpd_t *cc = c; if (result == c) { if ((cc = mpd_qncopy(c)) == NULL) { @@ -4435,7 +4435,7 @@ mpd_qfma(mpd_t *result, const mpd_t *a, const mpd_t *b, const mpd_t *c, mpd_qadd(result, result, cc, ctx, &workstatus); } - if (cc != c) mpd_del(cc); + if (cc != c) mpd_del((mpd_t *)cc); *status |= workstatus; } @@ -5727,7 +5727,7 @@ static inline void _mpd_qmul(mpd_t *result, const mpd_t *a, const mpd_t *b, const mpd_context_t *ctx, uint32_t *status) { - mpd_t *big = (mpd_t *)a, *small = (mpd_t *)b; + const mpd_t *big = a, *small = b; mpd_uint_t *rdata = NULL; mpd_uint_t rbuf[MPD_MINALLOC_MAX]; mpd_size_t rsize, i; diff --git a/Modules/_decimal/libmpdec/mpdecimal.h b/Modules/_decimal/libmpdec/mpdecimal.h index 676c72dbdb1..58bee022538 100644 --- a/Modules/_decimal/libmpdec/mpdecimal.h +++ b/Modules/_decimal/libmpdec/mpdecimal.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -32,6 +32,10 @@ #ifdef __cplusplus extern "C" { + #ifndef __STDC_LIMIT_MACROS + #define __STDC_LIMIT_MACROS + #define MPD_CLEAR_STDC_LIMIT_MACROS + #endif #endif @@ -55,18 +59,12 @@ extern "C" { #define MPD_HIDE_SYMBOLS_END #define EXTINLINE extern inline #else + #ifdef HAVE_STDINT_H + #include + #endif #ifdef HAVE_INTTYPES_H #include #endif - #ifdef HAVE_STDINT_H - #if defined(__cplusplus) && !defined(__STDC_LIMIT_MACROS) - #define __STDC_LIMIT_MACROS - #include - #undef __STDC_LIMIT_MACROS - #else - #include - #endif - #endif #ifndef __GNUC_STDC_INLINE__ #define __GNUC_STDC_INLINE__ 1 #endif @@ -835,6 +833,10 @@ MPD_PRAGMA(MPD_HIDE_SYMBOLS_END) /* restore previous scope rules */ #ifdef __cplusplus + #ifdef MPD_CLEAR_STDC_LIMIT_MACROS + #undef MPD_CLEAR_STDC_LIMIT_MACROS + #undef __STDC_LIMIT_MACROS + #endif } /* END extern "C" */ #endif diff --git a/Modules/_decimal/libmpdec/numbertheory.c b/Modules/_decimal/libmpdec/numbertheory.c index 10ce6dc1464..4e035477e28 100644 --- a/Modules/_decimal/libmpdec/numbertheory.c +++ b/Modules/_decimal/libmpdec/numbertheory.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/numbertheory.h b/Modules/_decimal/libmpdec/numbertheory.h index 10922546198..e94c157910c 100644 --- a/Modules/_decimal/libmpdec/numbertheory.h +++ b/Modules/_decimal/libmpdec/numbertheory.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/sixstep.c b/Modules/_decimal/libmpdec/sixstep.c index 7d0542d641d..92d513ebe18 100644 --- a/Modules/_decimal/libmpdec/sixstep.c +++ b/Modules/_decimal/libmpdec/sixstep.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/sixstep.h b/Modules/_decimal/libmpdec/sixstep.h index 0467007061b..4a8b015e3a9 100644 --- a/Modules/_decimal/libmpdec/sixstep.h +++ b/Modules/_decimal/libmpdec/sixstep.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/transpose.c b/Modules/_decimal/libmpdec/transpose.c index 5e5d4b6625e..55d6d899227 100644 --- a/Modules/_decimal/libmpdec/transpose.c +++ b/Modules/_decimal/libmpdec/transpose.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -169,7 +169,7 @@ squaretrans(mpd_uint_t *buf, mpd_size_t cols) /* * Transpose 2^n * 2^n matrix. For cache efficiency, the matrix is split into * square blocks with side length 'SIDE'. First, the blocks are transposed, - * then a square tranposition is done on each individual block. + * then a square transposition is done on each individual block. */ static void squaretrans_pow2(mpd_uint_t *matrix, mpd_size_t size) diff --git a/Modules/_decimal/libmpdec/transpose.h b/Modules/_decimal/libmpdec/transpose.h index 7e349ee08df..e1cd1fa17dd 100644 --- a/Modules/_decimal/libmpdec/transpose.h +++ b/Modules/_decimal/libmpdec/transpose.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/typearith.h b/Modules/_decimal/libmpdec/typearith.h index 614812c69e2..405237dac51 100644 --- a/Modules/_decimal/libmpdec/typearith.h +++ b/Modules/_decimal/libmpdec/typearith.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/umodarith.h b/Modules/_decimal/libmpdec/umodarith.h index 436761bc448..68d15188cb3 100644 --- a/Modules/_decimal/libmpdec/umodarith.h +++ b/Modules/_decimal/libmpdec/umodarith.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/vccompat.h b/Modules/_decimal/libmpdec/vccompat.h index 276e0372cb6..f58e023c628 100644 --- a/Modules/_decimal/libmpdec/vccompat.h +++ b/Modules/_decimal/libmpdec/vccompat.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2012 Stefan Krah. All rights reserved. + * Copyright (c) 2008-2016 Stefan Krah. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions diff --git a/Modules/_decimal/libmpdec/vcdiv64.asm b/Modules/_decimal/libmpdec/vcdiv64.asm index 31bba08b0cb..6b6645673ab 100644 --- a/Modules/_decimal/libmpdec/vcdiv64.asm +++ b/Modules/_decimal/libmpdec/vcdiv64.asm @@ -1,5 +1,5 @@ ; -; Copyright (c) 2008-2012 Stefan Krah. All rights reserved. +; Copyright (c) 2008-2016 Stefan Krah. All rights reserved. ; ; Redistribution and use in source and binary forms, with or without ; modification, are permitted provided that the following conditions diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 7df3e19fc60..91f291a011f 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -2477,6 +2477,95 @@ test_pytime_object_to_timespec(PyObject *self, PyObject *args) return Py_BuildValue("Nl", _PyLong_FromTime_t(sec), nsec); } +#ifdef WITH_THREAD +typedef struct { + PyThread_type_lock start_event; + PyThread_type_lock exit_event; + PyObject *callback; +} test_c_thread_t; + +static void +temporary_c_thread(void *data) +{ + test_c_thread_t *test_c_thread = data; + PyGILState_STATE state; + PyObject *res; + + PyThread_release_lock(test_c_thread->start_event); + + /* Allocate a Python thread state for this thread */ + state = PyGILState_Ensure(); + + res = PyObject_CallFunction(test_c_thread->callback, "", NULL); + Py_CLEAR(test_c_thread->callback); + + if (res == NULL) { + PyErr_Print(); + } + else { + Py_DECREF(res); + } + + /* Destroy the Python thread state for this thread */ + PyGILState_Release(state); + + PyThread_release_lock(test_c_thread->exit_event); + + PyThread_exit_thread(); +} + +static PyObject * +call_in_temporary_c_thread(PyObject *self, PyObject *callback) +{ + PyObject *res = NULL; + test_c_thread_t test_c_thread; + long thread; + + PyEval_InitThreads(); + + test_c_thread.start_event = PyThread_allocate_lock(); + test_c_thread.exit_event = PyThread_allocate_lock(); + test_c_thread.callback = NULL; + if (!test_c_thread.start_event || !test_c_thread.exit_event) { + PyErr_SetString(PyExc_RuntimeError, "could not allocate lock"); + goto exit; + } + + Py_INCREF(callback); + test_c_thread.callback = callback; + + PyThread_acquire_lock(test_c_thread.start_event, 1); + PyThread_acquire_lock(test_c_thread.exit_event, 1); + + thread = PyThread_start_new_thread(temporary_c_thread, &test_c_thread); + if (thread == -1) { + PyErr_SetString(PyExc_RuntimeError, "unable to start the thread"); + PyThread_release_lock(test_c_thread.start_event); + PyThread_release_lock(test_c_thread.exit_event); + goto exit; + } + + PyThread_acquire_lock(test_c_thread.start_event, 1); + PyThread_release_lock(test_c_thread.start_event); + + Py_BEGIN_ALLOW_THREADS + PyThread_acquire_lock(test_c_thread.exit_event, 1); + PyThread_release_lock(test_c_thread.exit_event); + Py_END_ALLOW_THREADS + + Py_INCREF(Py_None); + res = Py_None; + +exit: + Py_CLEAR(test_c_thread.callback); + if (test_c_thread.start_event) + PyThread_free_lock(test_c_thread.start_event); + if (test_c_thread.exit_event) + PyThread_free_lock(test_c_thread.exit_event); + return res; +} +#endif /* WITH_THREAD */ + static PyMethodDef TestMethods[] = { {"raise_exception", raise_exception, METH_VARARGS}, @@ -2574,6 +2663,10 @@ static PyMethodDef TestMethods[] = { {"pytime_object_to_time_t", test_pytime_object_to_time_t, METH_VARARGS}, {"pytime_object_to_timeval", test_pytime_object_to_timeval, METH_VARARGS}, {"pytime_object_to_timespec", test_pytime_object_to_timespec, METH_VARARGS}, +#ifdef WITH_THREAD + {"call_in_temporary_c_thread", call_in_temporary_c_thread, METH_O, + PyDoc_STR("set_error_class(error_class) -> None")}, +#endif {NULL, NULL} /* sentinel */ }; diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c index c86a27f26e8..680f630ebdd 100644 --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -301,7 +301,7 @@ faulthandler_fatal_error(int signum) return; } #endif - /* call the previous signal handler: it is called immediatly if we use + /* call the previous signal handler: it is called immediately if we use sigaction() thanks to SA_NODEFER flag, otherwise it is deferred */ raise(signum); } diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index f3234a45389..f97d70beb93 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -1742,7 +1742,7 @@ win32_xstat_impl(const char *path, struct win32_stat *result, /* FILE_FLAG_BACKUP_SEMANTICS is required to open a directory */ /* FILE_FLAG_OPEN_REPARSE_POINT does not follow the symlink. Because of this, calls like GetFinalPathNameByHandle will return - the symlink path agin and not the actual final path. */ + the symlink path again and not the actual final path. */ FILE_ATTRIBUTE_NORMAL|FILE_FLAG_BACKUP_SEMANTICS| FILE_FLAG_OPEN_REPARSE_POINT, NULL); @@ -1838,7 +1838,7 @@ win32_xstat_impl_w(const wchar_t *path, struct win32_stat *result, /* FILE_FLAG_BACKUP_SEMANTICS is required to open a directory */ /* FILE_FLAG_OPEN_REPARSE_POINT does not follow the symlink. Because of this, calls like GetFinalPathNameByHandle will return - the symlink path agin and not the actual final path. */ + the symlink path again and not the actual final path. */ FILE_ATTRIBUTE_NORMAL|FILE_FLAG_BACKUP_SEMANTICS| FILE_FLAG_OPEN_REPARSE_POINT, NULL); diff --git a/Modules/selectmodule.c b/Modules/selectmodule.c index 603a2b670b0..c492224ecbc 100644 --- a/Modules/selectmodule.c +++ b/Modules/selectmodule.c @@ -361,7 +361,7 @@ update_ufd_array(pollObject *self) assert(i < self->ufd_len); /* Never overflow */ self->ufds[i].fd = (int)PyLong_AsLong(key); - self->ufds[i].events = (short)PyLong_AsLong(value); + self->ufds[i].events = (short)(unsigned short)PyLong_AsLong(value); i++; } assert(i == self->ufd_len); @@ -369,6 +369,24 @@ update_ufd_array(pollObject *self) return 1; } +static int +ushort_converter(PyObject *obj, void *ptr) +{ + unsigned long uval; + + uval = PyLong_AsUnsignedLong(obj); + if (uval == (unsigned long)-1 && PyErr_Occurred()) + return 0; + if (uval > USHRT_MAX) { + PyErr_SetString(PyExc_OverflowError, + "Python int too large for C unsigned short"); + return 0; + } + + *(unsigned short *)ptr = Py_SAFE_DOWNCAST(uval, unsigned long, unsigned short); + return 1; +} + PyDoc_STRVAR(poll_register_doc, "register(fd [, eventmask] ) -> None\n\n\ Register a file descriptor with the polling object.\n\ @@ -381,12 +399,11 @@ poll_register(pollObject *self, PyObject *args) { PyObject *o, *key, *value; int fd; - short events = POLLIN | POLLPRI | POLLOUT; + unsigned short events = POLLIN | POLLPRI | POLLOUT; int err; - if (!PyArg_ParseTuple(args, "O|h:register", &o, &events)) { + if (!PyArg_ParseTuple(args, "O|O&:register", &o, ushort_converter, &events)) return NULL; - } fd = PyObject_AsFileDescriptor(o); if (fd == -1) return NULL; @@ -424,12 +441,12 @@ static PyObject * poll_modify(pollObject *self, PyObject *args) { PyObject *o, *key, *value; - int fd, events; + int fd; + unsigned short events; int err; - if (!PyArg_ParseTuple(args, "Oi:modify", &o, &events)) { + if (!PyArg_ParseTuple(args, "OO&:modify", &o, ushort_converter, &events)) return NULL; - } fd = PyObject_AsFileDescriptor(o); if (fd == -1) return NULL; @@ -727,11 +744,11 @@ static PyObject * internal_devpoll_register(devpollObject *self, PyObject *args, int remove) { PyObject *o; - int fd, events = POLLIN | POLLPRI | POLLOUT; + int fd; + unsigned short events = POLLIN | POLLPRI | POLLOUT; - if (!PyArg_ParseTuple(args, "O|i:register", &o, &events)) { + if (!PyArg_ParseTuple(args, "O|O&:register", &o, ushort_converter, &events)) return NULL; - } fd = PyObject_AsFileDescriptor(o); if (fd == -1) return NULL; @@ -747,7 +764,7 @@ internal_devpoll_register(devpollObject *self, PyObject *args, int remove) } self->fds[self->n_fds].fd = fd; - self->fds[self->n_fds].events = events; + self->fds[self->n_fds].events = (signed short)events; if (++self->n_fds == self->max_n_fds) { if (devpoll_flush(self)) diff --git a/Objects/abstract.c b/Objects/abstract.c index 1309c5158e3..124e7664fce 100644 --- a/Objects/abstract.c +++ b/Objects/abstract.c @@ -1132,7 +1132,7 @@ PyNumber_Absolute(PyObject *o) return type_error("bad operand type for abs(): '%.200s'", o); } -/* Return a Python int from the object item +/* Return a Python int from the object item. Raise TypeError if the result is not an int or if the object cannot be interpreted as an index. */ @@ -1146,21 +1146,21 @@ PyNumber_Index(PyObject *item) Py_INCREF(item); return item; } - if (PyIndex_Check(item)) { - result = item->ob_type->tp_as_number->nb_index(item); - if (result && !PyLong_Check(result)) { - PyErr_Format(PyExc_TypeError, - "__index__ returned non-int " - "(type %.200s)", - result->ob_type->tp_name); - Py_DECREF(result); - return NULL; - } - } - else { + if (!PyIndex_Check(item)) { PyErr_Format(PyExc_TypeError, "'%.200s' object cannot be interpreted " "as an integer", item->ob_type->tp_name); + return NULL; + } + result = item->ob_type->tp_as_number->nb_index(item); + if (!result || PyLong_CheckExact(result)) + return result; + if (!PyLong_Check(result)) { + PyErr_Format(PyExc_TypeError, + "__index__ returned non-int (type %.200s)", + result->ob_type->tp_name); + Py_DECREF(result); + return NULL; } return result; } @@ -1212,34 +1212,6 @@ PyNumber_AsSsize_t(PyObject *item, PyObject *err) } -/* - Returns the Integral instance converted to an int. The instance is expected - to be an int or have an __int__ method. Steals integral's - reference. error_format will be used to create the TypeError if integral - isn't actually an Integral instance. error_format should be a format string - that can accept a char* naming integral's type. -*/ -static PyObject * -convert_integral_to_int(PyObject *integral, const char *error_format) -{ - PyNumberMethods *nb; - if (PyLong_Check(integral)) - return integral; - nb = Py_TYPE(integral)->tp_as_number; - if (nb->nb_int) { - PyObject *as_int = nb->nb_int(integral); - if (!as_int || PyLong_Check(as_int)) { - Py_DECREF(integral); - return as_int; - } - Py_DECREF(as_int); - } - PyErr_Format(PyExc_TypeError, error_format, Py_TYPE(integral)->tp_name); - Py_DECREF(integral); - return NULL; -} - - PyObject * PyNumber_Long(PyObject *o) { @@ -1257,29 +1229,28 @@ PyNumber_Long(PyObject *o) } m = o->ob_type->tp_as_number; if (m && m->nb_int) { /* This should include subclasses of int */ - PyObject *res = m->nb_int(o); - if (res && !PyLong_Check(res)) { - PyErr_Format(PyExc_TypeError, - "__int__ returned non-int (type %.200s)", - res->ob_type->tp_name); - Py_DECREF(res); - return NULL; - } - return res; + return (PyObject *)_PyLong_FromNbInt(o); } - if (PyLong_Check(o)) /* An int subclass without nb_int */ - return _PyLong_Copy((PyLongObject *)o); trunc_func = _PyObject_LookupSpecial(o, &PyId___trunc__); if (trunc_func) { PyObject *truncated = PyEval_CallObject(trunc_func, NULL); PyObject *int_instance; Py_DECREF(trunc_func); - if (truncated == NULL) - return NULL; + if (truncated == NULL || PyLong_Check(truncated)) + return truncated; /* __trunc__ is specified to return an Integral type, but int() needs to return a int. */ - int_instance = convert_integral_to_int(truncated, - "__trunc__ returned non-Integral (type %.200s)"); + m = truncated->ob_type->tp_as_number; + if (m == NULL || m->nb_int == NULL) { + PyErr_Format( + PyExc_TypeError, + "__trunc__ returned non-Integral (type %.200s)", + truncated->ob_type->tp_name); + Py_DECREF(truncated); + return NULL; + } + int_instance = (PyObject *)_PyLong_FromNbInt(truncated); + Py_DECREF(truncated); return int_instance; } if (PyErr_Occurred()) diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c index 47898fe975c..9dcb74e8fdb 100644 --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -186,8 +186,17 @@ PyBytes_FromFormatV(const char *format, va_list vargs) switch (*f) { case 'c': - (void)va_arg(count, int); - /* fall through... */ + { + int c = va_arg(count, int); + if (c < 0 || c > 255) { + PyErr_SetString(PyExc_OverflowError, + "PyBytes_FromFormatV(): %c format " + "expects an integer in range [0; 255]"); + return NULL; + } + n++; + break; + } case '%': n++; break; @@ -267,8 +276,12 @@ PyBytes_FromFormatV(const char *format, va_list vargs) switch (*f) { case 'c': - *s++ = va_arg(vargs, int); + { + int c = va_arg(vargs, int); + /* c has been checked for overflow in the first step */ + *s++ = (unsigned char)c; break; + } case 'd': if (longflag) sprintf(s, "%ld", va_arg(vargs, long)); diff --git a/Objects/genobject.c b/Objects/genobject.c index 016bfa29752..2e74b8c9393 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -76,6 +76,7 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc) /* Generators always return to their most recent caller, not * necessarily their creator. */ + f->f_tstate = tstate; Py_XINCREF(tstate->frame); assert(f->f_back == NULL); f->f_back = tstate->frame; @@ -89,6 +90,8 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc) * cycle. */ assert(f->f_back == tstate->frame); Py_CLEAR(f->f_back); + /* Clear the borrowed reference to the thread state */ + f->f_tstate = NULL; /* If the generator just returned (as opposed to yielding), signal * that the generator is exhausted. */ diff --git a/Objects/longobject.c b/Objects/longobject.c index 6383507a5c6..4ae22ef31b6 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -116,6 +116,47 @@ long_normalize(register PyLongObject *v) return v; } +/* _PyLong_FromNbInt: Convert the given object to a PyLongObject + using the nb_int slot, if available. Raise TypeError if either the + nb_int slot is not available or the result of the call to nb_int + returns something not of type int. +*/ +PyLongObject * +_PyLong_FromNbInt(PyObject *integral) +{ + PyNumberMethods *nb; + PyObject *result; + + /* Fast path for the case that we already have an int. */ + if (PyLong_CheckExact(integral)) { + Py_INCREF(integral); + return (PyLongObject *)integral; + } + + nb = Py_TYPE(integral)->tp_as_number; + if (nb == NULL || nb->nb_int == NULL) { + PyErr_Format(PyExc_TypeError, + "an integer is required (got type %.200s)", + Py_TYPE(integral)->tp_name); + return NULL; + } + + /* Convert using the nb_int slot, which should return something + of exact type int. */ + result = nb->nb_int(integral); + if (!result || PyLong_CheckExact(result)) + return (PyLongObject *)result; + if (!PyLong_Check(result)) { + PyErr_Format(PyExc_TypeError, + "__int__ returned non-int (type %.200s)", + result->ob_type->tp_name); + Py_DECREF(result); + return NULL; + } + return (PyLongObject *)result; +} + + /* Allocate a new int object with size digits. Return NULL and set exception if we run out of memory. */ @@ -347,28 +388,17 @@ PyLong_AsLongAndOverflow(PyObject *vv, int *overflow) return -1; } - if (!PyLong_Check(vv)) { - PyNumberMethods *nb; - nb = vv->ob_type->tp_as_number; - if (nb == NULL || nb->nb_int == NULL) { - PyErr_SetString(PyExc_TypeError, - "an integer is required"); - return -1; - } - vv = (*nb->nb_int) (vv); - if (vv == NULL) + if (PyLong_Check(vv)) { + v = (PyLongObject *)vv; + } + else { + v = _PyLong_FromNbInt(vv); + if (v == NULL) return -1; do_decref = 1; - if (!PyLong_Check(vv)) { - Py_DECREF(vv); - PyErr_SetString(PyExc_TypeError, - "nb_int should return int object"); - return -1; - } } res = -1; - v = (PyLongObject *)vv; i = Py_SIZE(v); switch (i) { @@ -412,7 +442,7 @@ PyLong_AsLongAndOverflow(PyObject *vv, int *overflow) } exit: if (do_decref) { - Py_DECREF(vv); + Py_DECREF(v); } return res; } @@ -630,36 +660,25 @@ _PyLong_AsUnsignedLongMask(PyObject *vv) unsigned long PyLong_AsUnsignedLongMask(register PyObject *op) { - PyNumberMethods *nb; PyLongObject *lo; unsigned long val; - if (op && PyLong_Check(op)) - return _PyLong_AsUnsignedLongMask(op); - - if (op == NULL || (nb = op->ob_type->tp_as_number) == NULL || - nb->nb_int == NULL) { - PyErr_SetString(PyExc_TypeError, "an integer is required"); + if (op == NULL) { + PyErr_BadInternalCall(); return (unsigned long)-1; } - lo = (PyLongObject*) (*nb->nb_int) (op); + if (PyLong_Check(op)) { + return _PyLong_AsUnsignedLongMask(op); + } + + lo = _PyLong_FromNbInt(op); if (lo == NULL) return (unsigned long)-1; - if (PyLong_Check(lo)) { - val = _PyLong_AsUnsignedLongMask((PyObject *)lo); - Py_DECREF(lo); - if (PyErr_Occurred()) - return (unsigned long)-1; - return val; - } - else - { - Py_DECREF(lo); - PyErr_SetString(PyExc_TypeError, - "nb_int should return int object"); - return (unsigned long)-1; - } + + val = _PyLong_AsUnsignedLongMask((PyObject *)lo); + Py_DECREF(lo); + return val; } int @@ -1169,40 +1188,41 @@ PyLong_AsLongLong(PyObject *vv) PY_LONG_LONG bytes; int one = 1; int res; + int do_decref = 0; /* if nb_int was called */ if (vv == NULL) { PyErr_BadInternalCall(); return -1; } - if (!PyLong_Check(vv)) { - PyNumberMethods *nb; - PyObject *io; - if ((nb = vv->ob_type->tp_as_number) == NULL || - nb->nb_int == NULL) { - PyErr_SetString(PyExc_TypeError, "an integer is required"); + + if (PyLong_Check(vv)) { + v = (PyLongObject *)vv; + } + else { + v = _PyLong_FromNbInt(vv); + if (v == NULL) return -1; - } - io = (*nb->nb_int) (vv); - if (io == NULL) - return -1; - if (PyLong_Check(io)) { - bytes = PyLong_AsLongLong(io); - Py_DECREF(io); - return bytes; - } - Py_DECREF(io); - PyErr_SetString(PyExc_TypeError, "integer conversion failed"); - return -1; + do_decref = 1; } - v = (PyLongObject*)vv; + res = 0; switch(Py_SIZE(v)) { - case -1: return -(sdigit)v->ob_digit[0]; - case 0: return 0; - case 1: return v->ob_digit[0]; + case -1: + bytes = -(sdigit)v->ob_digit[0]; + break; + case 0: + bytes = 0; + break; + case 1: + bytes = v->ob_digit[0]; + break; + default: + res = _PyLong_AsByteArray((PyLongObject *)v, (unsigned char *)&bytes, + SIZEOF_LONG_LONG, IS_LITTLE_ENDIAN, 1); + } + if (do_decref) { + Py_DECREF(v); } - res = _PyLong_AsByteArray((PyLongObject *)vv, (unsigned char *)&bytes, - SIZEOF_LONG_LONG, IS_LITTLE_ENDIAN, 1); /* Plan 9 can't handle PY_LONG_LONG in ? : expressions */ if (res < 0) @@ -1283,36 +1303,25 @@ _PyLong_AsUnsignedLongLongMask(PyObject *vv) unsigned PY_LONG_LONG PyLong_AsUnsignedLongLongMask(register PyObject *op) { - PyNumberMethods *nb; PyLongObject *lo; unsigned PY_LONG_LONG val; - if (op && PyLong_Check(op)) - return _PyLong_AsUnsignedLongLongMask(op); - - if (op == NULL || (nb = op->ob_type->tp_as_number) == NULL || - nb->nb_int == NULL) { - PyErr_SetString(PyExc_TypeError, "an integer is required"); - return (unsigned PY_LONG_LONG)-1; + if (op == NULL) { + PyErr_BadInternalCall(); + return (unsigned long)-1; } - lo = (PyLongObject*) (*nb->nb_int) (op); + if (PyLong_Check(op)) { + return _PyLong_AsUnsignedLongLongMask(op); + } + + lo = _PyLong_FromNbInt(op); if (lo == NULL) return (unsigned PY_LONG_LONG)-1; - if (PyLong_Check(lo)) { - val = _PyLong_AsUnsignedLongLongMask((PyObject *)lo); - Py_DECREF(lo); - if (PyErr_Occurred()) - return (unsigned PY_LONG_LONG)-1; - return val; - } - else - { - Py_DECREF(lo); - PyErr_SetString(PyExc_TypeError, - "nb_int should return int object"); - return (unsigned PY_LONG_LONG)-1; - } + + val = _PyLong_AsUnsignedLongLongMask((PyObject *)lo); + Py_DECREF(lo); + return val; } #undef IS_LITTLE_ENDIAN @@ -1343,28 +1352,17 @@ PyLong_AsLongLongAndOverflow(PyObject *vv, int *overflow) return -1; } - if (!PyLong_Check(vv)) { - PyNumberMethods *nb; - nb = vv->ob_type->tp_as_number; - if (nb == NULL || nb->nb_int == NULL) { - PyErr_SetString(PyExc_TypeError, - "an integer is required"); - return -1; - } - vv = (*nb->nb_int) (vv); - if (vv == NULL) + if (PyLong_Check(vv)) { + v = (PyLongObject *)vv; + } + else { + v = _PyLong_FromNbInt(vv); + if (v == NULL) return -1; do_decref = 1; - if (!PyLong_Check(vv)) { - Py_DECREF(vv); - PyErr_SetString(PyExc_TypeError, - "nb_int should return int object"); - return -1; - } } res = -1; - v = (PyLongObject *)vv; i = Py_SIZE(v); switch (i) { @@ -1408,7 +1406,7 @@ PyLong_AsLongLongAndOverflow(PyObject *vv, int *overflow) } exit: if (do_decref) { - Py_DECREF(vv); + Py_DECREF(v); } return res; } diff --git a/PCbuild/pythoncore.vcxproj.filters b/PCbuild/pythoncore.vcxproj.filters index 09b4bb4d027..5647febdb76 100644 --- a/PCbuild/pythoncore.vcxproj.filters +++ b/PCbuild/pythoncore.vcxproj.filters @@ -402,13 +402,15 @@ Python - Python Python + + Include + @@ -915,7 +917,9 @@ PC - + + Objects +