mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-07-15 15:25:29 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
127783 commits 7 branches 620 tags 2.6 GiB
Commit graph

1739 commits

Author SHA1 Message Date
Victor Stinner
42bf77537e Rewrite PyUnicode_EncodeDecimal() to use the new Unicode API 
Add tests for PyUnicode_EncodeDecimal() and
PyUnicode_TransformDecimalToASCII().
 2011-11-21 22:52:58 +01:00
Antoine Pitrou
0a3229de6b Issue #13417: speed up utf-8 decoding by around 2x for the non-fully-ASCII case. 
This almost catches up with pre-PEP 393 performance, when decoding needed
only one pass.
 2011-11-21 20:39:13 +01:00
Victor Stinner
da29cc36aa Issue #13441: _PyUnicode_CheckConsistency() dumps the string if the maximum 
character is bigger than U+10FFFF and locale.localeconv() dumps the string
before decoding it.

Temporary hack to debug the issue #13441.
 2011-11-21 14:31:41 +01:00
Victor Stinner
9e30aa52fd Fix misuse of PyUnicode_GET_SIZE() => PyUnicode_GET_LENGTH() 
And PyUnicode_GetSize() => PyUnicode_GetLength()
 2011-11-21 02:49:52 +01:00
Victor Stinner
4ead7c7be8 PyObject_Str() ensures that the result string is ready 
and check the string consistency.

_PyUnicode_CheckConsistency() doesn't check the hash anymore. It should be
possible to call this function even if hash(str) was already called.
 2011-11-20 19:48:36 +01:00
Victor Stinner
b960b34577 PyUnicode_AsUTF32String() calls directly _PyUnicode_EncodeUTF32(), 
instead of calling the deprecated PyUnicode_EncodeUTF32() function
 2011-11-20 19:12:52 +01:00
Victor Stinner
77faf69ca1 _PyUnicode_CheckConsistency() also checks maxchar maximum value, 
not only its minimum value
 2011-11-20 18:56:05 +01:00
Victor Stinner
d5c4022d2a Remove the two ugly and unused WRITE_ASCII_OR_WSTR and WRITE_WSTR macros 2011-11-20 18:41:31 +01:00
Victor Stinner
2e9cfadd7c Reuse surrogate macros in UTF-16 decoder 2011-11-20 18:40:27 +01:00
Victor Stinner
ae4f7c8e59 charmap_encoding_error() uses the new Unicode API 2011-11-20 18:28:55 +01:00
Victor Stinner
ac931b1e5b Use PyUnicode_EncodeCodePage() instead of PyUnicode_EncodeMBCS() with 
PyUnicode_AsUnicodeAndSize()
 2011-11-20 18:27:03 +01:00
Victor Stinner
22168998f5 charmap encoders uses Py_UCS4, not Py_UNICODE 2011-11-20 17:09:18 +01:00
Victor Stinner
1f7951711c Catch PyUnicode_AS_UNICODE() errors 2011-11-17 00:45:54 +01:00
Ezio Melotti
11060a4a48 #13406: silence deprecation warnings in test_codecs. 2011-11-16 09:39:10 +02:00
Antoine Pitrou
78edf7576e Issue #13333: The UTF-7 decoder now accepts lone surrogates 
(the encoder already accepts them).
 2011-11-15 01:44:16 +01:00
Antoine Pitrou
5418ee0b9a Issue #13333: The UTF-7 decoder now accepts lone surrogates 
(the encoder already accepts them).
 2011-11-15 01:42:21 +01:00
Antoine Pitrou
31b92a534f Sanitize reference management in the utf-8 encoder 2011-11-12 18:35:19 +01:00
Antoine Pitrou
0290c7a811 Fix regression on 2-byte wchar_t systems (Windows) 2011-11-11 13:29:12 +01:00
Antoine Pitrou
44c6affc79 Avoid crashing because of an unaligned word access 2011-11-11 02:59:42 +01:00
Antoine Pitrou
de20b0b50e Issue #13149: Speed up append-only StringIO objects. 
This is very similar to the "lazy strings" idea.
 2011-11-10 21:47:38 +01:00
Victor Stinner
9f4b1e9c50 Fix and deprecated the unicode_internal codec 
unicode_internal codec uses Py_UNICODE instead of the real internal
representation (PEP 393: Py_UCS1, Py_UCS2 or Py_UCS4) for backward
compatibility.
 2011-11-10 20:56:30 +01:00
Victor Stinner
24729f36bf Prefer Py_UCS4 or wchar_t over Py_UNICODE 2011-11-10 20:31:37 +01:00
Victor Stinner
ebf3ba808e PyUnicode_DecodeCharmap() uses the new Unicode API 2011-11-10 20:30:22 +01:00
Victor Stinner
a98b28c1bf Avoid PyUnicode_AS_UNICODE in the UTF-8 encoder 2011-11-10 20:21:49 +01:00
Victor Stinner
3326cb6a36 Fix "unicode_escape" encoder 2011-11-10 20:15:25 +01:00
Victor Stinner
0e36826a04 Fix UTF-7 encoder on Windows 2011-11-10 20:12:49 +01:00
Martin v. Löwis
1db7c13be1 Port encoders from Py_UNICODE API to unicode object API. 2011-11-10 18:24:32 +01:00
Victor Stinner
62aa4d086a Strip trailing spaces 2011-11-09 00:03:45 +01:00
Victor Stinner
0a045efb49 Fix a compiler warning: use unsiged for maxchar in unicode_widen() 2011-11-09 00:02:42 +01:00
Victor Stinner
596a6c4ffc Fix the code page decoder 
* unicode_decode_call_errorhandler() now supports the PyUnicode_WCHAR_KIND
   kind
 * unicode_decode_call_errorhandler() calls copy_characters() instead of
   PyUnicode_CopyCharacters()
 2011-11-09 00:02:18 +01:00
Antoine Pitrou
a8f63c02ef Fix missing goto 2011-11-08 18:37:16 +01:00
Martin v. Löwis
d10759f6ed Make _PyUnicode_FromId return borrowed references. 
http://mail.python.org/pipermail/python-dev/2011-November/114347.html
 2011-11-07 13:00:05 +01:00
Martin v. Löwis
e9b11c1cd8 Change decoders to use Unicode API instead of Py_UNICODE. 2011-11-08 17:35:34 +01:00
Victor Stinner
e30c0a1014 Fix gdb/libpython.py for not ready Unicode strings 
_PyUnicode_CheckConsistency() checks also hash and length value for not ready
Unicode strings.
 2011-11-04 20:54:05 +01:00
Victor Stinner
2fc507fe45 Replace tabs by spaces 2011-11-04 20:06:39 +01:00
Martin v. Löwis
12be46ca84 Drop Py_UNICODE based encode exceptions. 2011-11-04 19:04:15 +01:00
Martin v. Löwis
3d325191bf Port code page codec to Unicode API. 2011-11-04 18:23:06 +01:00
Victor Stinner
fcd9653667 Fix a compiler warning in unicode_encode_ucs1() 2011-11-04 00:28:50 +01:00
Victor Stinner
fc026c98d8 Fix PyUnicode_EncodeCharmap() 2011-11-04 00:24:51 +01:00
Victor Stinner
7931d9a951 Replace PyUnicodeObject type by PyObject 
* _PyUnicode_CheckConsistency() now takes a PyObject* instead of void*
 * Remove now useless casts to PyObject*
 2011-11-04 00:22:48 +01:00
Victor Stinner
76a31a6bff Cleanup decode_code_page_stateful() and encode_code_page() 
* Fix decode_code_page_errors() result
 * Inline decode_code_page() and encode_code_page_chunk()
 * Replace the PyUnicodeObject type by PyObject
 2011-11-04 00:05:13 +01:00
Victor Stinner
7581cef699 Adapt the code page encoder to the new unicode_encode_call_errorhandler() 
The code is not correct, but at least it doesn't crash anymore.
 2011-11-03 22:32:33 +01:00
Brian Curtin
2787ea41fd Fix a compile error (apparently Windows only) introduced in 295fdfd4f422 2011-11-02 15:09:37 -05:00
Martin v. Löwis
23e275b3ad Port UCS1 and charmap codecs to new API. 2011-11-02 18:02:51 +01:00
Martin v. Löwis
9e8166843c Introduce PyObject* API for raising encode errors. 2011-11-02 12:45:42 +01:00
Martin v. Löwis
0d3072e98d Drop Py_UCS4_ functions. Closes #13246. 2011-10-31 08:40:56 +01:00
Victor Stinner
57ffa9d4ff PyUnicode_AsUnicodeCopy() uses PyUnicode_AsUnicodeAndSize() to get directly the length 2011-10-23 20:10:08 +02:00
Victor Stinner
af9e4b8c29 Fix PyUnicode_InternImmortal(): PyUnicode_InternInPlace() may changes *p 2011-10-23 20:07:00 +02:00
Victor Stinner
9faa384bed Cast directly to unsigned char, instead of using Py_CHARMASK 
We don't need "& 0xff" on an unsigned char.
 2011-10-23 20:06:00 +02:00
Victor Stinner
9db1a8b69f Replace PyUnicodeObject* by PyObject* where it was irrevelant 
A Unicode string can now be a PyASCIIObject, PyCompactUnicodeObject or
PyUnicodeObject. Aliasing a PyASCIIObject* or PyCompactUnicodeObject* to
PyUnicodeObject* is wrong
 2011-10-23 20:04:37 +02:00