mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-08-04 17:08:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 13

bpo-24214: Fixed the UTF-8 and UTF-16 incremental decoders. (GH-14304)

Browse source 
* The UTF-8 incremental decoders fails now fast if encounter
  a sequence that can't be handled by the error handler.
* The UTF-16 incremental decoders with the surrogatepass error
  handler decodes now a lone low surrogate with final=False.
This commit is contained in:
Serhiy Storchaka 2019-06-25 11:54:18 +03:00 • committed by GitHub
parent 9fe42b49c7
commit 894263ba80
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 36 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

6
Objects/stringlib/codecs.h
View file

@ -207,7 +207,7 @@ STRINGLIB(utf8_decode)(const char **inptr, const char *end,
goto InvalidContinuation1;
} else if (ch == 0xF4 && ch2 >= 0x90) {
/* invalid sequence
\xF4\x90\x80\80- -- 110000- overflow */
\xF4\x90\x80\x80- -- 110000- overflow */
goto InvalidContinuation1;
}
if (!IS_CONTINUATION_BYTE(ch3)) {
@ -573,10 +573,10 @@ STRINGLIB(utf16_decode)(const unsigned char **inptr, const unsigned char *e,
}
/* UTF-16 code pair: */
if (q >= e)
goto UnexpectedEnd;
if (!Py_UNICODE_IS_HIGH_SURROGATE(ch))
goto IllegalEncoding;
if (q >= e)
goto UnexpectedEnd;
ch2 = (q[ihi] << 8) | q[ilo];
q += 2;
if (!Py_UNICODE_IS_LOW_SURROGATE(ch2))