mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-08-25 03:04:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 6

bpo-24214: Fixed the UTF-8 and UTF-16 incremental decoders. (GH-14304)

Browse source 
* The UTF-8 incremental decoders fails now fast if encounter
  a sequence that can't be handled by the error handler.
* The UTF-16 incremental decoders with the surrogatepass error
  handler decodes now a lone low surrogate with final=False.
This commit is contained in:
Serhiy Storchaka 2019-06-25 11:54:18 +03:00 • committed by GitHub
parent 9fe42b49c7
commit 894263ba80
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 36 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

10
Objects/unicodeobject.c
View file

@ -4945,11 +4945,15 @@ unicode_decode_utf8(const char *s, Py_ssize_t size,
endinpos = startinpos + 1;
break;
case 2:
case 3:
case 4:
if (s == end || consumed) {
if (consumed && (unsigned char)s[0] == 0xED && end - s == 2
&& (unsigned char)s[1] >= 0xA0 && (unsigned char)s[1] <= 0xBF)
{
/* Truncated surrogate code in range D800-DFFF */
goto End;
}
/* fall through */
case 3:
case 4:
errmsg = "invalid continuation byte";
startinpos = s - starts;
endinpos = startinpos + ch - 1;