bpo-36103: change default buffer size of shutil.copyfileobj() (GH-12115)

It is changed from 16KiB to 64KiB.  The previous default value
is used since 1990.

coreutils chose 128 KiB as minimum buffer size for block device I/O.

But shutil.copyfileobj() can be used for non block devices.
So I choose more conservative value.

As my quick benchmark, performance difference between 64KiB and
128 KiB is up to ~5%.  On the other hand, performance difference
between 32 KiB and 64 KiB can be more than 10% when file is fully
buffered.

This is why 64 KiB is rational value.
This commit is contained in:
Inada Naoki 2019-03-02 13:31:01 +09:00 committed by GitHub
parent bcfa450f21
commit 4f19030618
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 5 additions and 2 deletions

View file

@ -49,7 +49,7 @@ if os.name == 'posix':
elif _WINDOWS:
import nt
COPY_BUFSIZE = 1024 * 1024 if _WINDOWS else 16 * 1024
COPY_BUFSIZE = 1024 * 1024 if _WINDOWS else 64 * 1024
_HAS_SENDFILE = posix and hasattr(os, "sendfile")
_HAS_FCOPYFILE = posix and hasattr(posix, "_fcopyfile") # macOS