cpython/Lib/test
Giampaolo Rodola 4a172ccc73
bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win) (#7160)
* have shutil.copyfileobj use sendfile() if possible

* refactoring: use ctx manager

* add test with non-regular file obj

* emulate case where file size can't be determined

* reference _copyfileobj_sendfile directly

* add test for offset() at certain position

* add test for empty file

* add test for non regular file dst

* small refactoring

* leave copyfileobj() alone in order to not introduce any incompatibility

* minor refactoring

* remove old test

* update docstring

* update docstring; rename exception class

* detect platforms which only support file to socket zero copy

* don't run test on platforms where file-to-file zero copy is not supported

* use tempfiles

* reset verbosity

* add test for smaller chunks

* add big file size test

* add comment

* update doc

* update whatsnew doc

* update doc

* catch Exception

* remove unused import

* add test case for error on second sendfile() call

* turn docstring into comment

* add one more test

* update comment

* add Misc/NEWS entry

* get rid of COPY_BUFSIZE; it belongs to another PR

* update doc

* expose posix._fcopyfile() for OSX

* merge from linux branch

* merge from linux branch

* expose fcopyfile

* arg clinic for the win implementation

* convert path type to path_t

* expose CopyFileW

* fix windows tests

* release GIL

* minor refactoring

* update doc

* update comment

* update docstrings

* rename functions

* rename test classes

* update doc

* update doc

* update docstrings and comments

* avoid do import nt|posix modules if unnecessary

* set nt|posix modules to None if not available

* micro speedup

* update description

* add doc note

* use better wording in doc

* rename function using 'fastcopy' prefix instead of 'zerocopy'

* use :ref: in rst doc

* change wording in doc

* add test to make sure sendfile() doesn't get called aymore in case it doesn't support file to file copies

* move CopyFileW in _winapi and actually expose CopyFileExW instead

* fix line endings

* add tests for mode bits

* add docstring

* remove test file mode class; let's keep it for later when Istart addressing OSX fcopyfile() specific copies

* update doc to reflect new changes

* update doc

* adjust tests on win

* fix argument clinic error

* update doc

* OSX: expose copyfile(3) instead of fcopyfile(3); also expose flags arg to python

* osx / copyfile: use path_t instead of char

* do not set dst name in the OSError exception in order to remain consistent with platforms which cannot do that (e.g. linux)

* add same file test

* add test for same file

* have osx copyfile() pre-emptively check if src and dst are the same, otherwise it will return immedialtey and src file content gets deleted

* turn PermissionError into appropriate SameFileError

* expose ERROR_SHARING_VIOLATION in order to raise more appropriate SameFileError

* honour follow_symlinks arg when using CopyFileEx

* update Misc/NEWS

* expose CreateDirectoryEx mock

* change C type

* CreateDirectoryExW actual implementation

* provide specific makedirs() implementation for win

* fix typo

* skeleton for SetNamedSecurityInfo

* get security info for src path

* finally set security attrs

* add unit tests

* mimick os.makedirs() behavior and raise if dst dir exists

* set 2 paths for OSError object

* set 2 paths for OSError object

* expand windows test

* in case of exception on os.sendfile() set filename and filename2 exception attributes

* set 2 filenames (src, dst) for OSError in case copyfile() fails on OSX

* update doc

* do not use CreateDirectoryEx() in copytree() if source dir is a symlink (breaks test_copytree_symlink_dir); instead just create a plain dir and remain consistent with POSIX implementation

* use bytearray() and readinto()

* use memoryview() with bytearray()

* refactoring + introduce a new _fastcopy_binfileobj() fun

* remove CopyFileEx and other C wrappers

* remove code related to CopyFileEx

* Recognize binary files in copyfileobj()
...and use fastest _fastcopy_binfileobj() when possible

* set 1MB copy bufsize on win; also add a global _COPY_BUFSIZE variable

* use ctx manager for memoryview()

* update doc

* remove outdated doc

* remove last CopyFileEx remnants

* OSX - use fcopyfile(3) instead of copyfile(3)

...as an extra safety measure: in case src/dst are "exotic" files (non
regular or living on a network fs etc.) we better fail on open() instead
of copyfile(3) as we're not quite sure what's gonna happen in that
case.

* update doc
2018-06-12 23:04:50 +02:00
..
audiodata
capath
cjkencodings
crashers
data
decimaltestdata
dtracedata
eintrdata
encoded_modules
imghdrdata
leakers
libregrtest bpo-33718: regrtest: use "xxx then yyy" result if re-run (GH-7521) 2018-06-08 09:53:51 +02:00
sndhdrdata
subprocessdata
support bpo-33773: Fix test.support.fd_count() on Linux/FreBSD (GH-7421) 2018-06-06 17:23:50 +02:00
test_asyncio bpo-33694: Fix typo in helper function name (GH-7522) 2018-06-08 10:32:06 +02:00
test_email bpo-27397: Make email module properly handle invalid-length base64 strings (#7583) 2018-06-12 15:46:22 +03:00
test_import
test_importlib bpo-33755: Fix importlib.resources isolation tests (#7412) 2018-06-05 09:40:45 -07:00
test_json
test_tools
test_warnings bpo-33375: Get filename for warnings from frame.f_code.co_filename (GH-6622) 2018-06-08 12:28:37 -07:00
tracedmodules
xmltestdata
__init__.py
__main__.py
_test_multiprocessing.py bpo-33532: Fix test_multiprocessing_forkserver.test_ignore() (GH-7319) 2018-06-01 16:48:34 +02:00
allsans.pem
ann_module.py
ann_module2.py
ann_module3.py
audiotest.au
audiotests.py
autotest.py
bad_coding.py
bad_coding2.py
bad_getattr.py
bad_getattr2.py
bad_getattr3.py
badcert.pem
badkey.pem
badsyntax_3131.py
badsyntax_future3.py
badsyntax_future4.py
badsyntax_future5.py
badsyntax_future6.py
badsyntax_future7.py
badsyntax_future8.py
badsyntax_future9.py
badsyntax_future10.py
badsyntax_pep3120.py
bisect.py
bytecode_helper.py
cfgparser.1
cfgparser.2
cfgparser.3
cmath_testcases.txt
coding20731.py
curses_tests.py
dataclass_module_1.py
dataclass_module_1_str.py
dataclass_module_2.py
dataclass_module_2_str.py
datetimetester.py bpo-33812: Corrected astimezone for naive datetimes. (GH-7578) 2018-06-10 17:02:58 -04:00
dh1024.pem
dis_module.py
doctest_aliases.py
double_const.py
empty.vbs
exception_hierarchy.txt
final_a.py
final_b.py
floating_points.txt
fork_wait.py
formatfloat_testcases.txt
future_test1.py
future_test2.py
gdb_sample.py
good_getattr.py
idnsans.pem
ieee754.txt
imp_dummy.py
inspect_fodder.py
inspect_fodder2.py
keycert.passwd.pem
keycert.pem
keycert2.pem
keycert3.pem
keycert4.pem
keycertecc.pem
list_tests.py
lock_tests.py
mailcap.txt
make_ssl_certs.py
mapping_tests.py
math_testcases.txt
memory_watchdog.py
mime.types
mock_socket.py
mod_generics_cache.py
mp_fork_bomb.py
mp_preload.py
multibytecodec_support.py
nokia.pem
nullbytecert.pem
nullcert.pem
outstanding_bugs.py
pickletester.py
profilee.py
pstats.pck
pycacert.pem
pycakey.pem
pyclbr_input.py
pydoc_mod.py
pydocfodder.py
pythoninfo.py bpo-33692: pythoninfo detect libedit on Python 3.6 (GH-7293) 2018-06-01 11:04:29 +02:00
randv2_32.pck
randv2_64.pck
randv3.pck
re_tests.py
regrtest.py
relimport.py
reperf.py
revocation.crl
sample_doctest.py
sample_doctest_no_docstrings.py
sample_doctest_no_doctests.py
secp384r1.pem
selfsigned_pythontestdotnet.pem
seq_tests.py
sgml_input.html
signalinterproctester.py
Sine-1000Hz-300ms.aif
sortperf.py
ssl_cert.pem
ssl_key.passwd.pem
ssl_key.pem
ssl_servers.py
ssltests.py
string_tests.py
test___all__.py
test___future__.py
test__locale.py
test__opcode.py
test__osx_support.py
test__xxsubinterpreters.py bpo-33615: Skip test__xxsubinterpreters (GH-7503) 2018-06-08 02:28:28 +02:00
test_abc.py
test_abstract_numbers.py
test_aifc.py
test_argparse.py bpo-11874: fix assertion failure in argparse metavar handling (GH-1826) 2018-06-08 20:12:49 +10:00
test_array.py
test_asdl_parser.py
test_ast.py bpo-32911: Revert bpo-29463. (GH-7121) (GH-7197) 2018-05-29 12:04:55 +03:00
test_asyncgen.py bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() (GH-7467) 2018-06-07 20:31:26 -04:00
test_asynchat.py
test_asyncore.py
test_atexit.py
test_audioop.py
test_augassign.py
test_base64.py
test_baseexception.py
test_bdb.py
test_bigaddrspace.py
test_bigmem.py
test_binascii.py bpo-33770: improve base64 exception message for encoded inputs of invalid length (#7416) 2018-06-10 10:01:50 +03:00
test_binhex.py
test_binop.py
test_bisect.py
test_bool.py
test_buffer.py
test_bufio.py
test_builtin.py
test_bytes.py
test_bz2.py
test_c_locale_coercion.py
test_calendar.py
test_call.py
test_capi.py
test_cgi.py
test_cgitb.py
test_charmapcodec.py
test_class.py
test_cmath.py
test_cmd.py
test_cmd_line.py
test_cmd_line_script.py
test_code.py
test_code_module.py
test_codeccallbacks.py
test_codecencodings_cn.py
test_codecencodings_hk.py
test_codecencodings_iso2022.py
test_codecencodings_jp.py
test_codecencodings_kr.py
test_codecencodings_tw.py
test_codecmaps_cn.py
test_codecmaps_hk.py
test_codecmaps_jp.py
test_codecmaps_kr.py
test_codecmaps_tw.py
test_codecs.py
test_codeop.py
test_collections.py
test_colorsys.py
test_compare.py
test_compile.py
test_compileall.py
test_complex.py
test_concurrent_futures.py bpo-33238: Add InvalidStateError to concurrent.futures. (GH-7056) 2018-05-30 10:15:06 +03:00
test_configparser.py bpo-32108: Don't clear configparser values if key is assigned to itself (GH-7588) 2018-06-12 13:37:51 -07:00
test_contains.py
test_context.py
test_contextlib.py
test_contextlib_async.py bpo-33786: Fix asynchronous generators to handle GeneratorExit in athrow() (GH-7467) 2018-06-07 20:31:26 -04:00
test_copy.py
test_copyreg.py
test_coroutines.py bpo-33562: Check the global asyncio event loop policy isn't set after any tests (GH-7328) 2018-06-01 20:34:09 -07:00
test_cprofile.py bpo-29235: Make cProfile.Profile a context manager (GH-6808) 2018-06-01 13:36:23 -07:00
test_crashers.py
test_crypt.py
test_csv.py
test_ctypes.py
test_curses.py
test_dataclasses.py bpo-33796: Ignore ClassVar for dataclasses.replace(). (GH-7488) 2018-06-07 14:43:59 -04:00
test_datetime.py
test_dbm.py bpo-33752: Fix a file leak in test_dbm. (GH-7376) 2018-06-05 16:03:00 +03:00
test_dbm_dumb.py
test_dbm_gnu.py
test_dbm_ndbm.py
test_decimal.py bpo-33750: Reset thread-local context precision in test_round(). (#7355) 2018-06-03 18:40:00 +02:00
test_decorators.py
test_defaultdict.py
test_deque.py
test_descr.py
test_descrtut.py
test_devpoll.py
test_dict.py
test_dict_version.py
test_dictcomps.py
test_dictviews.py
test_difflib.py
test_difflib_expect.html
test_dis.py
test_distutils.py
test_doctest.py
test_doctest.txt
test_doctest2.py
test_doctest2.txt
test_doctest3.txt
test_doctest4.txt
test_docxmlrpc.py
test_dtrace.py
test_dummy_thread.py
test_dummy_threading.py
test_dynamic.py
test_dynamicclassattribute.py
test_eintr.py
test_embed.py
test_ensurepip.py
test_enum.py
test_enumerate.py
test_eof.py
test_epoll.py
test_errno.py
test_exception_hierarchy.py
test_exception_variations.py
test_exceptions.py
test_extcall.py
test_faulthandler.py
test_fcntl.py
test_file.py bpo-33751: Fix test_file. (GH-7378) 2018-06-05 19:55:41 +03:00
test_file_eintr.py
test_filecmp.py
test_fileinput.py bpo-33753: Refactor creating temporary files in test_fileinput. (GH-7377) 2018-06-05 12:08:36 +03:00
test_fileio.py
test_finalization.py
test_float.py
test_flufl.py
test_fnmatch.py
test_fork1.py
test_format.py
test_fractions.py
test_frame.py
test_frozen.py
test_fstring.py
test_ftplib.py bpo-33618: Enable TLS 1.3 in tests (GH-7079) 2018-05-23 22:24:45 +02:00
test_funcattrs.py
test_functools.py bpo-32380: Create functools.singledispatchmethod (#6306) 2018-05-26 13:38:33 -07:00
test_future.py
test_future3.py
test_future4.py
test_future5.py
test_gc.py
test_gdb.py
test_generator_stop.py
test_generators.py
test_genericclass.py
test_genericpath.py
test_genexps.py
test_getargs2.py
test_getopt.py
test_getpass.py
test_gettext.py
test_glob.py
test_global.py
test_grammar.py
test_grp.py
test_gzip.py
test_hash.py
test_hashlib.py
test_heapq.py
test_hmac.py bpo-33604: Remove Pending from hmac Deprecation warning. (GH-7062) 2018-05-22 15:55:31 -07:00
test_html.py
test_htmlparser.py
test_http_cookiejar.py
test_http_cookies.py
test_httplib.py
test_httpservers.py
test_idle.py
test_imaplib.py
test_imghdr.py
test_imp.py
test_index.py
test_inspect.py bpo-33582: Emit deprecation warning for formatargspec (GH-6994) 2018-06-11 16:08:16 -04:00
test_int.py
test_int_literal.py
test_io.py bpo-33760: Fix file leaks in test_io. (GH-7361) 2018-06-03 22:53:56 -04:00
test_ioctl.py
test_ipaddress.py
test_isinstance.py
test_iter.py
test_iterlen.py
test_itertools.py
test_keyword.py
test_keywordonlyarg.py
test_kqueue.py
test_largefile.py
test_lib2to3.py
test_linecache.py
test_list.py
test_listcomps.py
test_locale.py
test_logging.py bpo-33802: Do not interpolate in ConfigParser while reading defaults (GH-7524) 2018-06-08 04:02:48 -07:00
test_long.py
test_longexp.py
test_lzma.py
test_macpath.py
test_mailbox.py
test_mailcap.py
test_marshal.py bpo-33720: Reduces maximum marshal recursion depth on release builds. (GH-7401) 2018-06-04 13:25:00 -07:00
test_math.py
test_memoryio.py
test_memoryview.py
test_metaclass.py
test_mimetypes.py
test_minidom.py bpo-33274: Compliance with DOM L1: return removed attribute (#7465) 2018-06-07 00:42:38 -04:00
test_mmap.py bpo-33767: Fix improper use of SystemError by mmap.mmap objects (GH-7381) 2018-06-05 15:59:41 +03:00
test_module.py
test_modulefinder.py
test_msilib.py
test_multibytecodec.py
test_multiprocessing_fork.py
test_multiprocessing_forkserver.py
test_multiprocessing_main_handling.py
test_multiprocessing_spawn.py
test_netrc.py
test_nis.py
test_nntplib.py
test_normalization.py
test_ntpath.py
test_numeric_tower.py
test_opcodes.py bpo-32911: Revert bpo-29463. (GH-7121) (GH-7197) 2018-05-29 12:04:55 +03:00
test_openpty.py
test_operator.py
test_optparse.py
test_ordered_dict.py
test_os.py
test_ossaudiodev.py
test_osx_env.py
test_parser.py
test_pathlib.py
test_pdb.py bpo-33562: Check the global asyncio event loop policy isn't set after any tests (GH-7328) 2018-06-01 20:34:09 -07:00
test_peepholer.py
test_pickle.py
test_pickletools.py
test_pipes.py
test_pkg.py
test_pkgimport.py
test_pkgutil.py
test_platform.py
test_plistlib.py
test_poll.py
test_popen.py
test_poplib.py bpo-33618: Enable TLS 1.3 in tests (GH-7079) 2018-05-23 22:24:45 +02:00
test_posix.py bpo-31044, test_posix: Reenable makedev() tests on FreeBSD (#7449) 2018-06-06 15:28:50 +02:00
test_posixpath.py
test_pow.py
test_pprint.py
test_print.py
test_profile.py
test_property.py
test_pstats.py
test_pty.py
test_pulldom.py
test_pwd.py
test_py_compile.py
test_pyclbr.py
test_pydoc.py
test_pyexpat.py
test_queue.py
test_quopri.py
test_raise.py
test_random.py
test_range.py
test_re.py
test_readline.py
test_regrtest.py bpo-33718: regrtest: use "xxx then yyy" result if re-run (GH-7521) 2018-06-08 09:53:51 +02:00
test_repl.py
test_reprlib.py
test_resource.py
test_richcmp.py
test_rlcompleter.py
test_robotparser.py
test_runpy.py
test_sax.py
test_sched.py
test_scope.py
test_script_helper.py
test_secrets.py
test_select.py
test_selectors.py
test_set.py
test_setcomps.py
test_shelve.py
test_shlex.py
test_shutil.py bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win) (#7160) 2018-06-12 23:04:50 +02:00
test_signal.py
test_site.py
test_slice.py
test_smtpd.py
test_smtplib.py
test_smtpnet.py
test_sndhdr.py
test_socket.py
test_socketserver.py
test_sort.py
test_source_encoding.py
test_spwd.py
test_sqlite.py
test_ssl.py bpo-31432: Revert unrelated code changes to _ssl.c and test_ssl (GH-7650) 2018-06-11 20:26:28 -04:00
test_startfile.py
test_stat.py
test_statistics.py
test_strftime.py
test_string.py
test_string_literals.py
test_stringprep.py
test_strptime.py
test_strtod.py
test_struct.py
test_structmembers.py
test_structseq.py
test_subclassinit.py
test_subprocess.py Improve the subprocess restore_signals=True test. (GH-7414) 2018-06-05 12:00:57 -07:00
test_sunau.py
test_sundry.py
test_super.py
test_support.py bpo-33773: Fix test.support.fd_count() on Linux/FreBSD (GH-7421) 2018-06-06 17:23:50 +02:00
test_symbol.py
test_symtable.py
test_syntax.py
test_sys.py
test_sys_setprofile.py
test_sys_settrace.py bpo-33562: Check the global asyncio event loop policy isn't set after any tests (GH-7328) 2018-06-01 20:34:09 -07:00
test_sysconfig.py
test_syslog.py
test_tarfile.py
test_tcl.py
test_telnetlib.py
test_tempfile.py bpo-27300: Add the errors parameter to tempfile classes. (GH-6696) 2018-05-23 08:07:01 +03:00
test_textwrap.py
test_thread.py
test_threaded_import.py
test_threadedtempfile.py
test_threading.py
test_threading_local.py
test_threadsignals.py
test_time.py
test_timeit.py
test_timeout.py
test_tix.py
test_tk.py
test_tokenize.py bpo-12486: Document tokenize.generate_tokens() as public API (#6957) 2018-06-05 10:26:39 -07:00
test_trace.py
test_traceback.py
test_tracemalloc.py
test_ttk_guionly.py
test_ttk_textonly.py
test_tuple.py
test_turtle.py
test_typechecks.py
test_types.py
test_typing.py
test_ucn.py
test_unary.py
test_unicode.py
test_unicode_file.py
test_unicode_file_functions.py
test_unicode_identifiers.py
test_unicodedata.py update to Unicode 11.0.0 (closes bpo-33778) (GH-7439) 2018-06-06 20:14:28 -07:00
test_unittest.py
test_univnewlines.py
test_unpack.py
test_unpack_ex.py
test_urllib.py
test_urllib2.py
test_urllib2_localnet.py
test_urllib2net.py
test_urllib_response.py
test_urllibnet.py
test_urlparse.py bpo-27485: Change urlparse tests to use private methods. (GH-7070) 2018-06-03 17:31:32 +03:00
test_userdict.py
test_userlist.py
test_userstring.py
test_utf8_mode.py
test_utf8source.py
test_uu.py bpo-33744: Fix test_uu. (GH-7350) 2018-06-03 17:22:42 +03:00
test_uuid.py
test_venv.py
test_wait3.py
test_wait4.py
test_wave.py
test_weakref.py
test_weakset.py
test_webbrowser.py
test_winconsoleio.py
test_winreg.py
test_winsound.py
test_with.py
test_wsgiref.py
test_xdrlib.py
test_xml_dom_minicompat.py
test_xml_etree.py bpo-33761: Fix a file leak in test_iterparse in test_xml_etree. (GH-7358) 2018-06-03 20:56:52 +03:00
test_xml_etree_c.py
test_xmlrpc.py bpo-33759: Fix test.test_xmlrpc.ServerProxyTestCase. (GH-7362) 2018-06-03 22:55:32 -04:00
test_xmlrpc_net.py
test_xxtestfuzz.py
test_yield_from.py
test_zipapp.py
test_zipfile.py
test_zipfile64.py
test_zipimport.py
test_zipimport_support.py
test_zlib.py
testcodec.py
testtar.tar
tf_inherit_check.py
threaded_import_hangers.py
time_hashlib.py
tokenize_tests-latin1-coding-cookie-and-utf8-bom-sig.txt
tokenize_tests-no-coding-cookie-and-utf8-bom-sig-only.txt
tokenize_tests-utf8-coding-cookie-and-no-utf8-bom-sig.txt
tokenize_tests-utf8-coding-cookie-and-utf8-bom-sig.txt
tokenize_tests.txt
win_console_handler.py
wrongcert.pem
xmltests.py
zip_cp437_header.zip
zipdir.zip