mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-12-23 09:19:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

gh-89792: Prevent test_tools from copying 1000M of "source" in freeze test (#101837)

Browse source 
Prevent test_tools from copying 1000M of "source"
    
It doesn't need a git repo, just the checkout.  We skip .git metadata, Doc/build, Doc/venv, and `__pycache__` subdirs, that developers often have in their clients to reduce the size of the source tree copy ten-fold.

This should significantly reduce IO and presumably time on buildbots during this long test.
This commit is contained in:
Gregory P. Smith 2023-02-11 20:54:28 -08:00 • committed by GitHub
parent 3eb12df8b5
commit 1d194235e4
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 16 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
Misc/NEWS.d/next/Tests/2023-02-11-20-28-08.gh-issue-89792.S-Y5BZ.rst Normal file
View file

@ -0,0 +1,3 @@
``test_tools`` now copies up to 10x less source data to a temporary
directory during the ``freeze`` test by ignoring git metadata and other
artifacts.

14
Tools/freeze/test/freeze.py
View file

@ -80,7 +80,19 @@ def copy_source_tree(newroot, oldroot):
if newroot == SRCDIR:
raise Exception('this probably isn\'t what you wanted')
shutil.rmtree(newroot)
shutil.copytree(oldroot, newroot)
def ignore_non_src(src, names):
"""Turns what could be a 1000M copy into a 100M copy."""
# Don't copy the ~600M+ of needless git repo metadata.
# source only, ignore cached .pyc files.
subdirs_to_skip = {'.git', '__pycache__'}
if os.path.basename(src) == 'Doc':
# Another potential ~250M+ of non test related data.
subdirs_to_skip.add('build')
subdirs_to_skip.add('venv')
return subdirs_to_skip
shutil.copytree(oldroot, newroot, ignore=ignore_non_src)
if os.path.exists(os.path.join(newroot, 'Makefile')):
_run_quiet([MAKE, 'clean'], newroot)