bpo-28494: Improve zipfile.is_zipfile reliability
The zipfile.is_zipfile function would only search for the EndOfZipfile
section header. This failed to correctly identify non-zipfiles that
contained this header. Now the zipfile.is_zipfile function verifies
the first central directory entry.
Changes:
* Extended zipfile.is_zipfile to verify zipfile catalog
* Added tests to validate failure of binary non-zipfiles
* Reuse 'concat' handling for is_zipfile
Co-authored-by: John Jolly <john.jolly@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
Co-authored-by: Rogdham <contact@rogdham.net>
* Add ZipFile.data_offset attribute
This attribute provides the offset to zip data from the start of the file, when available.
* Add blurb-it
* Try fixing class ref in NEWS
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
When `zipfile.Path.open` is called, the implementation will check
whether the path already exists in the ZIP file. However, this check is
only required when the ZIP file is in read mode. By swapping arguments
of the `and` operator, the short-circuiting will prevent the check from
being run in write mode.
This change will improve the performance of `open()`, because checking
whether a file exists is slow in write mode, especially when the archive
has many members.
GH-113977, GH-120754: Remove unbounded reads from zipfile
Read without a size may read an unbounded amount of data + allocate
unbounded size buffers. Move to capped size reads to prevent potential
issues.
Co-authored-by: Daniel Hillier <daniel.hillier@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
As reported in #117847 and #115366, an unpaired backtick in a docstring
tends to confuse e.g. Sphinx running on subclasses of standard library
objects, and the typographic style of using a backtick as an opening
quote is no longer in favor. Convert almost all uses of the form
The variable `foo' should do xyz
to
The variable 'foo' should do xyz
and also fix up miscellaneous other unpaired backticks (extraneous /
missing characters).
No functional change is intended here other than in human-readable
docstrings.
* Add name and mode attributes for compressed and archived file-like objects
in modules bz2, lzma, tarfile and zipfile.
* Change the value of the mode attribute of GzipFile from integer (1 or 2)
to string ('rb' or 'wb').
* Change the value of the mode attribute of ZipExtFile from 'r' to 'rb'.
This just documents the parameter that already exists.
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Avoid race conditions in the creation of directories during concurrent
extraction in tarfile and zipfile.
Co-authored-by: Samantha Hughes <shughes-uk@users.noreply.github.com>
Co-authored-by: Peder Bergebakken Sundt <pbsds@hotmail.com>
Make zipfile.ZipInfo.compress_level public.
A property is used to retain the behavior of the ._compresslevel.
People constructing zipfile.ZipInfo instances to pass into existing APIs to control per-file compression levels already treat this as public, there was never a reason for it not to be.
I used the more modern name compress_level instead of compresslevel as the keyword argument on other ZipFile APIs is called to be consistent with compress_type and a general long term preference of not runningwordstogether without a separator in names.
Allow extraction (no-op) of a "/" folder in a zipfile, they are commonly added by some archive creation tools.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* Refactor zipfile._strip_extra to use higher level abstractions for extras instead of a heavy-state loop.
* Add blurb
* Remove _strip_extra and use _Extra.strip directly.
* Use memoryview to avoid unnecessary copies while splitting Extras.
Fix Zip64 extensions not being properly applied in some cases:
Fixes an issue where adding a small file to a `ZipFile`
object while forcing zip64 extensions causes an extra Zip64 record to be
added to the zip, but doesn't update the `min_version` or file sizes in
the primary central directory header.
Also fixed an edge case in checking if zip64 extensions are required:
This fixes an issue where if data requiring zip64 extensions was added
to an unseekable stream without specifying `force_zip64=True`, zip64
extensions would not be used and a RuntimeError would not be raised when
closing the file (even though the size would be known at that point).
This would result in successfully writing corrupt zip files.
Deciding if zip64 extensions are required outside of the `FileHeader`
function means that both `FileHeader` and `_ZipWriteFile` will always be
in sync. Previously, the `FileHeader` function could enable zip64
extensions without propagating that decision to the `_ZipWriteFile`
class, which would then not correctly write the data descriptor record
or check for errors on close.
If anyone is actually using `ZipInfo.FileHeader` as a public API without
explicitly passing True or False in for zip64, their own code may still be
susceptible to that kind of bug unless they make a similar change to
where the zip64 decision happens.
Fixes#103861
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org>
This causes the zipfile module to also consider the character defined by
`os.altsep` (if there is one) to be a path separator and convert it to a
forward slash, as defined by the zip specification.
A logical no-op on all known platforms today as os.altsep is currently only set to a meaningful value on Windows (where it is "/").
The zipfile.Path open() and read_text() encoding parameter can be supplied as a positional argument without causing a TypeError again. 3.10.0b1 included a regression that made it keyword only.
Documentation update included as users writing code to be compatible with a wide range of versions will need to consider this for some time.
* gh-98098: Move zipfile into a package.
* Moved test_zipfile to a package
* Extracted module for test_path.
* Add blurb
* Add jaraco as owner of zipfile.Path.
* Synchronize with minor changes found at jaraco/zipp@d9e7f4352d.
* gh-98108: Sync with zipp 3.9.1 adding pickleability.
* gh-98098: Move zipfile into a package.
* Moved test_zipfile to a package
* Extracted module for test_path.
* Add blurb
* Add jaraco as owner of zipfile.Path.
* Synchronize with minor changes found at jaraco/zipp@d9e7f4352d.
