This cl supports the parsing and extraction of the zip entry who
has a large size than UINT32_MAX. Also add a few checks in the
entry writers to make sure callers have enough space for extraction.
As many users of the library assume the entry size to be 32 bits long,
we keep the 32 bit ZipEntry. We also keep the functions that expect
the 32 bit ZipEntry in the public header file. These 32 bit wrappers
could be removed later once all users recognize the 64 bit ZipEntry.
Bug: 150900468
Test: unit tests pass
Change-Id: Ia6760638ccf51e97dbef6bd55dff352f1e7ce816
The size fields in the data descriptor can be either 4 bytes or 8 bytes.
This depends on if the size are read from the zip64 extended field in
the local file header. This cl adds support to parse these cases.
Also fix a misconception in that the uncompressed and compressed size
doesn't need to exist together in the zip64 fields of the central
directory. But they still need to co-exist in the fields of the local
file header.
Bug: 150900468
Test: unit tests pass, python tests pass
Change-Id: Ia54f9bf56c85ff456ead90a136f7fddc5be5220c
Add the definition of zip64 related structs. Also add place holders in
the zip parsing code. In addition, this cl changes the variable type of
num of entries to uint64_t. The number was capped at UINT16_MAX in zip32
format.
Bug: 150900468
Test: unit tests pass
Change-Id: I51a39e7b993fa376e0d050a04b8d39abae8a9e15
To allow the ResourcesLoader API to load part of a file as an APK
that contains resources, an additional override of OpenArchiveFd
that contains read offset and length as parameters must be created.
This functionality allows for an APK stored in a zip file to be read
without having to write the APK to disk.
Bug: 142716192
Test: atest FrameworksResourceLoaderTests
Change-Id: I772fc8b462d71de0529717c420ced552103a6e3f
Merged-In: I772fc8b462d71de0529717c420ced552103a6e3f
Add the std::map implementation to be used later in zip64 format.
Also move the entry map classes to a separate file to make the hierarchy
clear.
Test: unittests pass
Change-Id: I74d95f53207cc8ca871b955e2a15c184d5497833
The current implementation of the hashtable uses less memory than
a std::map. As most of the zip files we encountered don't use the zip64
extension, we should keep the current implementation. And the interface
adds the flexibility for us to switch to std::map for zip64 format.
Bug: 150900468
Test: unit tests pass
Change-Id: Ifd008785c9ff416a27049f9e0c54d9eef985bd85
d77c99ebc3 changed MappedFile to return a
bogus zero-length mapping on failure rather than nullptr. None of the
calling code was changed, though, and it seems like doing so would be a
bad idea. Revert that part of the change.
Add missing tests, and tidy up some of the logging. Also remove
single-use or obfuscatory constants from the tests.
The new "empty.zip" was created by using zip(1) to create a zip file
with one entry, then using `zip -d` to remove it.
The new "zero-size-cd.zip" was created by using zip(1) to create a zip
file containing a single empty file, and then hex editing the two byte
"size of the central directory" field in the "end of central directory
record" structure at the end of the file. (This is equivalent to, but
much smaller than, the example zip file provided by the bug reporter.)
Bug: http://b/145925341
Test: treehugger
Change-Id: Iff64673bce7dae886ccbc9dd6c2bbe18de19f9d2
Didn't find anything when I ran it, but it did get me to fix the
const/non-const void* in the API.
Test: treehugger
Change-Id: If3849d974965e3e5ffcbdaf5e47921316d717410
This allows us to remove libziparchive's dependency on libutils.
Bug: http://b/79112958
Test: ran libbase and libziparchive tests, ran fastboot manually
Change-Id: I95c651976dad222863e5b8c37d4514b778f5dce7
Debugging memory allocations on the nexus launcher unveiled significant memory allocations for the hashatable used in libziparchive, ~1MB.
This is partly because of the ZipString struct storing an entry in the table. The struct stored a pointer to a string (on 64 bit, 8 bytes) and the length to read from that pointer, 2 bytes. Because of alignment, the structure consumed 16 bytes, wasting 6 bytes.
Now, we store entries in the hashtable as a ZipStringOffset. This new structure stores a 4 byte offset from a fixed location in the memory mapped file instead of the entire address, consuming 8 bytes with alignment.
Bug: 79416399
Test: Builds successfully and manual testing by opening launcher on Pixel 2 shows precisely 50% decrease in memory allocated for the hashtable. From 909312 bytes to 454656.
Change-Id: I28b43699233fbee7f63fccae2d4fe96fcc07e5c4
Remove dependencies on zip_archive specific data structures. In follow
up changes, this method will be promoted to a public API and used from
androidfw/ZipUtils as well as tools/zipalign, thereby allowing us to
remove a near-exact copy of this code and to simplify depdencies.
Test: zip_archive_test
Bug: 35246701
Change-Id: If24a9965fbd6fff308c8758859026684fd2af3b9
The use of ReadAtOffset is meant to allow concurrent access
to the zip archive once it has been loaded. There were places
where this was the case, and some places that did a seek + read
combination, which could lead to data races.
NOTE: On Windows, we are not using pread as the implementation of
ReadAtOffset, therefore the guarantees on Windows are weaker.
On Linux, pread allows the file descriptor to be read at a specific
offset without changing the read pointer. This allows inherited fd's
and duped fds to be read concurrently.
On Windows, we use the ReadFile API, which allows for an atomic seek +
read operation, but modifies the read pointer. This means that any mix
use of ReadAtOffset and Read will have races. Just using ReadAtOffset is
safe.
For the Windows case, this is fine as the libziparchive code now only
uses ReadAtOffset.
Bug: 62184114
Bug: 62101783
Test: make ziparchive-tests (existing tests pass)
Change-Id: Ia7f9a30af2216682cdd9d578d26e84bc46773bb9
Make them available to tests so that they can compare values
directly instead of comparing error code strings.
Test: zip_archive_test
Change-Id: I1c78c24651ff7549b5125a49a4e67bbd8295c617
Add two functions libziparchive that libminzip has. And create
corresponding unit tests.
1. Open a zip archive from a memory mapped region.
2. A new writer that takes a call back function pointer.
(Used by the OTA updater to stream the data.)
Test: Unit tests passed
Bug: 19472796
Change-Id: I2b2daec71174afe221030357e39bff5faea51e72
This allows someone to stream the data out of a zip archive
instead of extracting to a file or to memory.
Included in this change is a small cleanup of the makefile.
Change-Id: I8b679a679c3502ff4ea0bc4f9e918303657fa424