Support large Zip files

[lzhou2025]

The changes are to remove the 4GB size limit of Zip files and support LARGE FILE for JVM in 64-bit and 32-bit environments.

Fixes: #23441, #23458

[lzhou2025]

I believe all comments are resolved, please reopen if missed.

[keithc-ca]

The commit message would be improved by explicitly stating that this support is not yet complete for 32-bit platforms.

[lzhou2025]

I believe all comments are resolved, please reopen if missed.
For the legacy codes written decades ago, signed 32-bit integer (IDATA) is widely used in 32-bit platforms, will be replaced by I_64 in the next PR to support #23458 and LARGE FILE in 32-bit platforms.

[lzhou2025]

That IDATA is widely used in the zip codes appears because of OMR,

omrfile_read(struct OMRPortLibrary *portLibrary, intptr_t fd, void *buf, intptr_t nbytes)

intptr_t should be uintptr_t or size_t, because the read() calls of Unix like systems or POSIX are size_t, unsigned.

[keithc-ca]

omrfile_read(struct OMRPortLibrary *portLibrary, intptr_t fd, void *buf, intptr_t nbytes)

intptr_t should be uintptr_t or size_t, because the read() calls of Unix like systems or POSIX are size_t, unsigned.

Yes, perhaps the OMR API could be improved, but that is unrelated to the concepts involved here. The file format allows for large entries and files, perhaps larger than could exist in memory at any given time. Things could still be made to work if OMR only allowed reading 32kB in a single call (in support of even 32-bit zip files).

[lzhou2025]

omrfile_read(struct OMRPortLibrary *portLibrary, intptr_t fd, void *buf, intptr_t nbytes)

intptr_t should be uintptr_t or size_t, because the read() calls of Unix like systems or POSIX are size_t, unsigned.

Yes, perhaps the OMR API could be improved, but that is unrelated to the concepts involved here. The file format allows for large entries and files, perhaps larger than could exist in memory at any given time. Things could still be made to work if OMR only allowed reading 32kB in a single call (in support of even 32-bit zip files).

omrfile_read(struct OMRPortLibrary *portLibrary, intptr_t fd, void *buf, intptr_t nbytes)

intptr_t should be uintptr_t or size_t, because the read() calls of Unix like systems or POSIX are size_t, unsigned.

Yes, perhaps the OMR API could be improved, but that is unrelated to the concepts involved here. The file format allows for large entries and files, perhaps larger than could exist in memory at any given time. Things could still be made to work if OMR only allowed reading 32kB in a single call (in support of even 32-bit zip files).

The ZIP format supports up to 4Gb size of each individual uncompressed file. But depending how to decompress it, if trying to allocate that size of memory buffer in 32-bit system, I agree, that will exceed the maximum memory space.

[lzhou2025]

OMR layer doesn't know what size of data is required in applications. Before the zip functions call OMR interfaces, should allocate memory buffers, which is fit to uncompress logic.

[lzhou2025]

ZIP64 & LARGE FILE support in two PRs:

• This PR will support it for JVM/64-bit environment
• Next PR (Support ZIP64 for JVM/32-bit lzhou2025/openj9#1), will support it for JVM/32-bit environment

[lzhou2025]

I believe all comments related to this PR are resolved.

[keithc-ca]

This pattern is incompatible with DDR. Field types may not change over time, nor vary by platform.
For example, on 32-bit platforms this would yield

    // U16 totalEntries
    @com.ibm.j9ddr.GeneratedFieldAccessor(offsetFieldName="_totalEntriesOffset_", declaredType="U16")
    public U16 totalEntries() throws CorruptDataException {
        return new U16(getShortAtOffset(J9ZipCentralEnd32._totalEntriesOffset_));
    }

    // U16 totalEntries
    public U16Pointer totalEntriesEA() throws CorruptDataException {
        return U16Pointer.cast(nonNullFieldEA(J9ZipCentralEnd32._totalEntriesOffset_));
    }

and on 64-bit platforms

    // I32 totalEntries
    @com.ibm.j9ddr.GeneratedFieldAccessor(offsetFieldName="_totalEntriesOffset_", declaredType="I32")
    public IDATA totalEntries() throws CorruptDataException {
        return new I32(getIntAtOffset(J9ZipCentralEnd._totalEntriesOffset_));
    }

    // I32 totalEntries
    public IDATAPointer totalEntriesEA() throws CorruptDataException {
        return IDATAPointer.cast(nonNullFieldEA(J9ZipCentralEnd._totalEntriesOffset_));
    }

The return types of those methods differ so jdmpview on a 32-bit platform would encounter NoSuchMethodError accessing a system dump from a 64-bit platform and vice versa (even if we ignore the difficulties writing code that's compatible with both sets of signatures).

The 64-bit declarations must be used exclusively.