heavy memory usage when extracting large zip file

[sculxp]

instead of read the whole body into a buffer, can we try zipstream like java ZipInputStream?

github.com/xenking/zipstream

        zr := zipstream.NewReader(&zipFile)
	meta, err := zr.Next()
	if err != nil {
		if err != io.EOF {
			panic(err)
		}
	}
	log.Printf("file name: %s", meta.Name)
	compressedFile, err := ioutil.ReadAll(zr)
	if err != nil {
		panic(err)
	}
	log.Printf("file content: %s", string(compressedFile[:]))

[cmaglie]

@sculxp
can you provide an example that demonstrates the huge memory load?
In theory, we should already support streaming zip files.

[farazkhawaja]

any update on buffered extraction?

[cmaglie]

@farazkhawaja what do you mean with "buffered extraction"?

[farazkhawaja]

Nevermind. What i wanted to ask is what was originally asked. For large tarballs ie; 4-5 gb the pod memory spikes up to 10K-15K Mi during extraction. Is that a known issue?