Compress and Decompression

This Application is used for compression and decompression of folder and files in the output directory

Problem Statement Write a program that compresses files and folders into a set of compressed files such that of each compressed file doesn’t exceed a maximum size. The same program can be used for decompressing the files that it has generated earlier. The output of the decompression should be identical to the original input of the compression process.

When compression is needed, the program takes 3 parameters:

Path to Input directory. This directory will contain some files and folders. Path to Output directory. This is where the program should write the compressed files to. Maximum compressed size per file expressed in MB. When decompression is needed, the program takes 2 parameters:

Path to Input directory. This directory contains the compressed files that were generated by the same program. Path to Output directory. This is where the program should put the decompressed files & folders. Consider the following when writing the code:

Some files may be greater than the JVM memory. Some input files (even when compressed) may be greater than the maximum size allowed for output files. You can use zip for compression algorithm (using JDK’s implementation don’t use a third party library for that) but design the program to allow support different compression algorithms in the future. When compressing, generate as few files as possible. You are not required to generate the absolute minimum number of compressed files but doing so would be a plus. <> Make the compression process run in parallel to speed up the compression for a single directory. Write your production-quality code in Java or Scala. Please test your system adhering to best practices. Submit your solution by using the link that was included in the email

Solution Usage Compression usage : compress inputDirectory outputDirectory maxFileSizeInMB

For example : compress src/test/resources/testData src/test/resources/compressDir/ 100.0

Decompression usage : extract inputDirectory outputDirectory

For Example: extract src/test/resources/compressedTestDir/ src/test/resources/decompressDir/

Implementation

Compressing a directory, this application iterates all the files and read the data sequentially, compress it and write it to the zip folder

Decompression a zip file, this application read the data sequentially and decompress the file written in the extract folder

This program works for files which are larger than the available JVM memory as the entire file is never loaded to memory.

Load Testing Compressed test data(65.6 MB media files) containing 20 files with maxSplitSize of 5 MB in 2.5s to 13 zip files Decompressed 13 zip files to 65.6 MB of data containing 17 files in 0.5s

Compressed 1.7GB containing 150 files with maxSplitSize of 100MB in 200s Decompressed zip files to 1.7GB in 99s