Downloading Text file gives an issue while in resume mode.

[jagtapajittheneo]

I am trying to to download text file form a URL.

Grab downloads it perfectly first time.

But if I make some small change in source text file and try to download again; Grab append that change to end of a file.
now downloaded text file is of a no use.

Can you please help in this regards?

[cavaliercoder]

Interesting problem! At this stage, grab is stateless; it doesn't store file progress or checksums anywhere. This could be solved by the parent program using E-Tags or Last-Modified headers.

I may also be able to account for this issue in grab by comparing local and remote timestamps, but this is prone to error when either party has incorrect config.

I'll think on this some more. Thanks for raising the issue.

[cavaliercoder]

There's another complication to this issue:
Incomplete downloads will always appear to be more recent than a remote file, as the local timestamp is only set once the download is complete. This means incomplete downloads will always assume the local data is for the most recent version of the file and will therefore never be overwritten.

Example:

• remote file is dated c.2000
• grab downloads 50% of file - local timestamp is current system time
• remote file is modified at origin and now dated c.2010
• grab compares local time to remote time - local file appears to be up to date
• grab resumes download from 50% and corrupts the local file

[cavaliercoder]

I've created PR #19 to track development for this issue.

[cavaliercoder]

Please see notes on #21. For now, grab does not offer any correctness guarantees above those in net/http. The parent application can take advantage of checksums or implement their own state storage to overcome this issue.