download hang

[martingg88]

the download will be stop there forever if i specify chunksAtOnce more than 5. any idea?

[martingg88]

any update about this issue?

[zenflow]

Sorry for the delay.

I know that http servers typically pool connections and typically allow a maximum of 5 connections from the same client. Servers can be configured to accept more or less, but in my experience that is atypical. So it is unlikely that you will be able to take advantage of any more than 5 connections

...unless some of those connections were proxied through another IP address and appeared to be coming from a different client, which is a pretty cool idea. I just added an issue for this but I don't have any idea of when I will get around to it.

Anyways! Clearly the download should not hang when chunksAtOnce > 5 and I intend to correct this and add some real tape tests very soon... Stay tuned!

[martingg88]

i realize that node request module doesn't emit end event in consistently and thus cause the readable stream doesn't end nicely and finally the downloading hang there forever. here is the example for this issue.

I have issued 10 requests (etc: chunksAtOnce) and get back 10 response as well. however i do have an issue that the "end" event may be emitted less than 10. Any idea how to fix this issue?

[zenflow]

I wasn't able to reproduce this malfunction in tests/run.js (without downloading a different file) so I guess request fails to emit 'end' for only some files on http when there's a lot of connections going on.

Anyways, I made some changes 540f1a6, so a Chunk will emit 'end' when it has received all the needed data, and not rely on the request to emit 'end'.

Included in v0.3.5, already pushed to npm.

Please try it out and tell me if it works for the URLs you are trying to download.

[martingg88]

still have the same issue.
here is the outcome.

1. issue 100 http request.
2. get back 100 http response with status code 206
3. get back less than 100 events for end event for sure. it may be node request or http request issue.
4. added check point and confirm get back 100 events for data event. it could be self.position==self.size emit inconsistent end event?

[martingg88]

you have any idea about this issue?

[zenflow]

Can you share a URL on the web that this is happening for? I'll try to reproduce your results

[martingg88]

sure.here is a few step to get youtube encoded url.

1. visit to http://en.savefrom.net/ and paste the following link into text box.
   https://www.youtube.com/watch?v=cFP4QFDG3vE
2. after that, right click "download" button can copy the link address.
3. finally, you should be able to use the link address you copy for download purpose.
   4 remember set chunkatsize to 100

[martingg88]

do you have skype?

[martingg88]

are you able to reproduce the issue?