Expose response object

[HaNdTriX]

First of all thanks a lot for this nifty crawler. I am really enjoying your api design!

Nevertheless, would it be possible to expose the entire response object in the handler api? Currently supercrawler only exposes contentType, response.body and url.

This would allow us to crawl a lot more information:

• crawl response headers
• handle http status codes/messages
• performing custom redirects

I can create a PR if such a change would be acceptable.

[brendonboshell]

Yes, I would accept a pull request for this provided the change was backwards compatible. Adding a response property to the ctx object seems the most obvious route?