After cloning the repo:

• Add CRAWL_HOME in crawl-envs
• ./crawl.sh --help

---

usage:
./crawl.sh --prepareToCrawl <site-name> #prepare the site for crawl
./crawl.sh --crawl <site-name> <depth> <topN(optional)>#start crawl
./crawl.sh --help #print this message

---