-
web-crawljs now has dynamicSchemas that allow you change the structure of nextSelector, nextSelectBy, fetchSelector and fetchSelectBy if the url matches a url yu specify.
-
The loop and loopFn have been renamed to depth and depthFn.
-
The issue with finalFn in the formal version is now fixed.
-
The function format the next url to crawl. It takes a url and returns a new url. this url can be a string or an Object that is supported by the request module for making requests.