« Changelog History


spidy Web Crawler v1.2 Release Notes

Release Date: 2017-09-07 // over 6 years ago

  • ➕ Added domain restrictions. Crawling can now be limited to a certain domain, such as wsj.com, https://www.wsj.com, or https://www.wsj.com/article. Can be set when entering configuration settings or in the config files.
    🛠 Also more bugfixes and MIME types because those are cool.