Spidy (/spˈɪdi/) is the simple, easy to use command line web crawler.
Given a list of web links, it uses Python requests to query the webpages, and lxml to extract all links from the page.
Created by rivermont (/rɪvɜːrmɒnt/) and FalconWarriorr (/fælcʌnraɪjɔːr/), and developed with help from these awesome people. Looking for technical documentation? Check out DOCS.md Looking to contribute to this project? Have a look at CONTRIBUTING.md, then check out the docs.
spidy Web Crawler alternatives and related packages
Based on the "Web Crawling" category
* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest. Visit our partner's website for more details.
Do you think we are missing an alternative of spidy Web Crawler or a related project?
spidy Web Crawler RecommendationsThere are no recommendations yet. Be the first to promote spidy Web Crawler!
Have you used spidy Web Crawler? Share your experience. Write a short recommendation and spidy Web Crawler, you and your project will be promoted on Awesome Python.
Recommend spidy Web Crawler
Recently added spidy Web Crawler resources
Be the first to add one!