10

8

6

4

2


9.4
0.0

9.3
0.0

7.9
0.0

7.6
0.0

7.4

6.8

7.1
0.0

19 Web Content Extracting packages and projects

  • TWINT

    9.4 0.0 Python
    An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
  • newspaper

    9.3 0.0 L3 Python
    newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
  • The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
    Promo
  • python-goose

    7.9 0.0 HTML
    Html Content / Article Extractor, web scrapping lib in Python
  • textract

    7.6 0.0 HTML
    extract text from any document. no muss. no fuss.
  • sumy

    7.4 6.8 L5 Python
    Module for automatic summarization of text documents and HTML pages.
  • toapi

    7.1 0.0 Python
    Every web site provides APIs.
  • python-readability

    6.6 3.4 Python
    fast python port of arc90's readability tool, updated to match latest readability.js!
  • trafilatura

    6.4 8.6 Python
    Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
  • html2text

    5.7 6.2 L1 Python
    Convert HTML to Markdown-formatted text.
  • Goose3

    4.2 6.6 HTML
    A Python 3 compatible version of goose http://goose3.readthedocs.io/en/latest/index.html
  • micawber

    3.9 4.8 L5 Python
    a small library for extracting rich content from urls
  • lassie

    3.7 0.0 L4 HTML
    Web Content Retrieval for Humans™
  • opengraph

    3.0 0.0 L5 Python
    A python module to parse the Open Graph Protocol
  • inscriptis -- HTML to text conversion library, command line client and Web service

    2.6 8.0 Python
    A python based HTML to text conversion library, command line client and Web service.
  • Haul

    2.4 0.0 L5 Python
    An Extensible Image Crawler
  • htmldate

    2.0 7.6 Python
    Fast and robust date extraction from web pages, with Python or on the command-line
  • sanitize

    1.5 0.0 L4 Python
    Bringing sanity to world of messed-up data
  • JSONPATH

    1.0 6.3 Python
    A query expression for extracting data from JSON.
  • Data Extractor

    0.9 6.4 Python
    Combine XPath, CSS Selectors and JSONPath for Web data extracting.

Add another 'Web Content Extracting' Package