Skip to main content

Library of web-related functions

Project description

https://secure.travis-ci.org/scrapy/w3lib.png?branch=master

Overview

This is a Python library of web-related functions, such as:

  • remove comments, or tags from HTML snippets

  • extract base url from HTML snippets

  • translate entites on HTML strings

  • convert raw HTTP headers to dicts and vice-versa

  • construct HTTP auth header

  • converting HTML pages to unicode

  • sanitize urls (like browsers do)

  • extract arguments from urls

Requirements

Python 2.7 or Python 3.3+

Install

pip install w3lib

Documentation

See http://w3lib.readthedocs.org/

License

The w3lib library is licensed under the BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

w3lib-1.9.0.tar.gz (12.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

w3lib-1.9.0-py2.py3-none-any.whl (15.3 kB view details)

Uploaded Python 2Python 3

File details

Details for the file w3lib-1.9.0.tar.gz.

File metadata

  • Download URL: w3lib-1.9.0.tar.gz
  • Upload date:
  • Size: 12.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for w3lib-1.9.0.tar.gz
Algorithm Hash digest
SHA256 b124659467de0a161f17ade88d616c2270356c5eeea66aea20285d92efb515f3
MD5 91411a8b0b52279fd889c7ba12f2aad5
BLAKE2b-256 33940f0aef4fc65e0d3c1c21545bd635350389539397a893786e09ef7f8c8405

See more details on using hashes here.

File details

Details for the file w3lib-1.9.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for w3lib-1.9.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 5332b7f36ae2e086536f7ea15aced881a34c69816e246755a259da0074b7878c
MD5 fbecbb660efe720efdf55a5b3903405d
BLAKE2b-256 cb87571b640bb0692c0d23fbf79335ff98545a68db204e67779d337f5c58b67c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page