scrapes search engine pages for query titles, descriptions and links
Project description
Search Engine Parser
Package to query popular search engines and scrape for result titles, links and descriptions
Installation
pip install search_engine_parser
Development
git clone git@github.com:bisoncorps/search-engine-parser.git
Documentation
Running the tests
cd search-engine-parser/
python tests/__init__.py
Usage
Query Results can be scraped from popular search engines as shown in the example snippet below
from search_engine_parser import YahooSearch, GoogleSearch, BingSearch
import pprint
search_args = ('preaching to the choir', 1)
gsearch = GoogleSearch()
ysearch = YahooSearch()
bsearch = BingSearch()
gresults = gsearch.search(*search_args)
yresults = ysearch.search(*search_args)
bresults = bsearch.search(*search_args)
a = {
"Google": gresults,
"Yahoo": yresults,
"Bing": bresults}
# pretty print the result from each engine
for k, v in a.items():
print(f"-------------{k}------------")
pprint.pprint(v)
# print first title from google search
print(gresults["titles"][0])
# print 10th link from yahoo search
print(yresults["links"][9])
# print 6th description from bing search
print(bresults["descriptions"][5])
Using the cli
usage: cli.py [-h] [-e ENGINE] -q QUERY [-p PAGE] [-t TYPE] [-r RANK]
SearchEngineParser
optional arguments:
-h, --help show this help message and exit
-e ENGINE, --engine ENGINE
Engine to use for parsing the query e.g yahoo
(default: google)
-q QUERY, --query QUERY
Query string to search engine for
-p PAGE, --page PAGE Page of the result to return details for (default: 1)
-t TYPE, --type TYPE Type of detail to return i.e links, desciptions or
titles
-r RANK, --rank RANK Rank of detail to return e.g 5 (default: 1)
Contribution
You are very welcome to modify and use them in your own projects.
Please keep a link to the original repository. If you have made a fork with substantial modifications that you feel may be useful, then please open a new issue on GitHub with a link and short description.
License (MIT)
This project is opened under the MIT 2.0 License which allows very broad use for both academic and commercial purposes.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for search-engine-parser-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | a1378bb7e5b8662e40603c2fe688155bb309fa8c80267ce2158eac4f6750100e |
|
MD5 | e4a30cd55a8b1d833f8bef7b09e628f1 |
|
BLAKE2b-256 | 260c91410c0726b2be7e8ff027f049691f38b72f0ce7bff4ca0544dac0cb04df |