totororequests 1.2.0

Smart Python module for sending HTTP(S) requests through Tor network

Totoro

Totoro is a simple Python module for sending HTTP(S) requests through Tor network.

It provides a way to create a parallel Tor process or connect to an existing Tor running service, and supports all controller authentication methods (none, socket, cookie and password).

Basically it adds an abstract layer on top of the stem module and combines it with requests.

Requirements :

• Python 3
• Python dependencies : requests, stem, fake_useragent

Tested on : Kali 2020.1

License : MIT

Getting started

Installation

Ensure you have Tor installed on your system :

• Debian based : apt install tor
• RHEL based : yum install tor

Then install Totoro : pip3 install totororequests

Example #1 : run a Tor process

In this example we use Totoro in the easiest way. Totoro will run Tor in a new process and generate a random password to protect the controller from being accessed outside the Python script context.

#!/usr/bin/env python3

from totororequests import Totoro

# Get Totoro
toro = Totoro()

# Start Tor process
if toro.start():
    print('Tor is running')

# Get Tor status (will check both SocksPort and ControlPort/ControlSocket connection)
status = toro.status() # True / False

# Force the use of a VPN when sending requests over Tor
toro.require_vpn()

# Get IP information (real IP and Tor IP)
ipinfo = toro.ipinfo()
# {'vpn': {'ip': '85.124.56.92', ...}, 'tor': {'ip': '199.249.230.82', ...}}

# Send a 'direct' request
sess, resp = toro.dirreq('GET', 'https://google.com')

# Send a request over Tor network
sess, resp = toro.torreq('GET', 'https://google.com')

# Stop Tor
if toro.stop():
    print('Tor is not running anymore')

Example #2 : connect to a running Tor instance

In this example we connect Totoro to a running instance of Tor, potentially not on the same host. If you want to change your public IP address later, you'll need to authenticate to the controller (see the associated section).

#!/usr/bin/env python3

from totororequests import Totoro

toro = Totoro()

# Connect to Tor already-running service
if toro.connect(host='127.0.0.1', port=9050):
    print('Successfully connected')

# Authenticate to the controller
authok = toro.authenticate(method='password', port=9051, password='PASSWORD')
if authok:
    print('Successfully authenticated')

# Send a request over Tor network
sess, resp = toro.torreq('GET', 'https://google.com')

# Request a new identity
if authok:
    change_identity(sync=True)

# Send another request over Tor network (with new identity)
sess, resp = toro.torreq('GET', 'https://google.com')

Send requests

Totoro is just a wrapper around the requests library.

To send a request over Tor network, you may use these nice methods :

# toro.get()
# toro.post()
# toro.put()
# toro.delete()
# toro.patch()
# toro.head()
# toro.options()

# Example :
sess, resp = toro.post('https://example.com/login', data={'username':'admin', 'password':'admin'})

As you can see, the method returns a tuple with a Session object (sess) and a Response object (resp).

The resp variable is what you want, it contains the headers of the response and its content.

The sess variable is useful because you can reuse it for a later request. A Session object allows you to persist certain parameters across requests (read more) :

sess, resp2 = toro.get('https://example.com/admin', session=sess)

Doing that you will reuse the Cookies (for instance) you got from the previous request on /login.

Authenticate to the Controller

As a client, it is possible to control Tor's behavior. Totoro lets you request a new identity (= a new IP address), but you must authenticate to the Tor Controller in order to do that.

Two options :

• You have started a Tor process with the start() method (like in Example#1) : in that case you have nothing to do, Totoro is already authenticated to the new controller. It means you can call change_identity().
• You have connected to an already running instance of Tor using connect() (like in Example#2): before requesting a new identity you must follow the steps below.

The controller may be configured in different ways. Totoro supports all the authentication methods. Depending on your torrc configuration, follow the appropriate section.

Auth : None

If your Tor Controller has opened a Control port with no authentication, like this (torrc) :

ControlPort 9051

You can authenticate this way :

toro.authenticate(method=None, port=9051)

Auth : (Safe)Cookie

If your Tor Controller uses (safe)Cookie authentication, when starting Tor a cookie file will be created, generally with 600 rights. This means you should run your Python script with the same user you started Tor, or you'll run into problems. Here is the standard configuration for Cookie auth (torrc) :

ControlPort 9051
CookieAuthentication 1
CookieAuthFile /home/user/.tor/control.authcookie

And you authenticate with Totoro like that :

toro.authenticate(method='cookie', port=9051)

Auth : Password

If you use Password authentication, you must provide a password in order to authenticate to the controller. In the torrc configuration, you need the Hash of the password, obtained with the OpenPGP S2K algorithm :

ControlPort 9051
CookieAuthentication 0
HashedControlPassword 16:FA9FED70DB6AEDE160DE15E9F1CEAE70DEA72B7D4505DC10782FF21AF3

To easily get the hash, use the tor command :

$ tor --hash-password EXAMPLE_PASSWORD

And in your Python code :

toro.authenticate(method='password', port=9051, password='EXAMPLE_PASSWORD')

Auth : Socket

Another way to communicate with the controller is using a Control socket (torrc) :

ControlSocket /home/user/.tor/control.socket

Totoro can connect this way :

toro.authenticate(method='socket', socket="/home/user/.tor/control.socket")

Documentation

Totoro(warnings=False)

Creates the main object.

By default you'll NOT get warnings on stdout/stderr when you perform an HTTPS request with verify=False. To make the warnings appear you can pass warnings=True. Behind the scene it simply does : urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

Totoro.connect(host, port)

Connects to a Tor service. Call it only if you're connecting to an external instance. If you use the classic approach with start(), it creates a Tor process and does everything for you.

Note: You MUST use start() OR connect()+authenticate(), NOT both.

toro.connect(host='127.0.0.1', port=9050)

Returns True or False.

Totoro.start(socks_port=9050, control_port=9051, tor_binary=None, password=None)

Starts a Tor process. That's the traditional way of using Totoro.

Note: You MUST use start() OR connect()+authenticate(), NOT both.

The start process is as follow :

• Choose a password to limit access to the future controller :
  • If a password is provided, use it ;
  • If not, generate a random [a-zA-Z0-9]{25} password.
• Compute the hash of the password :
  • If the tor_binary path is provided (/usr/bin/tor), use the binary to get the hashed password ;
  • If not, compute the hash with the OpenPGP S2K algorithm.
• Start the Tor process, and use socks_port and control_port to configure this newly created instance.

# If ports 9050 and 9051 are available, keep it simple :
toro.start()

Returns True or False.

Totoro.status()

Returns the status of Tor service by checking the connectivity with its Control Port/Socket and Socks port.

if toro.status():
    print('Service available')

Returns True or False.

Totoro.stop(kill=False)

Stops the previously created Tor process.

By default it sends a SIGTERM signal to the process. You can brutally kill it with kill=True.

toro.stop()

Returns True or False.

Totoro.require_vpn(choice=True)

Enables or disables the "VPN Strict Mode".

When enabled, your requests will fail if you're not connected to a VPN tunnel.

toro.require_vpn()

During the script execution you may want to enable/disable this behavior. You can use the parameter to change the setting.

toro.require_vpn(False)

Totoro.vpn_status()

Checks if the system is connected to a VPN.

It basically checks your tun0 interface and routes associated.

if toro.vpn_status():
    print('VPN connection : OK')

Returns True or False.

Totoro.ipinfo()

Performs a direct and a Tor request and returns your IP information.

It uses the IP Geolocation service.

toro.ipinfo()
# {
#     'direct': {
#         'ip': 'xx.xx.xx.xx',
#         'hostname': 'XXXX.abo.wanadoo.fr',
#         'country_code2': 'FR',
#         'country_code3': 'FRA',
#         'country_name': 'France',
#         'state_prov': 'Ile-de-France',
#         'district': '',
#         'city': 'Montmorency',
#         'zipcode': 'ZZZZZ',
#         'latitude': 'XXX',
#         'longitude': 'XXX',
#         'security': {
#             'threat_score': 0,
#             'is_tor': False,
#             'is_proxy': False,
#             'proxy_type': '',
#             'is_anonymous': False,
#             'is_known_attacker': False,
#             'is_cloud_provider': False
#         }
#     },
#     'tor': {
#         'ip': '199.249.230.82',
#         'hostname': 'tor29.quintex.com',
#         'country_code2': 'US',
#         'country_code3': 'USA',
#         'country_name': 'United States',
#         'state_prov': 'Texas',
#         'district': '',
#         'city': 'San Angelo',
#         'zipcode': 'ZZZZZ',
#         'latitude': 'XXX',
#         'longitude': 'XXX',
#         'security': {
#             'threat_score': 7,
#             'is_tor': True,
#             'is_proxy': False,
#             'proxy_type': ' ',
#             'is_anonymous': True,
#             'is_known_attacker': False,
#             'is_cloud_provider': False
#         }
#     }
# }

The returned dictionary is composed by :

• a direct attribute if you're not connected to a VPN ;
• a vpn attribute if you're connected to a VPN ;
• a tor attribute if you're connected to Tor network.

See the IP Geolocation API for information on the returned JSON object.

Totoro.dirreq([...], session=None)

Performs a direct request (with your official public connection).

Use it exactly as you'd use the requests.request method. You must pass the method parameter to specify the HTTP verb.

sess, resp = toro.dirreq('GET', 'https://google.com')

Returns a tuple composed by the Session object and Response.

The session parameter may be passed to reuse a Session from a previous request (and persist data like cookies).

Why this method ? Why not using directly requests without Totoro for this purpose? You're right. The only advantage of this method compared to standard requests.method() is that you can benefit from the VPN Strict Mode and the fake User-Agent.

Totoro.torreq([...], session=None)

Performs a request over the Tor network.

The same as above, you may use it like requests.request. It automatically adds the proxies settings to use Tor.

sess, resp = toro.torreq('GET', 'https://google.com')

Returns a tuple composed by the Session object and Response.

The session parameter may be passed to reuse a Session from a previous request (and persist data like cookies).

In practice you'll use the below helpers...

get(), post(), put(), patch(), delete(), options(), head()

Helpers to make it more user-friendly. It's the same you can use with requests.

All these methods send the request over Tor. They redirect all the parameters to the torreq() method.

sess, resp = toro.get('https://example.com/admin', cookies={'PHPSESSID':'XXXXXX'})
sess, resp = toro.post('https://example.com/login', data={'username':'admin', 'password':'admin'})
# [...]

Then it also returns a tuple with (Session, Response), and you may pass the session parameter as well.

Totoro.annoy(url, times=1, threads=10, sync=False)

Send times requests to url and immediately drops them (without waiting for response).

Consequences : As HTTP is over the TCP protocol, a TCP handshake needs to be performed. When using this method you'll start the handshake and cancel it instantly. It will just tickle / SYN flood the remote server.

Change threads param to control the number of threads used to send the fake requests.

When using sync=True, the requests are sent synchronously, and it blocks your script.

toro.annoy('https://example.com', times=100)

Note : Do not use it to generate fake logs because it won't work. Use make_noise() instead.

Totoro.make_noise(urls, times=1, threads=10, shuffle=False, sync=False, timeout=5)

Sends a series of HTTP requests, times times, in parallel threads and without waiting for responses.

Each request is complete (unlike with the annoy() method) so it appears in the accesslog of the remote server and you may use it to generate fake logs.

The urls parameter must be a list of URL to fetch, in the following formats :

• Just the URL (default to GET verb) : https://example.com/admin
• Verb + [space] + URL : POST https://example.com/admin

When using shuffle=True :

• The total number of sent requests is times (it sends times times a random request from the list) ;
• The requests are sent in whatever order, and randomly picked.

When using shuffle=False :

• The total number of sent requests is times * len(urls) (it sends times times the entire list of requests) ;
• The requests are sent in the order of the list ;
• There is no guarantee that they are received in exact same order (and generally won't).

For instance, this instruction :

toro.make_noise([
    'https://example.com/',
    'https://example.com/favicon.ico',
    'GET https://example.com/css/bootstrap.min.css',
    'GET https://example.com/js/jquery.min.js',
    'POST https://example.com/login',
    'DELETE https://example.com/user/15/avatar'
], times=15, shuffle=True)

...will send 15 requests, randomly picked from the urls list.

When using sync=True, the requests are sent synchronously, and it blocks your script.

Change the timeout param to set a timeout (in seconds) for each request in the thread pool.

Totoro.authenticate(method=None, port=None, socket=None, password=None)

Sets the authentication parameters and instantiate the connection to the controller. Call it only if you're connecting to an external instance.

Note: You MUST use start() OR connect()+authenticate(), NOT both.

Parameters :

• method : specify the authentication method (None, cookie, socket, password)
• port : controller port (required except if method="socket")
• socket : socket path (required only if method="socket")
• password : authentication password (only if method="password")

See the section "Authenticate to the Controller" for more details.

Returns True or False.

Totoro.controller()

Returns the controller object. It is an instance of stem.control.Controller.

Totoro just uses it to change the identity, but you may want to do more...

ctrl = toro.controller()

print('Protocol Info :')
print(ctrl.get_protocolinfo())

print('Circuit Status :')
print(ctrl.get_info('circuit-status'))

Totoro.change_identity(sync=False)

Requests a new identity (and likely a new IP address).

By default it just sends a NEWNYM signal to Tor controller and don't wait. That means it's not synchronous and it may take a few seconds for the new identity to be effective.

toro.change_identity()

You may want to wait until the new identity is OK :

toro.change_identity(sync=True)

Caution : it may block the script during several seconds.

Exceptions

Sometimes an exception can be raised by the Totoro engine.

• TotoroException : different reasons, generally when you provide incorrect parameters to Totoro methods (see the error message for more information) ;
• TorNotRunningTotoroException : you tried to send a request through Tor but Tor is not running ;
• VPNNotConnectedTotoroException : you tried to send a request while VPN Strict Mode is enable, and your VPN connection seems to be broken.

Changelog

Version history :

• 1.2.0 - Added dependencies in setup.py, hid SSL warnings by default, improved newnym request behavior, and made Totoro compatible with requests to .onion domains
• 1.1.3 - Added default User-Agent (fallback) to avoid exceptions when it's not possible to fetch the list from internet
• 1.1.1 & 1.1.2 - Optimization on thread methods (annoy & make_noise) : proper pool closure, requests timeout, better and more elegant code
• 1.1.0 - New methods : annoy() and make_noise()
• 1.0.0 - Initial version of Totoro