Contrib Modules

These modules implement various extra features, that may not be ready for prime time or that require optional third-party dependencies.

SNI-support for Python 2

SSL with SNI-support for Python 2. Follow these instructions if you would like to verify SSL certificates in Python 2. Note, the default libraries do not do certificate checking; you need to do additional work to validate certificates yourself.

This needs the following packages installed:

  • pyOpenSSL (tested with 0.13)
  • ndg-httpsclient (tested with 0.3.2)
  • pyasn1 (tested with 0.1.6)

You can install them with the following command:

pip install pyopenssl ndg-httpsclient pyasn1

To activate certificate checking, call inject_into_urllib3() from your Python code before you begin making HTTP requests. This can be done in a sitecustomize module, or at any other time before your application begins using urllib3, like this:

try:
    import urllib3.contrib.pyopenssl
    urllib3.contrib.pyopenssl.inject_into_urllib3()
except ImportError:
    pass

Now you can use urllib3 as you normally would, and it will support SNI when the required modules are installed.

Activating this module also has the positive side effect of disabling SSL/TLS compression in Python 2 (see CRIME attack).

If you want to configure the default list of supported cipher suites, you can set the urllib3.contrib.pyopenssl.DEFAULT_SSL_CIPHER_LIST variable.

Module Variables

var DEFAULT_SSL_CIPHER_LIST:
 The list of supported SSL/TLS cipher suites.

Google App Engine

The urllib3.contrib.appengine module provides a pool manager that uses Google App Engine’s URLFetch Service.

Example usage:

from urllib3 import PoolManager
from urllib3.contrib.appengine import AppEngineManager, is_appengine_sandbox

# This substitution will be done automagically once appengine code
# graduates from the contrib module.
if is_appengine_sandbox():
    # AppEngineManager uses AppEngine's URLFetch API behind the scenes
    http = AppEngineManager()
else:
    # PoolManager uses a socket-level API behind the scenes
    http = PoolManager()

# The client API should be consistent across managers, though some features are not available
# in URLFetch and you'll get warnings when you try to use them (like granular timeouts).
r = http.request('GET', 'https://google.com/')

There are limitations to the URLFetch service and it may not be the best choice for your application. App Engine provides three options for urllib3 users:

  1. You can use AppEngineManager with URLFetch. URLFetch is cost-effective in many circumstances as long as your usage is within the limitations.

  2. You can use a normal PoolManager by enabling sockets. Sockets also have limitations and restrictions and have a lower free quota than URLFetch. To use sockets, be sure to specify the following in your app.yaml:

    env_variables:
    GAE_USE_SOCKETS_HTTPLIB : 'true'

  3. If you are using Managed VMs, you can use the standard PoolManager without any configuration or special environment variables.

SOCKS Proxies

New in version 1.14.

The urllib3.contrib.socks module enables urllib3 to work with proxies that use either the SOCKS4 or SOCKS5 protocols. These proxies are common in environments that want to allow generic TCP/UDP traffic through their borders, but don’t want unrestricted traffic flows.

To use it, either install PySocks or install urllib3 with the socks extra, like so:

$ pip install -U urllib3[socks]

The SOCKS module provides a SOCKSProxyManager that can be used when SOCKS support is required. This class behaves very much like a standard ProxyManager, but allows the use of a SOCKS proxy instead.

Using it is simple. For example, with a SOCKS5 proxy running on the local machine, listening on port 8889:

from urllib3.contrib.socks import SOCKSProxyManager

http = SOCKSProxyManager('socks5://localhost:8889/')
r = http.request('GET', 'https://www.google.com/')

The SOCKS implementation supports the full range of urllib3 features. It also supports the following SOCKS features:

  • SOCKS4
  • SOCKS4a
  • SOCKS5
  • Usernames and passwords for the SOCKS proxy

The SOCKS module does have the following limitations:

  • No support for contacting a SOCKS proxy via IPv6.
  • No support for reaching websites via a literal IPv6 address: domain names must be used.