Pool Manager

class urllib3.PoolManager(num_pools=10, headers=None, **connection_pool_kw)

Bases: urllib3.request.RequestMethods

Allows for arbitrary requests while transparently keeping track of necessary connection pools for you.

Parameters
  • num_pools – Number of connection pools to cache before discarding the least recently used pool.

  • headers – Headers to include with all requests, unless other headers are given explicitly.

  • **connection_pool_kw – Additional parameters are used to create fresh urllib3.connectionpool.ConnectionPool instances.

Example:

>>> manager = PoolManager(num_pools=2)
>>> r = manager.request('GET', 'http://google.com/')
>>> r = manager.request('GET', 'http://google.com/mail')
>>> r = manager.request('GET', 'http://yahoo.com/')
>>> len(manager.pools)
2
clear()

Empty our store of pools and direct them all to close.

This will not affect in-flight connections, but they will not be re-used after completion.

connection_from_context(request_context)

Get a ConnectionPool based on the request context.

request_context must at least contain the scheme key and its value must be a key in key_fn_by_scheme instance variable.

connection_from_host(host, port=None, scheme='http', pool_kwargs=None)

Get a ConnectionPool based on the host, port, and scheme.

If port isn’t given, it will be derived from the scheme using urllib3.connectionpool.port_by_scheme. If pool_kwargs is provided, it is merged with the instance’s connection_pool_kw variable and used to create the new connection pool, if one is needed.

connection_from_pool_key(pool_key, request_context=None)

Get a ConnectionPool based on the provided pool key.

pool_key should be a namedtuple that only contains immutable objects. At a minimum it must have the scheme, host, and port fields.

connection_from_url(url, pool_kwargs=None)

Similar to urllib3.connectionpool.connection_from_url().

If pool_kwargs is not provided and a new pool needs to be constructed, self.connection_pool_kw is used to initialize the urllib3.connectionpool.ConnectionPool. If pool_kwargs is provided, it is used instead. Note that if a new pool does not need to be created for the request, the provided pool_kwargs are not used.

proxy = None
urlopen(method, url, redirect=True, **kw)

Same as urllib3.connectionpool.HTTPConnectionPool.urlopen() with custom cross-host redirect logic and only sends the request-uri portion of the url.

The given url parameter must be absolute, such that an appropriate urllib3.connectionpool.ConnectionPool can be chosen for it.

class urllib3.ProxyManager(proxy_url, num_pools=10, headers=None, proxy_headers=None, **connection_pool_kw)

Bases: urllib3.poolmanager.PoolManager

Behaves just like PoolManager, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs.

Parameters
  • proxy_url – The URL of the proxy to be used.

  • proxy_headers – A dictionary containing headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication.

Example:
>>> proxy = urllib3.ProxyManager('http://localhost:3128/')
>>> r1 = proxy.request('GET', 'http://google.com/')
>>> r2 = proxy.request('GET', 'http://httpbin.org/')
>>> len(proxy.pools)
1
>>> r3 = proxy.request('GET', 'https://httpbin.org/')
>>> r4 = proxy.request('GET', 'https://twitter.com/')
>>> len(proxy.pools)
3
connection_from_host(host, port=None, scheme='http', pool_kwargs=None)

Get a ConnectionPool based on the host, port, and scheme.

If port isn’t given, it will be derived from the scheme using urllib3.connectionpool.port_by_scheme. If pool_kwargs is provided, it is merged with the instance’s connection_pool_kw variable and used to create the new connection pool, if one is needed.

urlopen(method, url, redirect=True, **kw)

Same as HTTP(S)ConnectionPool.urlopen, url must be absolute.