Request

class Request(url, params=None, headers=None, *, timeout=None, max_retries=5, use_proxy=False, get_method='get', head_method='head', **request_kwargs)

RELOAD_ATTRS = ('response', 'head_response', 'success', 'head_success', 'text', 'json', 'bs', 'browser', 'page')

RESET_ATTRS = ('response', 'head_response', 'success', 'head_success', 'text', 'json', 'bs', 'browser', 'page')

coroutine all(requests, *, timeout=None, predicate=None)

Get all requests that fulfill predicate

Parameters

• requests (Iterable[Request]) – Iterable of Request instances

• timeout (Optional[float]) – timeout for ALL requests together

• predicate (Optional[Callable[[Request], Awaitable[bool]]]) – condition for a Request to pass (defaults to head_success)

Return type

List[Request]

Returns

List of Requests that fulfilled predicate

browser

bs

Return type

BeautifulSoup

classmethod create_soup(html)

Return type

BeautifulSoup

coroutine first(requests, *, timeout=None, predicate=None)

Get first request that fulfills predicate (or None)

Parameters

• requests (Iterable[Request]) – Iterable of requests

• timeout (Optional[float]) – Timeout for ALL requests together

• predicate (Optional[Callable[[Request], Awaitable[bool]]]) – Predicate to fulfill (defaults to head_success)

Return type

Optional[Request]

Returns

Optional Request instance

classmethod from_state(state)

Return type

Request

head_response

Return type

ClientResponse

head_success

Return type

bool

headers

json

Return type

Dict[str, Any]

page

coroutine perform_request(self, method, **kwargs)

Return type

ClientResponse

raw_finalised_url

Return type

str

redirected_url

Return type

URL

reload()

reset(attrs=None)

Return type

None

response

Return type

ClientResponse

coroutine staggered_request(self, method, url, **kwargs)

Return type

ClientResponse

state

Get a json serializable dictionary containing the state of this request.

Return type

dict

Returns

Dict

success

Return type

bool

text

Return type

str

track_telemetry(url, method, using_proxy)

Return type

None

coroutine try_req(req, *, predicate=None)

Return request if it passes predicate, otherwise None

Parameters

• req (Request) – Request to check

• predicate (Optional[Callable[[Request], Awaitable[bool]]]) – Predicate to check on req, defaults to head_success

Return type

Optional[Request]

Returns

req if it passes predicate, else None

url

Return type

str

yarl

class UrlFormatter(fields=None, proxy_domains=None)

add_field(key, value, *, use_proxy=None)

Return type

None

add_fields(fields=None, **kwargs)

Return type

None

add_pool(pool, *, use_proxy=None)

Return type

None

coroutine get_value(self, key, args, kwargs)

Return type

Any

should_use_proxy(url)

Return type

bool

use_proxy(key, use=True)