thatscrapper
Subpackages
Submodules
Package Contents
Classes
A selenium.webdriver adapter. |
|
Attributes
- class thatscrapper.Crawler(browser: str = 'firefox', headless: bool = False)
A selenium.webdriver adapter.
An instance of Window calss cam perform a series of automated actions on webpages. Designed to handle sites with heavy use of javascript.
- driver
- Type
selenium.webdriver
- timeout
- property driver
- property logger
- quitdriver()
- goto(url: str)
- element(value, by='name', step=0.5, timeout=10)
element method.
Select and element from current page.
- Parameters
value (str) – value of the attribute or tag defined in ‘by’
by (str, optional) – attribute, by default ‘name’. See a list of attributes at ATTR_SELECTOR.keys()
step (float, optional) – timeout step, by default 0.5
timeout (int, optional) – timeout until throw error, by default 10
- Returns
Element retrieved.
- Return type
WebElement
- element_id(value, step=0.5, timeout=10)
element_id
Retrieve element from current page by it’s id value.
- Parameters
value (str) – id’s value.
step (float, optional) – timeout step, by default 0.5
timeout (int, optional) – timeout until throw error, by default 10
- Returns
Element retrieved.
- Return type
WebElement
- elements(value, by='name', step=0.5, timeout=10)
- child_of(element, value, by='name', step=0.5, timeout=10)
- children_of(element, value, by='name', step=0.5, timeout=10)
- click_element(element)
- click(value, by='name', step=0.5, timeout=10)
- click_id(id_value, step=0.5, timeout=10)
- send_to_element(element: WebElement, key, enter=False)
send_key similar to Window.send
Send ‘key’ to WebElement ‘element’
- Parameters
element (WebElement) – Valid WebElement from selenium.
key (Valid Selenium key or text.) –
- Returns
Element which key was sent to.
- Return type
WebElement
- send(key, value: str, by='name', enter=False, step=0.5, timeout=10)
send
Send a valid ‘key’ to element with selector ‘by’ and corresponding ‘value’.
- Parameters
key (Valid Selenium key or text.) –
value (str) – _description_
by (str, optional) – _description_, by default ‘name’
step (float, optional) – timeout step, by default 0.5
timeout (int, optional) – timeout until throw error, by default 10
- Returns
Element which key was sent to.
- Return type
WebElement
- arrow_down_element(element, n_times: int = 1, enter=False)
arrow_down
Press keyboard arrow down n_times at element.
- Parameters
element (WebElement) – Valid WebElement from selenium
n_times (int, optional) – Number of times pressing down key, by default 1
- arrow_down(value: str, by='name', step=0.5, timeout=10, n_times: int = 1, enter=False)
arrow_down
Select element by given selector ‘by’ and corresponding value, then send keyboard arrow down n_times.
- Parameters
value (str) – value of the selected attributes
by (str, optional) – attribute, by default “name”
step (float, optional) – timeout setp, by default 0.5
timeout (int, optional) – timeout, by default 10
n_times (int, optional) – times of pressing arrow up, by default 1
enter (bool, optional) – If True, ‘enter’ key is sent to element, by default False
- arrow_up_element(element, n_times: int = 1, enter=False)
arrow_down
Presse keyboard arrow up n_times
- Parameters
element (WebElement) – Valid WebElement from selenium
n_times (int, optional) – Number of times pressing down key, by default 1
- arrow_up(value: str, by='name', step=0.5, timeout=10, n_times: int = 1, enter=False)
arrow_up
Select element by given selector ‘by’ and corresponding value, then send keyboard arrow up n_times.
- Parameters
value (str) – value of the selected attributes
by (str, optional) – attribute, by default “name”
step (float, optional) – timeout setp, by default 0.5
timeout (int, optional) – timeout, by default 10
n_times (int, optional) – times of pressing arrow up, by default 1
enter (bool, optional) – If True, ‘enter’ key is sent to element, by default False
- get_items(parent, click=True)
- scroll_page()
- google(query, step=0.5, timeout=10)
- source()
- close()
Closes the current window.
- quit(clean=False)
Quits the driver and close every associated window.
- class thatscrapper.Key
Bases:
selenium.webdriver.common.keys.Keys- enter
- esc
- delete
- down
- up
- tab
- thatscrapper.ATTR_SELECTOR