LiveHTML function

Interact with a live web page

You construct an LiveHTML object with read_html_live() and then interact, like you're a human, using the methods described below. When debugging a scraping script it is particularly useful to use $view(), which will open a live preview of the site, and you can actually see each of the operations performed on the real site.

rvest provides relatively simple methods for scrolling, typing, and clicking. For richer interaction, you probably want to use a package that exposes a more powerful user interface, like selendir.

Examples

## Not run:

# To retrieve data for this paginated site, we need to repeatedly push
# the "Load More" button
sess <- read_html_live("https://www.bodybuilding.com/exercises/finder")
sess$view()

sess %>% html_elements(".ExResult-row") %>% length()
sess$click(".ExLoadMore-btn")
sess %>% html_elements(".ExResult-row") %>% length()
sess$click(".ExLoadMore-btn")
sess %>% html_elements(".ExResult-row") %>% length()
## End(Not run)

Public fields

• session: Underlying chromote session object. For expert use only.

Methods

Public methods

• list("LiveHTML$new()")

• list("LiveHTML$print()")

• list("LiveHTML$view()")

• list("LiveHTML$html_elements()")

• list("LiveHTML$click()")

• list("LiveHTML$get_scroll_position()")

• list("LiveHTML$scroll_into_view()")

• list("LiveHTML$scroll_to()")

• list("LiveHTML$scroll_by()")

• list("LiveHTML$type()")

• list("LiveHTML$press()")

• list("LiveHTML$clone()")

Method new()

initialize the object

Usage

LiveHTML$new(url)

Arguments

• url: URL to page.

Method print()

Called when print()ed

Usage

LiveHTML$print(...)

Arguments

• ...: Ignored

Method view()

Display a live view of the site

Usage

LiveHTML$view()

Method html_elements()

Extract HTML elements from the current page.

Usage

LiveHTML$html_elements(css, xpath)

Arguments

• css, xpath: CSS selector or xpath expression.

Method click()

Simulate a click on an HTML element.

Usage

LiveHTML$click(css, n_clicks = 1)

Arguments

• css: CSS selector or xpath expression.

• n_clicks: Number of clicks

Method get_scroll_position()

Get the current scroll position.

Usage

LiveHTML$get_scroll_position()

Method scroll_into_view()

Scroll selected element into view.

Usage

LiveHTML$scroll_into_view(css)

Arguments

• css: CSS selector or xpath expression.

Method scroll_to()

Scroll to specified location

Usage

LiveHTML$scroll_to(top = 0, left = 0)

Arguments

• top, left: Number of pixels from top/left respectively.

Method scroll_by()

Scroll by the specified amount

Usage

LiveHTML$scroll_by(top = 0, left = 0)

Arguments

• top, left: Number of pixels to scroll up/down and left/right respectively.

Method type()

Type text in the selected element

Usage

LiveHTML$type(css, text)

Arguments

• css: CSS selector or xpath expression.

• text: A single string containing the text to type.

Method press()

Simulate pressing a single key (including special keys).

Usage

LiveHTML$press(css, key_code, modifiers = character())

Arguments

• css: CSS selector or xpath expression. Set to NULL

• key_code: Name of key. You can see a complete list of known keys at https://pptr.dev/api/puppeteer.keyinput/.

• modifiers: A character vector of modifiers. Must be one or more of "Shift, "Control", "Alt", or "Meta".

Method clone()

The objects of this class are cloneable with this method.

Usage

LiveHTML$clone(deep = FALSE)

Arguments

• deep: Whether to make a deep clone.