Navigation

These BotAction's provide simple ways to change the Page's URL.

Go To

Navigates the page to the url provided, unless the active url is the url provided, then it emits a warning in the console.

Copy
const goTo = (url: string, goToOptions?: Partial<WaitForOptions>): BotAction =>
  async(page) => {
    if (page.url() === url) {
      return
    }

    await page.goto(url, enrichGoToPageOptions(goToOptions))
  }

goToOptions are provided aa they are enriched with safe defaults. See Puppeteer's Documentation on page.goto() for details.

For an usage example, see Functional Bot.

Go Back

Similar to hitting the browser's "Back" button, to load the previous URL.

Copy
const goBack = (options?: NavigationOptions): BotAction =>
  async(page) => {
    await page.goBack(options)
  }

Example:

Copy
await chain(
  goTo('https://duckduckgo.com/'),
  goTo('https://google.com'),
  goBack() // goes back to duckduckgo.com
)

Go Forward

Similar to hitting the brower's "Forward" button, to go forward after going back.

Copy
const goForward = (options?: NavigationOptions): BotAction =>
  async(page) => {
    await page.goForward(options)
  }

Example:

Copy
await chain(
  goTo('https://duckduckgo.com/'),
  goTo('https://google.com'),
  goBack(), // goes back to duckduckgo.com
  goForward() // goes back to google.com
)

Reload

Similar to hitting the brower's "Refresh" button, to reload the page with the same URL.

Copy
const reload = (options?: NavigationOptions): BotAction =>
  async(page) => {
    await page.reload(options)
  }

Example:

Copy
await chain(
  goTo('https://duckduckgo.com/'),
  reload() // reloads duckduckgo.com
)

Wait for Navigation

Waits for the page to navigate to a new URL or reloads. It is useful for when you run code which indirectly changes the page, like submitting a form by clicking "Submit".

Copy
const waitForNavigation: BotAction = async(page) => {
  await page.waitForNavigation()
}

For an usage example, see the Instagram login() BotAction.

Scroll To

This BotAction evaluates a scrollToElement() helper function in the context of the Puppeteer page based on the HTML selector provided. It calls the HTML node element's scrollIntoView() method. Once that method is called, it artificially waits for it to complete scrolling with a timespan set in milliseconds, default of 2500.

Copy
const scrollTo = (htmlSelector: string, waitTimeForScroll: number = 2500): BotAction =>
  chain(
    evaluate(scrollToElement, htmlSelector),
    wait(waitTimeForScroll)
  )

For an example, see LinkedIn's Feed ifPostNotLoadedTriggerLoadingThenScrape() BotAction, which scrolls to an off screen feed post, to cause lazy loading of its content.

Helpers

enrichGoToPageOptions()

Puppeteer page's goto() method has many options to customize the navigation. See their documentation for details on available options.

Copy
const enrichGoToPageOptions = (overloadDefaultOptions: Partial<DirectNavigationOptions> = {}): DirectNavigationOptions => ({
  waitUntil: 'networkidle0',
  ...overloadDefaultOptions
})

scrollToElement()

This helper function is ran inside the Page's context to scroll to a particular HTML node element.

Copy
const scrollToElement = (htmlSelector: string) =>
  document.querySelector(htmlSelector)?.scrollIntoView({behavior: 'smooth'})