LinkedIn

Botmation Documentation

These BotAction's are for LinkedIn's web app.

Overview

The current set of BotActions facilitate the login flow and basic feed management such as liking posts. Here is a working example.

These are all included in the @botmation/linkedin package.

Install

In addition to installing @botmation/core, install @botmation/linkedin:

npm i -s @botmation/linkedin

Simple functions for navigating to various parts of the LinkedIn web app.

goHome

const goHome: BotAction =
  goTo('https://www.linkedin.com/', {waitUntil: 'domcontentloaded'})

goToFeed

const goToFeed: BotAction =
  goTo('https://www.linkedin.com/feed/', {waitUntil: 'domcontentloaded'})

goToMessaging

const goToMessaging: BotAction =
  goTo('https://www.linkedin.com/messaging/', {waitUntil: 'domcontentloaded'})

goToNotifications

const goToNotifications: BotAction =
  goTo('https://www.linkedin.com/notifications/', {waitUntil: 'domcontentloaded'})

Auth

These BotAction's focus on logging into the LinkedIn web app.

const login = (emailOrPhone: string, password: string): BotAction =>
  chain(
    errors('LinkedIn login()')(
      goTo('https://www.linkedin.com/login'),
      click('form input[id="username"]'),
      type(emailOrPhone),
      click('form input[id="password"]'),
      type(password),
      click('form button[type="submit"]'),
      waitForNavigation,
      log('LinkedIn Login Complete')
    )
  )

This BotAction is a composition that uses errors()() to wrap the main assembled BotAction's, in case something goes wrong here (ie a selector is updated so click() fails), dev's will have an easier time debugging when errors are thrown here.

The composition is declarative, therefore needs little explaining. login() is a higher-order function that takes a emailOrPhone and password strings to automatically perform the login flow in the web app.

isGuest

const isGuest: ConditionalBotAction = pipe()(
  // data feature for user notifications
  getLocalStorageItem('voyager-web:badges'),
  map(value => value === null) // Local Storage returns null if not found
)

After some investigating, it appears that Local Storage is used to maintain the state of the application's features, where each key is a major app feature. The particular key here references, what appears to be, a "Notifications" feature which is global to the app (in the Layout) and belongs only to Users.

isLoggedIn

const isLoggedIn: ConditionalBotAction = pipe()(
  // data feature for user notifications
  getLocalStorageItem('voyager-web:badges'),
  map(value => typeof value === 'string') // Local Storage returns string value if found
)

Feed

These BotAction's focus on operating in the main Feed of LinkedIn's web app.

Scrape Feed Post

This BotAction takes a specific post html attribute data-id value to scrape it with the provided HTML parser.

const scrapeFeedPost = (postDataId: string): BotAction<CheerioStatic> =>
  $('.application-outlet .feed-outlet [role="main"] [data-id="'+ postDataId + '"]')

Scrape Feed Posts

This BotAction is composed with the $$ Scrapers BotAction. It will grab each post from the DOM. LinkedIn's web app lazily loads offscreen posts, so some of these may be empty div containers.

const scrapeFeedPosts: BotAction<CheerioStatic[]> =
  $$(feedPostsSelector)

If Post Not Loaded Cause Loading Then Scrape

Linkedin's feed lazily loads the content of its offscreen posts. It uses div containers, each with their own data-id attribute, as placeholders for the content to be loaded in. This BotAction tests to see if a particular scraped post was fully loaded and if not, it causes the app to load it by scrolling to it. Then it scrapes the fully loaded container.

const ifPostNotLoadedCauseLoadingThenScrape = (post: CheerioStatic): BotAction<CheerioStatic> =>
  pipe(post)(
    errors('LinkedIn causeLazyLoadingThenScrapePost()')(
      pipeCase(postHasntFullyLoadedYet)(
        scrollTo('.application-outlet .feed-outlet [role="main"] [data-id="'+ post('[data-id]').attr('data-id') + '"]'),
        scrapeFeedPost(post('[data-id]').attr('data-id') + '')
      ),
      map(casesSignalToPipeValue)
    )
  )

Like User Post

It takes a scraped feed post, to grab identifying information for the "Like" button. It will throw and catch an error if the "Like" button was already liked, given how Puppeteer's page.click() handles elements not found. A "liked" button has a slightly different selector.

const likeUserPost = (post: CheerioStatic): BotAction =>
  errors('LinkedIn like() - Could not Like Post: Either already Liked or button not found')(
    click( 'div[data-id="' + post('div[data-id]').attr('data-id') + '"] button[aria-label="Like ' + post(feedPostAuthorSelector).text() + '’s post"]')
  )

Like User Posts From

It takes a spread array of strings of the exact names of people, to like their posts in the feed. The function itself goes beyond the required scope, but to serve as a decent starting point for more complex feed interactions.

const likeUserPostsFrom = (...peopleNames: string[]): BotAction =>
  pipe()(
    scrapeFeedPosts,
    forAll()(
      post => pipe(post)(
        ifPostNotLoadedCauseLoadingThenScrape(post),
        switchPipe()(
          pipeCase(postIsPromotion)(
            map((promotionPost: CheerioStatic) => promotionPost('[data-id]').attr('data-id')),
            log('Promoted Content')
          ),
          abort(),
          pipeCase(postIsJobPostings)(
            map((jobPostingsPost: CheerioStatic) => jobPostingsPost('[data-id]').attr('data-id')),
            log('Job Postings')
          ),
          abort(),
          pipeCase(postIsUserInteraction)(
            map((userInteractionPost: CheerioStatic) => userInteractionPost('[data-id]').attr('data-id')),
            log(`Followed User's Interaction (ie like, comment, etc)`)
          ),
          abort(),
          pipeCase(postIsUserPost)(
            pipeCase(postIsAuthoredByAPerson(...peopleNames))(
              likeUserPost(post),
              log('User Article "liked"')
            ),
            emptyPipe,
            log('User Article')
          ),
          abort(),
          // default case
          pipe()(
            map((unhandledPost: CheerioStatic) => unhandledPost('[data-id]').text()),
            log('Unhandled Post Case')
          )
        )
      )
    )
  )

Messaging

These BotAction's focus on operating in the main Messaging area of LinkedIn's web app.

toggleMessagingOverlay

const toggleMessagingOverlay: BotAction =
  click(messagingOverlayHeaderSelector)

By default, when someone logs into the LinkedIn web app, the "Messaging" center in the bottom-right is open, covering part of the web app. This BotAction will toggle that "Messaging" overlay open and close.

Selectors

Helpful HTML element selectors in the LinkedIn web app:

// Selectors for Messaging Overlay
const messagingOverlayHeaderSelector = 'header.msg-overlay-bubble-header'

// Selectors for the main News Feeds
const feedPostsSelector = '.application-outlet .feed-outlet [role="main"] div[data-id]'
const feedPostAuthorSelector = '.feed-shared-actor__title'

Helpers

These functions are not BotAction's, but useful in creating a web bot for Linkedin.

postIsUserPost()

This function is a ConditionalCallback that fits with pipeCase() and pipeCases(). It tests the provided CheerioStatic function, a "post" from the feed, on whether or not it fits the criteria for a User post, the common published posts.

const postIsUserPost: ConditionalCallback<CheerioStatic> = (post: CheerioStatic) => {
  const sharedActorFeedSupplementaryInfo = post('.feed-shared-actor__supplementary-actor-info').text().trim().toLowerCase()

  return sharedActorFeedSupplementaryInfo.includes('1st') || sharedActorFeedSupplementaryInfo.includes('following')
}

It checks for a particular part of the HTML where the connection meta information is displayed, ie to what degree of connection are you, or if not connected, are you following this User.

postIsAuthoredByAPerson()

This higher-order function returns a ConditionalCallback that fits with pipeCase() and pipeCases(). It tests the provided CheerioStatic function, a "post" from the feed, on whether or not it was authored by a particular person.

const postIsAuthoredByAPerson = (...peopleNames: string[]): ConditionalCallback<CheerioStatic> => (post: CheerioStatic) =>
  peopleNames.some(name => name.toLowerCase() === post(feedPostAuthorSelector).text().toLowerCase())

postIsPromotion()

const postIsPromotion: ConditionalCallback<CheerioStatic> = (post: CheerioStatic) =>
  post('.feed-shared-actor__sub-description').text().trim().toLowerCase().includes('promoted')

postIsJobPostings()

const postIsJobPostings: ConditionalCallback<CheerioStatic> = (post: CheerioStatic) => {
  const dataId = post('[data-id]').attr('data-id')

  if(!dataId) return false

  const dataIdParts = dataId.split(':')

  return dataIdParts.length >= 5 &&
         dataIdParts[2] === 'aggregate' &&
         dataIdParts.slice(3).some(part => part === 'jobPosting')
}

postIsUserInteraction()

The LinkedIn web app sometimes presents posts to highlight an User you're connected with, or following, has recently "reacted" to a post in the feed. This includes "liking", "loving", "celebrating", "commenting on" posts.

const postIsUserInteraction: ConditionalCallback<CheerioStatic> = (post: CheerioStatic) => {
  const feedPostSiblingText = post('h2.visually-hidden:contains("Feed post") + div span').text().trim().toLowerCase()

  return feedPostSiblingText.includes('likes this') ||
         feedPostSiblingText.includes('loves this') ||
         feedPostSiblingText.includes('celebrates this') ||
         feedPostSiblingText.includes('commented on this')
}

postHasntFullyLoadedYet()

The LinkedIn web app lazily loads offscreen content to expediate rendering performance of the feed. In doing such, it leaves container div's that represent posts, that may get scraped, but lack all the important information to make the informed decision. Therefore, this ConditionalCallback can be used to run code for loading a post fully, in case it has not.

const postHasntFullyLoadedYet: ConditionalCallback<CheerioStatic> = (post: CheerioStatic) => {
  return post('[data-id]').text().trim() === ''
}

Edit this page on GitHub