Sequencing Click to Load additional Elements, then, Extraction Logic

Questions & Answers about Helium Scraper 3
Post Reply
BizInC
Posts: 1
Joined: Tue May 04, 2021 5:12 am

Sequencing Click to Load additional Elements, then, Extraction Logic

Post by BizInC » Tue May 04, 2021 5:19 am

All, Thanks in advance.

Q: How to make 50 Clicks on Load More button for loading additional items on single page?

Then, after some delay, execute the Browser.Navigate Function.

webmaster
Site Admin
Posts: 512
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Sequencing Click to Load additional Elements, then, Extraction Logic

Post by webmaster » Tue May 11, 2021 8:38 pm

Note that clicking a button 50 times and then extracting the content involves exact same steps as, in a site that has pages, clicking the next button to turn the pages 50 times, and then extracting from the 51st page.

So you can use Browser.TurnPages to repeatedly click the button (or even better, Browser.TurnPagesAndWait, since the content may take a while to load). Then limit it to 50 pages by wrapping it in a Sequence.Take. And finally, to extract from the last "page" only, wrap that in a Sequence.Last. The whole thing would look like this:

Code: Select all

Sequence.Last
   ·  Sequence.Take
         ·  50
         ·  Browser.TurnPagesAndWait
               ·  Select.LoadMoreButton
               ·  1000
Note that, when typing that out you'd actually do it the other way around: first Sequence.Last, then Sequence.Take, and then Browser.TurnPagesAndWait. Or if you already have a selector called LoadMoreButton you can just copy/paste the whole thing.
Juan Soldi
The Helium Scraper Team

Post Reply