Turning pages without "Next" button

Here we will be posting premade Helium Scraper projects and helpful stuff.
Post Reply
webmaster
Site Admin
Posts: 521
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Turning pages without "Next" button

Post by webmaster » Wed Jun 29, 2011 3:22 am

There are a few cases on which a set of results pages won't provide a "Next" button, but instead, just a list of page numbers.

The attached project is a simple solution to this problem. To use it, just import it into the project you are working on, or use it as a starting point, and then create a kind that selects the page number that comes after the current page. Then navigate to another page and add the page number that comes after that page. So if you are, for instance, at page 1, select the link that takes you to page 2, and create a kind called "Next" with it. Then navigate to page 2, select the link that takes you to page 3 and add it to your "Next" kind.

After this, when you navigate to any other page, this kind will always select the next page number just as if it were a "Next" button and you can use it as such for every automation purpose. It will also work with the Go through all pages premade project.

Without this project, if you try to follow the steps I describe above, your kind will most likely end up selecting all page numbers, instead of just the next one. This is because this project contains a set of JavaScript gatherers that gather properties from elements that are located before the element you select when creating a kind, which on this case means that it will gather properties from the current page number when you select the next page number. Since the current page number is different that all other page numbers (i.e. is not a clickable link), Helium Scraper will be able to distinguish between the next page number and any other page number.

I've tested it on several search engines and sites and it worked OK. But if you notice that your kind starts selecting every page number instead of just the next one, let me know and I'll add whatever JavaScript gatherer is necessary to have it working properly.
Attachments
PageNumberProperties.hsp
(291.48 KiB) Downloaded 2184 times
Juan Soldi
The Helium Scraper Team

luisantafe
Posts: 10
Joined: Tue May 03, 2011 9:15 pm

Re: Turning pages without "Next" button

Post by luisantafe » Sat Jul 02, 2011 4:08 am

Hello, i would like to know how can I import this premade to my project. thanks

webmaster
Site Admin
Posts: 521
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Turning pages without "Next" button

Post by webmaster » Sat Jul 02, 2011 9:21 am

Just use the File -> Import command from the main menu.
Juan Soldi
The Helium Scraper Team

design
Posts: 8
Joined: Thu Jul 19, 2012 1:27 pm

Re: Turning pages without "Next" button

Post by design » Tue Jul 24, 2012 6:09 pm

hi

I have given my more than 60 mins for this online premedes not solutions arises to me

could you give me step by step instruction in this forum as I have to navigate the page with out next

i had imported the go through the page number or also tried turn pages with out next button I am but going crazy

please do help me

design
Posts: 8
Joined: Thu Jul 19, 2012 1:27 pm

Re: Turning pages without "Next" button

Post by design » Tue Jul 24, 2012 6:11 pm

this action tree is not called by any actiion tree

webmaster
Site Admin
Posts: 521
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Turning pages without "Next" button

Post by webmaster » Tue Jul 24, 2012 8:57 pm

Hi,

You'll see the "This action tree is not called by any action tree" message when you attempt to run a premade actions tree that is supposed to be run from an Execute Actions Tree action in another actions tree. When you add an Execute Actions Tree action you get to select which actions tree to run. Here you would select the premade actions tree you just imported.

Regarding your other post above, I'm not sure I understand what the problem is. Can you attach the project you're working on? Also, which premade are you using, is it the one from File -> Online Premade or is it the one attached on this post?
Juan Soldi
The Helium Scraper Team

design
Posts: 8
Joined: Thu Jul 19, 2012 1:27 pm

Re: Turning pages without "Next" button

Post by design » Wed Jul 25, 2012 3:48 am

Hi

I am trying to extract business name with phone linked to business name through all the pages from page 1


you can see the in the link http://www.hotfrog.co.uk/Products/wedding-cars there is business name and phone no are linked inside with out the next only page number give

please help me

webmaster
Site Admin
Posts: 521
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Turning pages without "Next" button

Post by webmaster » Thu Jul 26, 2012 8:34 pm

Hi,

What I would do here is use the URL Variations premade since result pages include the page number in the URL. Trying to do it by clicking on the next page up to page 10 and then clicking on Next 10 Pages each time would be a harder way of doing it.

Just import the premade by going File -> Online Premades and select URL Variations (make sure you read the description) and click Import. Then, add an Execute Actions Tree action to a new actions tree and select Navigate URLs. For the URL, enter this (note the dollar symbol at the end; this will be replaced by the page number):

http://www.hotfrog.co.uk/Products/wedding-cars/$

On First Page enter 1, and on Last Page enter 106 (which is the past page for wedding cars). Then press OK and add whatever actions you need to perform inside each results page as children of the Execute Actions Tree action. I assume you want to navigate inside each business' details page, so you'd add as children action a Navigate Each that navigates through each link that takes you to a details page, and then as a children of the Navigate Each action an Extract action that extracts from the details page whatever data you need.

Let me know if you need any more help.
Juan Soldi
The Helium Scraper Team

David W

Re: Turning pages without "Next" button

Post by David W » Sun Sep 16, 2012 4:58 pm

Hey Juan,

I'm also having trouble importing this to my current project. I have downloaded the project and gone to FIle>Import, and click import. It then prompts that this is from an outside source, and I allow it. However, I am then unable to find it in my current project. Any suggestions?

Thanks,

David

webmaster
Site Admin
Posts: 521
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Turning pages without "Next" button

Post by webmaster » Wed Sep 19, 2012 5:20 pm

This project only adds a few javascript gatherers (you can see their code at Project –> JavaScript Gatherers). Perhaps you were expecting to see new actions or kinds? As the project’s description states, you’d import the project and then create a kind that selects the next page number. Please refer to the project’s description for more details.
Juan Soldi
The Helium Scraper Team

Post Reply