Manta

Questions and answers about anything related to Helium Scraper
Post Reply
jley
Posts: 9
Joined: Thu Sep 29, 2011 10:14 pm

Manta

Post by jley » Thu Sep 29, 2011 11:10 pm

I'm attempting to scrape Manta, but can't seem to get past the way the data is given from Manta.

I specify the data I want at Manta and it returns a page of links. Each link contains the data I want and I can scrape it - but none of these pages has a next link. This is not a problem since I'm working off the original page of links. However, when I get to the last link, there is nowhere to go. If I select Kind:Next and specify at least one, Helium stops because it's not there. If I go back to the starting page and select kind Next, I simply collect the 2nd page of data over and over. (At least I think that's whats going on...) I tried to use the PageNumberProperties, but haven't figured out how to incorporate that - I got it imported, but have not found documentation on how to use it yet. If I manually select page 2, page 3 and so on and then run the actions (w/o the got starting page) it works - but that's a bit tedious. Help would be appreciated!

Thanks.
Attachments
DubScrape.hsp
(462.74 KiB) Downloaded 205 times

webmaster
Site Admin
Posts: 501
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Manta

Post by webmaster » Sat Oct 01, 2011 8:29 am

Hi,

See the attached project. First of all, those two Execute JS actions shouldn't need to be there on normal circumstances, but because of a bug in the Navigate Each action (that will be fixed on the next update), it fails to go back to the original page as it should when scraping heavily loaded sites such as Manta.

Notice the Navigate: Next action (instead of your Select Kind action). Also, I'm using the Force Select action because you are using a navigation timeout of 5 seconds, which makes sense for a site like Manta which can take minutes to complete loading a page. This will prevent trying to navigate through the Next button if it's not there yet.
Attachments
DubScrape2.hsp
(366.3 KiB) Downloaded 237 times
Juan Soldi
The Helium Scraper Team

jley
Posts: 9
Joined: Thu Sep 29, 2011 10:14 pm

Re: Manta

Post by jley » Sun Oct 02, 2011 2:32 am

Thanks for your help Juan! The new setup is working very nicely.

Post Reply