How to navigate a tree?

Questions and answers about anything related to Helium Scraper
Post Reply
doug
Posts: 22
Joined: Tue Aug 16, 2011 12:44 am

How to navigate a tree?

Post by doug » Sun Sep 04, 2011 10:21 pm

I am navigating a sequence of pages (level 1). Each page has a table of links to other pages at level 2. I've been extracting the level 2 links to a table for later stepping through the URL's.
Is there an easy/clever way to do depth first?

webmaster
Site Admin
Posts: 495
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: How to navigate a tree?

Post by webmaster » Mon Sep 05, 2011 7:22 pm

Hi,

You can always use a Navigate Each inside another Navigate Each. But if your project is very big and it will take a while to complete, I'd do it the way you are doing it and extract these URLs on bunches so that you don't need to restart the whole thing if something goes wrong.

Another option would be to extract level1 links and then use a Navigate URLs action to navigate through them and a Navigate Each action inside that navigates through level2 links.
Juan Soldi
The Helium Scraper Team

Post Reply