How do I use a table of URLs as my starting point?

Questions and answers about anything related to Helium Scraper
Post Reply
rmbraaten
Posts: 10
Joined: Wed Jun 01, 2011 12:00 am

How do I use a table of URLs as my starting point?

Post by rmbraaten » Wed Mar 28, 2012 4:20 pm

I'm looking for a way to scrape only the URLs listed in a linked table as a way to check previously scraped pages for updated content. So, for example, assume I've scraped some place listing pages from yp. One month later, I want to go back and check a filtered set of these pages for updated content or to see if anything has changed, but don't want to have to check all of the pages via a "deep yellow page" extraction method. I already have the URL for each page in my table, so I'd like my table to be the starting point of a deep search rather than category page on yp. Is that possible?

I'm not sure if that's what your "Write from Database" sample project is designed to do. If so, can you provide some more information, perhaps with some sample data? I can't figure out where to start.

Thanks!

mb

Post Reply