More than one site to same .csv?
More than one site to same .csv?
Can you scrape different sites pulling different content into the same .csv?
Re: More than one site to same .csv?
Are you extracting all these sites from the same project? Also, do the result tables have the same structure?
Juan Soldi
The Helium Scraper Team
The Helium Scraper Team
Re: More than one site to same .csv?
I haven't worked out how to scrape multiple sites in the same project to be honest. If this can be done, then yes, it will be in the same projectwebmaster wrote:Are you extracting all these sites from the same project? Also, do the result tables have the same structure?
I'm guessing it might be easier to scrape each site individually and do some post-processing, but with the (hopefully) up and coming update that enables us to create stand-alone scrapers, I'm wondering if this is something that could be created.
Re: More than one site to same .csv?
Well, what I would do is use any spreadsheet program (here is a free one) and then paste all my data from one table and then paste data at the end from another table and so on.
But you can also extract from multiple "Extract" actions to a single table, as long as these have the same structure and are in the same project, by using the same table name (when you get the "Replace existing table?" prompt just say yes). The easiest way to do this is by duplicating an existing "Extract" action by right clicking it and selecting "Duplicate Node". Then you can drag it and drop it to another actions tree, and change the kinds being used which most likely will be necessary.
To extract from more than one site from one project you can use one actions tree for each side. Also, you could add at the beginning of each tree a "Execute JavaScript" action with this code:
which will take you to "www.somesite.com". Furthermore, you can create another actions tree that executes all your other actions trees one by one by using the "Execute Actions Tree" action.
But if all these sites happen to have exactly the same structure (such as when they use the same template) you could just use a single actions tree.
Hope this made sense.
But you can also extract from multiple "Extract" actions to a single table, as long as these have the same structure and are in the same project, by using the same table name (when you get the "Replace existing table?" prompt just say yes). The easiest way to do this is by duplicating an existing "Extract" action by right clicking it and selecting "Duplicate Node". Then you can drag it and drop it to another actions tree, and change the kinds being used which most likely will be necessary.
To extract from more than one site from one project you can use one actions tree for each side. Also, you could add at the beginning of each tree a "Execute JavaScript" action with this code:
Code: Select all
window.location.href = "http://www.somesite.com";
But if all these sites happen to have exactly the same structure (such as when they use the same template) you could just use a single actions tree.
Hope this made sense.
Juan Soldi
The Helium Scraper Team
The Helium Scraper Team
Re: More than one site to same .csv?
Thanks for the detailed reply.
It's great that scraping multiple sites to a single project is possible. Roll on the stand-alone scraper feature
It's great that scraping multiple sites to a single project is possible. Roll on the stand-alone scraper feature