How to get data from page in extraction-db of the next page?

Questions and answers about anything related to Helium Scraper
Post Reply
flotschie
Posts: 16
Joined: Wed May 15, 2013 6:24 pm

How to get data from page in extraction-db of the next page?

Post by flotschie » Wed May 15, 2013 10:05 pm

Hi,

I have two tables. The first table contains the column "URLs" and 2 additional columns (let's call them A and B). The data for this table is extracted from the starting page.

I use the Navigate URLs command to loop through the URLs from the first table.

The second table contains data I extract from the pages called by the Navigate URLs command. How can I get the respective data in the columns A and B from the first table into the second one?

flotschie
Posts: 16
Joined: Wed May 15, 2013 6:24 pm

Re: How to get data from page in extraction-db of the next p

Post by flotschie » Thu May 16, 2013 10:17 pm

I think i found the answer in your blog. I could use a JavaScript action after the extraction of each page into table 2. The JS should use something like

Code: Select all

Global.DataBase.Query("INSERT INTO [Table2] VALUES (select a,b from table1 where table1.url=currenturl) where url=currenturl");
The sql query is maybe not 100% working but you will get the intension :)
I will try it out soon

flotschie
Posts: 16
Joined: Wed May 15, 2013 6:24 pm

Re: How to get data from page in extraction-db of the next p

Post by flotschie » Mon May 20, 2013 9:53 am

It was somehow working (using UPDATE instead of INSERT of course) but it took literally FOREVER to update like 140 rows or something. Any suggestions are welcome, otherwise i have to export two tables instead of one and do the updating somewhere else which is not very convenient.

Post Reply