Helium Scraper

So I am running through multiple pages and extracting the information of each page to a database, but instead of the rows picking up where they left off they are overwriting the existing lines starting from row 1 with each new page. I was wondering if there was an obvious way around this........

Either to have the new data extracted with each new pass to the next empty row in the existing database

Or possibly export the database to a separate file using JS at the end of each pass(with incrementally increasing file name or something so it doesnt get overwritten with each pass)

Or even have a new table created with each pass(dummy1, dummy2, etc)

thanks

Wait... how are you overwriting the table rows??? I'm adding onto the ones I have at the moment using "Extract to <table>" action. How do you overwrite them?

I dont think its overwriting the table line by line, I think its just replace my existing table with a new one, but im using the same 'extract to [table] function to achieve this

Interesting... I'd like the overwrite function when I restart a scrape - at the moment it simply appends to the existing data.

Have you set any of your columns to Unique when creating your extract action? If so, rows with the same values in Unique columns will be updated if the Unique columns are the same as the values that are about to be extracted.

Also, note that the table will be recreated if you edit your Extract action.