Helium Scraper

Hi,

I am in the process of automating a project, and need my data to extract to a CSV on completion.

How can I achive this?

Thanks,

Richard

It's really easy to export your tables to a CSV. There are two ways to do this after you have run your action(s) and have your database tables built.

The first way is to open the Database panel and then double-click on the table you wish to export. From there, simply click on the Export to CSV icon located above the table you just opened.

The second way, which I feel is a better way, is to open the Database panel and click on Export icon located above the list of tables and then save the file as a MS Access file to your computer. This will save ALL of your tables in your current project. Next, launch Excel and open this file you just saved. NOTE: change the file type to "All Files" so that you will see the .mdb file. Upon opening, it will ask you to choose which table (if you have created more than one) you want to open in Excel. From there you can make whatever edits you wish and then save it as a CSV.

Thanks for the your reply.

I need my project to do this for me at the end of the scrape. I plan to schedule the project to run once a day and need it to output to a CSV file as the final step.

Thanks.

Hi,

This is currently not possible. Why is exporting the whole thing manually (maybe after a few days of extraction) not an option? Remember that you can use an external database so even if you run Helium Scraper from command line all the data will be stored in the same database.

Anyway, I'll take this as a feature request for an upcoming version.

Hi,

I would appreciate this being added to the next release. It would be a big benefit to us.

I need this feature since we are extracting stock information from our suppliers web site, and uploading this to our own web site. We do this once a day, so a fully automated solution would mean we do not have to perform the task manually.

Another benefit for us would be the time we could run the scrape, our supplier's stock levels are updated at midnight, so time is critical.

When do you think this will be available?

Thanks again,

Richard

Hi,

It there a date for the feature being added?

Thanks,

Richard

Hi Rich,

We don't have a date yet, but if I'd have to guess I'd say about a month.

Hi,

I received the updated Helium today, thank you. I've been testing the export function and noticed not all of my columns are being exported in the CSV.

I have a two column table to export, but only the first colum is populated with data, the column name is in the exported file.

This also happens if I manually export using the Export to CSV button. How can I force both columns to export?

Thanks,

Rich

Hi Rich,

Is the text in the second column by any chance multiline? It has happened to me that when I open a file in Excel some cells appear to be empty until I double click the cell to see the whole content.

If this is not the case, can you send me your project or some part of the project that I can use to reproduce this behavior?

Hi,

Yes, you're right. The export is inserting a blank line. If I manually copy and past these in to excel it's fine.

I've sent you a copy of the project. Thanks for the quick reply.

Rich