Helium Scraper

Hello,

I love your product, and am making a very good use of it. Thanks for making it. Well worth the $$$.

I have couple of problems, though:

1) When using the multiple processes feature (like the attached screenshot), it requires a human intervention every time a child process returns. My extraction takes about 30 minutes to 1 hour, and this means I have to stand by and be ready to hit enter 100 times... Is there any way to avoid this and automate the entire process?

2) One of the child processes sometimes hangs for no obvious reason with a black child window. It seems to prevent this symptom when I make all of the child windows very small (so that they don't blink as much) every time they appear. This is a tedious process.

I have to do (1) and (2) for every one of my routine website scan/extractions. It would help very very much if you could give me a clue on how to mitigate the above issues.

Thanks!

Soo

Opps, forgot to attach the screenshot.
Here we go:

Hi,

Why exactly do you need to press enter for every child process? Do they show any message box you need to accept?

Also, I don't think that setup is going to work since you're clearing the SubMenuURL table while another process may be using it as a source. What I'd to is first extract all the available URLs into the SubMenuURL table and, after checking the table and making sure all the URLs are there, set up another actions tree to spawn all the child processes using this table (without ever having to clear it).

Hi, thanks very much for the response.

Yes, every time a child process returns, there's a pop-up box for which I have to press an enter. I normally have to close about 100 of these pop-up boxes for every extraction.. I"m using the latest version of Helium Scraper and IE 11.

As for the SubMenuURL, I doubt if it gets referred to by the child processes while they're running. Extraction itself seems to be working okay. I'd like to try your suggestion however - Once the table with all the URLs is ready (Let's say about 2000 URLs), how many processes would you recommend I spawn? Assuming I'd have to allocate the data in the SubMenuURL table to each child process, is there an easy way of doing this?

Cheers,
Soo

Hi,

What does this popup say?

About 4-5 processes should be OK. You'd really just do the same you're doing in Actions tree 1 with the Execute tree: Clear Tables and the Start Processes at: SubMenuURL, URL actions removed and then in another actions tree add a single Start Processes at: SubMenuURL, URL action, which you'd run after you've extracted all the URLs.