Running out of resources

Questions and answers about anything related to Helium Scraper
Post Reply
benmyers
Posts: 1
Joined: Mon Apr 29, 2013 10:23 am

Running out of resources

Post by benmyers » Mon Apr 29, 2013 10:42 am

Hi,

My Project keeps running out of resources.

I have attached it below.

Can anyone help.

Ben
Attachments
Hot Toner5.zip
(33.51 KiB) Downloaded 281 times

preichow
Posts: 8
Joined: Tue May 28, 2013 4:07 am

Re: Running out of resources

Post by preichow » Mon Apr 28, 2014 6:28 pm

Helium appears to have some kind of massive memory leak. I have 8 GB on my machine, and I run out of resources after scraping about 1,000 records. It's a massive inconvenience.
To get around this, I reboot frequently. I also limit the size of the projects, and I frequently copy and I export output tables. Good luck...

yeti
Posts: 9
Joined: Sat Feb 02, 2013 1:01 am

Re: Running out of resources

Post by yeti » Sun Aug 31, 2014 11:23 am

This happens to me regularly also, and I have a high spec PC with 16GB of RAM.

Even basic tasks cause it to run out of resources, so it must be leaking memory somewhere. Works fine once you restart it, but it is annoying.

mehdi
Posts: 8
Joined: Tue Aug 26, 2014 11:56 am

Re: Running out of resources

Post by mehdi » Tue Sep 09, 2014 7:29 am

Hi, you should read the FAQ and other posts on the topic, this is very common with HS and seems that it is due to IE instead.
The workaround is to use multi-process scraper, please take a look on this tutorial

http://www.youtube.com/watch?v=_yP65Hzavao

In order to have the multiprocess efficient, make sure that the (top) parent process is the one that does the less possible, and that the second or third level detail is the child process which runs in parallel. Every time a child process ends, it will close down automatically hence it will free any used memory.

Post Reply