Helium Scraper Database limit

Questions and answers about anything related to Helium Scraper
Post Reply
edvukass
Posts: 13
Joined: Wed Jan 30, 2013 3:58 pm

Helium Scraper Database limit

Post by edvukass » Mon Jul 15, 2013 9:42 am

Hi, I have came across this problem. Scraper is stopping and keep throwing "cannot open database" message. It seems that I have reached the database limit. "mdb" file size is 2gb. It doesn't let me finish scraping or at least extract to a file the info I have already scraped. Is there any solution to that?I have attached 2 messages I'm getting bellow.
Attachments
out_of_memory.png
I'm getting this message When trying to run query to open joined tables and extract them to a CSV file
out_of_memory.png (3.47 KiB) Viewed 2747 times
cannot_open_database.png
I'm getting this message when trying to keep scraping.
cannot_open_database.png (5.75 KiB) Viewed 2747 times

crookedleaf
Posts: 38
Joined: Tue Dec 11, 2012 6:44 pm

Re: Helium Scraper Database limit

Post by crookedleaf » Thu Sep 05, 2013 1:49 pm

edvukass wrote:Hi, I have came across this problem. Scraper is stopping and keep throwing "cannot open database" message. It seems that I have reached the database limit. "mdb" file size is 2gb. It doesn't let me finish scraping or at least extract to a file the info I have already scraped. Is there any solution to that?I have attached 2 messages I'm getting bellow.
the problem actually is more than likely Access, not Helium. Access itself has a 2gb limit on database files. one thing you could try is compacting/repairing the database. one thing i have noticed is sometimes the database bloats to unnecessary size for no reason. even after clearing it completely of data, i have had the database file be well over 1gb. make a copy of the database (always best to back it up just to be safe), then open the original database, go to database tools, and click "compact/repair database".

Post Reply