Massive javascript not executed

Let us know if anything goes wrong with our baby :)
Post Reply
Kyen
Posts: 5
Joined: Fri Jun 01, 2012 4:21 am

Massive javascript not executed

Post by Kyen » Fri Jun 08, 2012 7:39 am

I have some Javascript that I need to run that's about 59,000 lines. I know that seems a bit excessive, but it seems to be about the only thing I can do. It's a massive Switch statement, actually. If I have this in my Javascript at all, the Javascript action is ignored enitrely. Commands that come before the switch don't even execute. I put the switch into a function, and the other commands would execute, but calling the function immediately crashed Helium.

As a side note, if I try copying and pasting the same switch statement as an if/else, Helium just crashes.

Thanks!

webmaster
Site Admin
Posts: 491
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Massive javascript not executed

Post by webmaster » Tue Jun 12, 2012 2:44 am

Hi,

I tried running a pretty long javascript and all I got was a message telling me it couldn't inject it in the current page. When it crashes, do you get any message or does it just stops responding? Any chance you can send this script so we can test it (in a zipped file if possible :D )?

Anyway, sounds to me like this information should be in a database. Is this script auto-generated? Since this is a switch statement, you should be able to create a table with two columns: input - output. Then run a query and get your desired output from whatever your input is. This would emulate a switch statement with a way shorter script.
Juan Soldi
The Helium Scraper Team

Kyen
Posts: 5
Joined: Fri Jun 01, 2012 4:21 am

Re: Massive javascript not executed

Post by Kyen » Fri Jun 22, 2012 11:59 am

Sorry for the significantly delayed response. If I didn't keep the switch in a function, Helium would automatically use the default case if I had one, and no case if I didn't, even when the parameters met the case. I've been just running it in batches, taking chunks of the relevant switch case when necessary. Unfortunately, I can only scrape 80 pages at a time before I run out of memory and I have to close the program and restart.

*Edit - When it crashed, it just stopped responding.

Again, I appreciate your help!
Attachments
naver.rar
Compressed Script
(134 KiB) Downloaded 300 times

webmaster
Site Admin
Posts: 491
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Massive javascript not executed

Post by webmaster » Sat Jun 23, 2012 9:00 pm

Why does this code run on document-ready? Are you just placing the whole script in a Execute JavaScript action?

Regarding the out of memory problem, since you're navigating through a bunch or URL's, this project might be just what you need.
Juan Soldi
The Helium Scraper Team

Post Reply