Page 1 of 1

Helium Scraper new features

Posted: Sat Jul 16, 2011 11:30 pm
by webmaster
  • Online Premades: Now you can download and import premade projects directly from Helium Scraper. There are two ways of doing this: from the File -> Online Premades menu item, and from the New action -> Execute Actions Tree -> More... item in the actions panel. This last way will also add a Execute Actions Tree action that executes the actions tree contained in the project and show the setup dialog, which emulates the way built-in actions work.
  • Text Gatherers: A new tool accessible from the Project -> Text Gatherers menu item has been added. This tool will let you create a JavaScript gatherer without writing any code whatsoever, that will transform and split any text before being extracted.
  • Recent Files: A list of recent files has been added to the File menu item.
  • Go To URL: The Go To URL action has been added.
  • Smaller Files: The embedded database is now automatically compacted before a project is saved, which makes Helium Scraper files smaller.
  • User Interface Enhancements: Now the left panel can be minimized, icons have been added to several menu items, and forms don't show up in the taskbar anymore.
  • No "Save changes...?" when unnecessary: You won't be prompted anymore to save changes when closing the application, creating project or opening a project if no changes have been made to the current project.
  • MailTo: The JS_MailTo property gatherer has been added, which gets the email from links that point to email addresses.
  • float data type added: The data type float has been added to the list of data types when creating a data table from the Create table button in the database panel or when creating an Extract action.
  • Improved Actions: The Navigate Each and Navigate URL actions are now slightly faster because of the mechanism Helium Scraper uses to navigate back to the original page. This affects specially the Navigate Each action when used with the Simulate Click option on.
  • Limited log lines: The log now cannot have more than 20000 lines. If this limit is reached, the oldest line is removed every time a new line is added.

Re: Helium Scraper new features

Posted: Mon Jul 18, 2011 7:31 pm
by Tommy
Great update Juan,

The online premades is an excellent addition and I'm looking forward to see how it'll grow...

Is there any more detailed information on how to use the new Text Gatherer's feature? I've had a play but I'm still unsure. Would be great if I could learn how to use the slice and replace features fully. Do you plan on adding further transformations in the future?


Re: Helium Scraper new features

Posted: Mon Jul 18, 2011 7:38 pm
by Tommy
Ok, I've played some more :)

I take it back. The Text Gatherer is very intuitive and a great feature for editing scraped text on the fly :)

Re: Helium Scraper new features

Posted: Mon Jul 18, 2011 7:44 pm
by webmaster
The Text Gatherers feature tool is really very simple. The Replace transformation replaces every occurrence of one or more characters for another set of characters.

The Delimited Slice transformation breaks the text into slices, and then you get to select the slice to be used by changing the Slice position. For instance, if you have the text "This is some random text" and use a space as a delimiter, you would get 5 slices, i.e, each word. The slice 0 is "This", the slice 1 is "is" and so on. If you use the "From last slice" option, the slice 0 would be "text", the slice -1 would be "random" and so on.

The Fixed Slice transformation selects a fixed section of text by giving the position of the first character and the last character, counting from the start or end, depending on whether the From End option is off or on.

Then, after pressing Done, you will be prompted to type a name for the JavaScript gatherer that will be produced, which will apply your transformations to any text being extracted if used as a property under the Property column when setting up an Extract action.

Hope this helped.

Re: Helium Scraper new features

Posted: Mon Jul 18, 2011 7:48 pm
by Tommy

Looks like I replied just before you ;)

It's a great feature.

Re: Helium Scraper new features

Posted: Mon Jul 18, 2011 7:50 pm
by webmaster
It is isn't it? :D

Re: Helium Scraper new features

Posted: Mon Jul 18, 2011 8:28 pm
by webmaster
Anyway, here is a quick video that shows how it works: