Search found 507 matches

by webmaster
Thu Nov 19, 2020 8:11 pm
Forum: Premades
Topic: Scroll Slowly to Bottom
Replies: 0
Views: 25

Scroll Slowly to Bottom

There are pages where the content won't load until we scroll down to where the content is located, and scrolling all the way to the bottom of the page at once sometimes fails to load the content. This premade scrolls slowly to the bottom of the page so all content is loaded. It takes a speed argumen...
by webmaster
Wed Oct 21, 2020 7:03 pm
Forum: Q/A
Topic: Button link extraction
Replies: 1
Views: 611

Re: Button link extraction

Have you tried using Gather.Link? If that doesn't work then it's probably a JavaScript button and that'll depend on the particular implementation so I'd need to see the actual page.
by webmaster
Wed Oct 21, 2020 7:01 pm
Forum: Q/A
Topic: JSON-LD LD+JSON Java Linking Data
Replies: 1
Views: 596

Re: JSON-LD LD+JSON Java Linking Data

Hi Ed,

No sure what you mean by Java Linking Data. Are you trying to grab info from a web request? Can you send a sample URL?
by webmaster
Fri Oct 09, 2020 5:59 pm
Forum: Q/A
Topic: How to operate with variables and conditions?
Replies: 2
Views: 282

Re: How to operate with variables and conditions?

Core.ToBoolean just parses a "true" or "false" value into a boolean, so it won't work with empty strings. But you can write your own function using String.IsMatch and the negation operator. The following will return false when the string is empty and true when not: function (text) not · String.IsMa...
by webmaster
Fri Sep 25, 2020 1:12 am
Forum: Q/A
Topic: Is any way to change selector on the fly?
Replies: 1
Views: 309

Re: Is any way to change selector on the fly?

You could start by importing this premade from the wizard. Just right click Project Explorer -> Globals -> Import Premade and visit the premade page. Then you'll be asked for a global name (you can just keep the default), and a selector and gatherer. For the gatherer, just use Gather.Text and as a s...
by webmaster
Sat Aug 29, 2020 2:53 am
Forum: Templates
Topic: AliExpress
Replies: 0
Views: 808

AliExpress

To use this template, place it on an empty folder and open it with Helium Scraper 3. Check this video for a quick walkthrough. The template has two globals that extract different pieces of information: ProductLinks : This global extracts top-level information without visiting any actual product page...
by webmaster
Sat Jun 13, 2020 5:10 pm
Forum: Q/A
Topic: Way Scroll Element Inside Page?
Replies: 1
Views: 1541

Re: Way Scroll Element Inside Page?

All scrolling functions scroll the currently selected element, which defaults to the whole page. So if you select the scrollable element it will scroll that element instead of the whole page: Select.ScrollableElement InfiniteScroll · Select.ListItem · 1000 · true To select the scrollable element, ju...
by webmaster
Tue Jun 09, 2020 9:12 pm
Forum: Q/A
Topic: Common Crawler: is it possible to download the found html files?
Replies: 2
Views: 1214

Re: Common Crawler: is it possible to download the found html files?

We've just updated Common Crawler to include the Sequence.WriteFile function. If you don't get an update prompt, this may be because we've migrated the publish location to AWS. If so, just uninstall it and reinstall it from here . Once you have the latest version (3.2.4.9) you can do this to save th...
by webmaster
Tue May 19, 2020 10:31 pm
Forum: Q/A
Topic: SQLite
Replies: 2
Views: 1499

Re: SQLite

There's no way to automatically flush the data, but you can save the project with File -> Save while it's still running without having to stop the extraction. Regarding the dot, not 100% sure about this but I think you can use brackets like "[Some.Thing]" in Access. Anyway, it'd make sense to be abl...
by webmaster
Tue May 19, 2020 10:27 pm
Forum: Q/A
Topic: Saving full HTML of URLs
Replies: 1
Views: 1275

Re: Saving full HTML of URLs

You can use Gather.HTML to get the current page HTML (or the HTML of any particular element when the element is selected), and since version 3.2.4.8 you can use Sequence.WriteFile to write files with arbitrary text content. In your case, you could do something like this, supposing all the pages you'...