URL Variations

[webmaster]

This is a quick way to generate a list of URLs when a number in the URL is the only thing that changes, typically the page number or the starting item, such as in this example:

• https://www.google.com/search?q=helium+scraper&start=50
• https://www.google.com/search?q=helium+scraper&start=60
• https://www.google.com/search?q=helium+scraper&start=70

The function takes 4 arguements:

• url: The base URL. Must contain a @ symbol to be replaced with each number.
• start: The starting number.
• step: The step between each number.
• count: The number of URLs to generate.

To use this on your project, follow these instructions:

{
"api": "1.0.2.1",
"parameters": {
"url": {
"caption": "URL",
"type": "text"
},
"start": {
"caption": "Start",
"type": "number",
"default": 1
},
"step": {
"caption": "Step",
"type": "number",
"default": 1
},
"count": {
"caption": "Count",
"type": "number",
"default": 10
}
}
}

Code: Select all

function (url start step count)
   Sequence.FromValues
      ·  List.Range
            ·  0
            ·  count
   as index
   return
      ·  String.Replace
            ·  url
            ·  "@"
            ·  Core.ToString
                  ·  +
                        ·  start
                        ·  *
                              ·  step
                              ·  index

To load the output URLs, right click the new action and select Output Result, and then give a descriptive name to the output variable, such as url. Then, add a Browser.Load action and pass this variable as the argument.

The following example will generate the 3 URLs on the list above and navigate to each of them. This example assumes you've already pasted the code above and called it Variations:

Code: Select all

Variations
   ·  "https://www.google.com/search?q=helium+scraper&start=@"
   ·  50
   ·  10
   ·  3
as url
Browser.Load
   ·  url

[sawal86]

I have ready list of links and I need to make Helium 3 to navigate each link and extract needed info. I have had a few tries, but couldn't resolve it. I have created database with 2 columns: ID and link. Help please!

[webmaster]

Try this: https://youtu.be/7itQkY3wUMg