Navigate URL's character limit

Questions and answers about anything related to Helium Scraper
Post Reply
crookedleaf
Posts: 38
Joined: Tue Dec 11, 2012 6:44 pm

Navigate URL's character limit

Post by crookedleaf » Wed Dec 19, 2012 7:28 pm

it seems like I am running out of characters on the Navigate URL's list. Here is the URL i am trying to run (i have x'ed out the domain, but the amount of x's are the same amount of letters)

Code: Select all

https://www.xxxxxxxxxxxxxxxxxxxxxx.com/ffr/secure/member/reservations/searchpost.do?selectedUnitTypes=STD&selectedUnitTypes=1BR&selectedUnitTypes=2BR&checkinDateDate=01%2F18%2F2013&flexibleDay=4&lengthOfStayDate=3&includeLockOff=true&selectedResorts=0037&whenType=byCheckinDate&whereType=byResort
the original URL used to be twice as long because it was full of useless variables, but i have removed as many unnecessary variables as i could. this is as short as i can get the URL. when I go to save the table, i get this message:
The field is too small to accept the amount of data you attempted to add. Try inserting or pasting less data.
here are the details of the error message:

Code: Select all

<?xml version="1.0" encoding="utf-16"?>
<ErrorReport xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <InnerError>
    <InnerError>
      <Message>The field is too small to accept the amount of data you attempted to add.  Try inserting or pasting less data.</Message>
      <TypeName>System.Data.OleDb.OleDbException</TypeName>
    </InnerError>
    <Message>Failed to run query: 'INSERT INTO [URLs] ([HTTP]) VALUES ('https://www.xxxxxxxxxxxxxxxxxxxxxx.com/ffr/secure/member/reservations/searchpost.do?selectedUnitTypes=1BR&selectedUnitTypes=2BR&checkinDateDate=01%2F09%2F2013&flexibleDay=4&lengthOfStayDate=3&includeLockOff=true&selectedResorts=0064&whenType=byCheckinDate&whereType=byResort')'</Message>
    <TypeName>System.Exception</TypeName>
  </InnerError>
  <Message>The field is too small to accept the amount of data you attempted to add.  Try inserting or pasting less data.</Message>
  <TypeName>System.Exception</TypeName>
  <StackTrace>   at Player.DataBaseManager.ExecuteNonQuery(String sqlQuery)
   at WebScraperAndPlayer.DataTableEditor.a(Dictionary`2 A_0, DataView A_1)
   at WebScraperAndPlayer.DataTableEditor.b()</StackTrace>
  <TimeStamp>2012-12-19T11:19:42.1292984-08:00</TimeStamp>
  <Version>2.3.8.8</Version>
  <WorkingSet>131563520</WorkingSet>
  <Is64Bit>true</Is64Bit>
  <WinVersion>Microsoft Windows NT 6.1.7601 Service Pack 1</WinVersion>
  <BrowserVersion>9.0.8112.16457</BrowserVersion>
</ErrorReport>
is there ANY way to get past this restriction? my normal list of URL's is around 200 that need to be scraped.

webmaster
Site Admin
Posts: 512
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Navigate URL's character limit

Post by webmaster » Thu Dec 20, 2012 3:03 am

How did you create the URLs table?. The HTTP columns needs to be a Memo data type, which allows unlimited amount of characters, instead of Text, which allows up to 255.
Juan Soldi
The Helium Scraper Team

crookedleaf
Posts: 38
Joined: Tue Dec 11, 2012 6:44 pm

Re: Navigate URL's character limit

Post by crookedleaf » Thu Dec 27, 2012 7:40 pm

after deleting the table and recreating it with Memo fields, it does hold the full URL. thank you :D is there any way to change the field types or anything without just deleting the table and recreating it?

webmaster
Site Admin
Posts: 512
Joined: Mon Dec 06, 2010 8:39 am
Contact:

Re: Navigate URL's character limit

Post by webmaster » Thu Jan 10, 2013 2:44 pm

You could run SQL:

Code: Select all

ALTER TABLE [MyTable] ALTER COLUMN [MyColumn] Memo
where MyTable is your table and MyColumn is the column to alter. Here is more about alter table.
Juan Soldi
The Helium Scraper Team

Post Reply