Page 4 of 5
Re: Download file from WWW
Posted: Sun Mar 12, 2017 9:18 am
by edk
hbtip library itself does not support https.
You need to add support for SSL to be able to use https protocol.
Re: Download file from WWW
Posted: Sun Mar 12, 2017 10:20 am
by edk
mol wrote: ↑Sun Mar 12, 2017 6:06 am
Can you add to your code lines, which changes http AGENT to Mozilla or IE?
Marku add after oSoap:open
Code: Select all
oSoap:setRequestHeader ("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)")
edk.
Re: Download file from WWW
Posted: Tue Mar 14, 2017 11:58 am
by edk
Hi RPC.
RPC wrote: ↑Sat Mar 11, 2017 6:49 pm
(...) Could you please send a program to download the info in a CSV/text file. I would be very grateful. (...)
Here is a program that retrieves data from the NSEIndia.
- NSE.7z
- (1.82 MiB) Downloaded 304 times
Unfortunately I can't download the file "CSV" (from the link in the corner), because the link executes the script on the server, I can't run it.
However, I'm thinking that the downloaded html table by my program will allow you to easily obtain the datas.
- nse.png (19.83 KiB) Viewed 6603 times
PS. Especially in the program I used two methods to obtain data to show (teach) how this can be done.
The first method uses OLE, the second uses hbTIP.
Edward.
Re: Download file from WWW
Posted: Tue Mar 14, 2017 5:56 pm
by RPC
Hi Edward,
Many thanks for the program. You are just great !.
I can download the index information from nseindia website now.
The information is in html format. Can you please tell me a way to read this file and import the information in preferably CSV file.
Thanks
RPC
Re: Download file from WWW
Posted: Tue Mar 14, 2017 7:01 pm
by mol
You should parse downloaded html code to find <table> <row> <cell> tags.
Re: Download file from WWW
Posted: Tue Mar 14, 2017 7:55 pm
by edk
mol wrote: ↑Tue Mar 14, 2017 7:01 pm
You should parse downloaded html code to find <table> <row> <cell> tags.
+1
RPC, you don't need read a html file.
All datas are at
recv variable.
Just find
<div id='csvContentDiv' style='display:none;'> and cut string to
</div>
Char
: is record's separator (new line in csv format)
Re: Download file from WWW
Posted: Wed Mar 15, 2017 2:51 pm
by edk
RPC
Change in the code line:
to:
Code: Select all
csv:=MakeCSV(recv)
strfile(csv,STRTRAN(iType,' ','_')+qDate(fromDate)+"-"+qDate(toDate)+".csv")
and add this function:
Code: Select all
Function makeCSV( cString )
Local StartTag:="<div id='csvContentDiv' style='display:none;'>"
Local StopTag:="</div>"
Local PosTag:=AT(StartTag, cString)
Local csv:=""
IF PosTag>0
csv:=SUBSTR(cString,PosTag+LEN(StartTag))
csv:=SUBSTR(csv,1,AT(StopTag,csv)-1)
csv:=STRTRAN(csv,":",CRLF)
csv:=STRTRAN(csv,'"','')
csv:=STRTRAN(csv,',',';')
ENDIF
RETURN csv
Edward.
Re: Download file from WWW
Posted: Wed Mar 15, 2017 5:59 pm
by RPC
Hi Edward
Great ! I got the info in csv file.
Many Thanks.
Further I want to know about following two variables
Query:="?indexType="+STRTRAN(iType,' ','%20')+"&"+"fromDate="+qDate(fromDate)+"&"+"toDate="+qDate(toDate)
Can this be used to download from any website, say I want to download from another website-
www.bseindia.com-similar information. can i use the same query ? or each website has different format of query ?
cGet:="/products/dynaContent/equities/indices/historicalindices.jsp"+Query
the website's address is as follows
https://www.nseindia.com/products/conte ... x_data.htm
why did you put "dynaContent" instead of "Content" and "historicalindices.jsp" instead of "historical_index_data.htm"
Please guide.
Thanks
RPC
Re: Download file from WWW
Posted: Wed Mar 15, 2017 8:18 pm
by edk
The queries depends of host. Different query is for Google and others for Yahoo, and others for each host.
You need to analyze the code html web pages for which you want to query. Or look for information from the webmaster to send queries.
If you examine the code html
https://www.nseindia.com/products/conte ... x_data.htm, you'll see what doing the "get data" button
Apart from that, some hosts may need to define headers. Look for tools for html packet capture and see what the browser sends to the host.
Edward.
Indexes list:
- p4.png (75.62 KiB) Viewed 5425 times
Dates:
- p3.png (67.67 KiB) Viewed 5425 times
"Get Data":
- p1.png (83.94 KiB) Viewed 5425 times
The query:
- p2a.png (89.4 KiB) Viewed 5425 times
Note the & sign.
In Clipper you can not use it directly because it will be interpreted as a macro. Therefore, it occurs separately: "&" + "fromDate" instead of "&fromDate".
Re: Download file from WWW
Posted: Thu Mar 16, 2017 8:38 pm
by RPC
Hi Edward
Brilliantly explained !
I have understood quite a lot from the way you have explained it.
I would like to learn more about this so that I can download from another website.
Could you tell me what should I read on the internet, perhaps a quick tutorial on "querying", which would teach me.
If it is handy, could you please point out to some links.
Many thanks once again.