Jump to content
Tuts 4 You

How to access the internet?


LCF-AT

Recommended Posts

Hi guys,


 


can anyone of you tell me how to access any internet site via APIs xy?So I don't know how to explain it correctly but what I am looking for is very simple.So what I wanna do is to access any site link xy and reading the source code into memory.All this I wanna do with a extern tool / code without to use any browser etc.The only thing what I need to know is...


 


1.) What APIs can I use to call internetsites / specific links


2.) How to read source code of the link xy into memory


 


So I imagine like this....



push ascii string // http://siteABC.com/12345.html
call API xy or with login datas push ascii string // http://name:pass@siteABC.com/12345.html
call API xy check whether link request & login was successfully
je ReadSource
jmp exit
///////////////////////
ReadSource:
push site handle
call ReadMemory // source code in memory x
.......
.......
.......
///////////////////////
End:
push logout
call exit

...something like this I mean so you know right.Also the same I wanna do with sites where I need to login with my login datas without to confirm any messagebox which you normaly get in browser etc.So the question is how I could do this?Maybe you could help again. :)


 


Thank you


Link to comment

1) InternetOpen + InternetOpenUrl + InternetReadFile. Search for it - there are plenty of http downloader sources in ASM (hey, most malware downloads some parts from the net! ;) )


2) InternetOpen + InternetConnect + HttpOpenRequest + HttpSendRequest + InternetReadFile. Yes, it's slightly more complicated, but InternetConnect allows you to specify user/pass.


Link to comment

Hi,I got a way in my app,but I am not sure it works in your situation.


 


So 



 


 


2.) How to read source code of the link xy into memory

 


Maybe it looks like these



HINTERNET hNet = ::InternetOpen("Internet Access",PRE_CONFIG_INTERNET_ACCESS,NULL,INTERNET_INVALID_PORT_NUMBER,0);
if(NULL == hNet)
{
return FALSE;
}
HINTERNET hUrlFile = ::InternetOpenUrl(hNet,"Your Site string",NULL,0,INTERNET_FLAG_TRANSFER_BINARY|INTERNET_FLAG_NO_CACHE_WRITE|INTERNET_FLAG_RELOAD,0) ;
if(NULL == hUrlFile)
{
InternetCloseHandle(hNet);
return FALSE;
}
BOOL bRead =FALSE;
DWORD dwSize =0,dwDownLoaded,dwTotalWrite =0,dwError =0;
bRead =InternetQueryDataAvailable(hUrlFile,&dwSize,0,0);
if (bRead)
{
char *pszBuffer = (char *)calloc(dwSize+0x1,sizeof(char)); // alloc memory for read
bRead = ::InternetReadFile(hUrlFile,pszBuffer,dwSize,&dwDownLoaded);
HANDLE hFile = CreateFileA("A file to write", GENERIC_WRITE,
FILE_SHARE_WRITE, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL,
NULL); //Create Read file or using string struct to cache pszBuffer if (hFile !=INVALID_HANDLE_VALUE)
{
while (dwDownLoaded !=0)
{
pszBuffer[dwDownLoaded]='\0';
dwTotalWrite +=dwDownLoaded;
bRead =WriteFile(hFile,pszBuffer,dwDownLoaded,&dwDownLoaded,NULL);
memset(pszBuffer,0,dwSize);
InternetReadFile(hUrlFile,pszBuffer,dwSize,&dwDownLoaded);
}
dwError =GetLastError();
}
free(pszBuffer);
CloseHandle(hFile);
}
InternetCloseHandle(hUrlFile);
InternetCloseHandle(hNet);

Hope it can help you a little.


  • Like 1
Link to comment

There is also a method using winsock, socket, recv, send etc... 


Using this method however, you have to write the headers and all that jazz yourself. The higher level functions of wininet do that for you though.


Link to comment

Hello kao & White,


 


thanks for your informations about what APIs I need to check & exsample. :) So now I need to study this API description etc so I never used any internet API before.


 


Question: So I see I get already some problems with the API parameter NAMES XY as "PRE_CONFIG_INTERNET_ACCESS" etc so I can't push these as name in Olly so for these paramater I need the push values as 100 for exsample.How can I handle this problem without to use WinASM itself? :( Also I didn't work a long time no more with WinASM but if I remember right then I need to add a wininet.lib for right?Hmmmm!


 


Ok listen.Do you have or know any exsample exe files already which I could check?Or maybe you could create any simple one how does access any site (google etc) and does read the source.Just only a question of course before I test endless to find the right values xy for some parameter names you know. :)


 


EDIT: Good ok Encrypto but also I have no ideas about winsock etc.VB stuff right?So all in all I am just looking for any simple solution to access any internet site + reading source code of this site into memory and thats all already.So I don't wanna do it manually and thats the reason why I am looking for a nice & lazy way. :)


 


Thank you


Edited by LCF-AT
Link to comment

LCF, the appropriate values are defined within the header files (wininet.h and similar), just include them in WinAsm. And yes, you'll also have to link the library. That's as lazy as it gets.


Link to comment

Hi again,


 


ok I got it now a little bit to create a project in WinASM.All working so far as I imagine at the moment but I have one problem.So I also use the InternetQueryDataAvailable API but there I get not the right lenght value back!Just get a smaller size back and not the full size of the source.....why?



Invoke InternetQueryDataAvailable, FileHandle,addr Lenght,0,0

Have I to use any other parameter values at the end?So I wanna read the entire page source not just some bytes xy.Maybe I have done something wrong with the API before InternetOpenA & InternetOpenUrlA but also I see that I have not all parameter NAMES in my WinASM as postet by the exsample of White...so this below I don't have....



INTERNET_FLAG_TRANSFER_BINARY|INTERNET_FLAG_NO_CACHE_WRITE|INTERNET_FLAG_RELOAD

....so at the moment I can just read the entire source blind (alloc 100000 bytes in the hope that the source is not larger).Of course its a bad solution for this but maybe you know what the problem is in my case above.


 


greetz


Link to comment

Hi, sorry for a later resp.


 


1.


Header Declared in Wininet.h.


Library Use Wininet.lib.


DLL Requires Wininet.dll.


2.



BOOL InternetQueryDataAvailable(
__in HINTERNET hFile,
__out LPDWORD lpdwNumberOfBytesAvailable,
__in DWORD dwFlags,
__in DWORD_PTR dwContext
);

And MSDN Remarks,



This function returns the number of bytes of data that are available to be read immediately by a subsequent call to InternetReadFile. If there is currently no data available and the end of the file has not been reached, the request waits until data becomes available. The amount of data remaining will not be recalculated until all available data indicated by the call to InternetQueryDataAvailable is read.

3. For now,do you need any example ?


 


 


EDIT:



 


 


not the full size of the source

 


If you have a file and size's 1GB, do you have a free memory to cache that big file ? :)  :no:


So I choose a loop to read the available size till its end.

Edited by White、、
Link to comment

Hi again,


 


ok but how to make a loop with InternetQueryDataAvailable API to get the entire size of the source of any site?So I have test a little bit with different sites I for some sites I get the full source size back and for other sites not also if they have a smaller source size so thats a little bit strange.Thats the reason why I now use a reading size of 100000h bytes to be sure to get the full source size without to use the InternetQueryDataAvailable API. :)



invoke VirtualAlloc,0,100000h,MEM_COMMIT,PAGE_EXECUTE_READWRITE
....
Invoke InternetReadFile,FileHandle,SECTION,100000h,addr BytesRead
....

So yesterday I made got already some success with my test app to read the source / find strings xy + export them into log file. :)


 


So now I have a another question about searching text strings into source.So how can I do this in the best and simplest way?Just imagine I have load the full source in memory and now I wanna check for some ASCII strings inside and to log the data which comes right after the strings.


 


Exsample: I wanna find the string ABCDEFGH.So at the moment I just use a pattern dword compare of ABCD = 44434241 & 48474645



cmp dword ptr [edi], 44434241 ; ABCD cmp dword ptr [edi+4], 48474645 ; EFGH

But this is not so good so I dont wanna always check for opcodes.So I wanna do directly with the strings to find them but how and what API can I use for this?Lets say I have a text source size of 1000h bytes and somewhere inside are these strings to find and now I wanna check this source only one time to find all of them without to know where the strings starts.Normaly I could use commands as "REPE CMPS BYTE PTR ES:[EDI],BYTE PTR DS:[ESI]" etc but for this I need first to find the string start to compare it then but this method I don't like.


 


So I am looking for a API like this..



push addr text start
push size of full text lenght to check = 1000 bytes for exsample
push string to find pointer = ASCII "ABCD"
call API_XY_Who_Does_Search_For_It return = 0 = String not found in text
or
return = VA in eax where string starts of first found
ecx = rest size to check so that I can re-call the API Something like this you know

...but I dont remember any API who could to this etc.So at the other hand I could use the API CompareString but then I have to call this API for each byte (no idea whether this is good or bad because performance speed etc).


 


Ok so I think there is no big different if I use opcode check as strings or compare APIs about the speed.So now I use this compare API. :)


 


PS: I also see that I have no API parameter to choose for the CompareString API in WInASM. :( So I cant choose NORM_IGNORECASE for exsample so I get nothing I can just enter this manually!=?But why?Kernel lib & inc are in the header above.Strange.


 


greetz


Link to comment

1. A loop with InternetQueryDataAvailable



do
{
dwTotalWrite +=dwSize;
} while (InternetQueryDataAvailable(hUrlFile,&dwSize,0,0));

2. find string for opcodes,it is better using strstr command.



char str[]="abcdefgh";
char *str1=strstr(str,"f");
printf("%s",str1); output:fgh
Edited by White、、
Link to comment

Hi,


 


ok listen.I have a problem with the InternetReadFile API so I still get not the full source site back into memory.Mostly the first time I get some 1000 - 2000 bytes back and the rest is missing but if I try it again right after the first time (same again InternetOpenUrlA & InternetReadFile) then I get the full lenght of site source.The question now is why there is a problem with the API?So this sucks again!So now I have to read the memory and check the end for...



</body>
</html>

...for exsample which is the end of page source in my case.This is bad again so I don't wanna add tons of extra checks just because the API xy is not working correctly or whatever you know.Hmmm!Also note that I don't know the page source size of course which also gets updated each time.What now?


 


So I think the API comparestring is ok so far for me.Seems to work without problems & good speed but thanks for the strstr exsample.


 


Thanks again and maybe you have a solution for this size problem what I have and why the API does not work as it should (right?). :)


 


greetz


Link to comment

em, I am not really understand what is wrong with them,cause I am not a good coder.   :) Maybe you can google it for the reason,


 


I have test the part of  "InternetQueryDataAvailable", not quiet as good as I described, sorry.


this time I create a sample and the source code is below. So you can translate it to asm.


 


Example & src.rar


Link to comment

Hi,


 


thanks for your exsample file but I can't load / run it so there is any msvcr110.dll & msvcp110.dll problem for me (FlsGetValue not found in kernel32).I use XP so maybe you can compile it again.


 


But also with your exsample code the API just reads always just 296 bytes via InternetQueryDataAvailable but the site xy has some 1000 bytes.Hmmmmm!


 


Listen: The manually way is so simple....enter link xy in browser / press strg+U (open source page) / copy entire site to text file = Done. :) No idea why the APIs have again any trouble to do this correctly so maybe you know any other way which could work better without problems etc.


 


Thank you


 


EDIT: Ok wait....so can I also just repeat the InternetReadFile again & again till lpdwNumberOfByteRead is set to 00?So I have test it a little and this could or seems to work so after a few times the bytes getting more and then it will set to 00 but now I have the entire source into. :)


Edited by LCF-AT
Link to comment

Hi again,


 


I have a another question about the InternetOpenUrl API.So I see this API need always some little time (2-3 seconds) each time if I call any link.So is there a way to call this API faster etc?So the problem is that I have to call very much links (500 and later much more) and for each link I have to use this InternetOpenUrl API which takes a long time.So what can I do in this case to make it faster etc?So at my test app I did grab 500 links and the app dose need 5-6 minutes till it has checked them all.Thats not so good. :( Hmmm.


 


Just imagine I call any internet site which has many under sites 


hompage/_001.html


hompage/_002.html


hompage/_003.html


.....


hompage/_500.html


 


and each site I have to check for strings xy (if found on site then log to file later).Maybe you have any idea whether its needed to call each time the InternetOpenUrl or not or you have any other ideas etc.


 


Thank you


Link to comment

If you do not need to post any data to the url, you could connect to the url manually using raw sockets.

Which should remove some of the overhead created by the InternetXXX functions.

However you will still have the delay of the page loading on your connection speed regardless of what API you use. So there is going to be some delay no matter what.

Link to comment

 

 

can't load / run it

You need M.S visual C++ runtime library 2012 ,just install it, about 1.68 MB.

 

 

 

just reads always just 296 bytes via InternetQueryDataAvailable

Of course, you can just delete that part and just use InternetReadFile to read source and dwSize set to a large number till lpdwNumberOfByteRead is 0.

 

 

 

InternetOpenUrl.... make it faster

em,I have no idea with it, in my case, each time it takes about 400 ms.

Link to comment

Hi,


 


good ok so it seems that I have to live with the delay issue of InternetOpenUrl API. :( Also I have no ideas about raw sockets etc (VB stuff right?).Are these working faster anyhow?So as I said I only want to read the page sources.



M.S visual C++ runtime library 2012

So I have already the msvcr110.dll & msvcp110.dll.Anyway,so can you post a link where to download this pack on ms?


 


Yes so I think only to use the InternetReadFile API in a loop till lpdwNumberOfByteRead is set 0 seems to be the better solution instead to use InternetQueryDataAvailable API. :)


 


No idea why InternetOpenUrl takes so long time in my case. :( My internet connection is 1.1 MB/s download.Maybe its to less and you have any higher download speed. :)


 


Ok thanks again for your help guys.


 


greetz


Link to comment

Hey White,


 


thanks for the link now your file works. :) I have check it now and it works almost for 100 %. :) Your file does download the t4y site till line 7156



<a rel="nofollow" href='https://forum.tuts4you.com/priva

but it has 7254 lines...



</html>

...anyway so then I just keep to use InternetReadFile only to get all datas. :)


 


Thanks again.


 


EDIT: One question.So why I always get a ERROR_INVALID_HANDLE after using InternetCloseHandle API?



EAX 00000001
ECX 001C5500
EDX 000E0001
EBX 00CC000C
ESP 0012FA84
EBP 0012FAA0
ESI 408C90CA wininet.InternetCloseHandle
EDI 0178F890
EIP 004015C1 Example.004015C1
C 0 ES 0023 32bit 0(FFFFFFFF)
P 1 CS 001B 32bit 0(FFFFFFFF)
A 0 SS 0023 32bit 0(FFFFFFFF)
Z 1 DS 0023 32bit 0(FFFFFFFF)
S 0 FS 003B 32bit 7FFDE000(FFF)
T 0 GS 0000 NULL
D 0
O 0 LastErr ERROR_INVALID_HANDLE (00000006)
EFL 00000246 (NO,NB,E,BE,NS,PE,GE,LE)

greetz


Edited by LCF-AT
Link to comment

em, I aware that there is something "wrong" with InternetQueryDataAvailable before posting when the first time I use that api to get available size, I also had checked it by google and found several topics poster with the same problem,but forgotten the solution.   :(


 


About the InternetCloseHandle api,I checked the source again ,and set a code before and after it. <dwError =GetLastError();>


Found dwError is always 0.


 


 


Edited by White、、
Link to comment

Hehe again any strange API issue right. :) Anyway so I don't use this API anymore if the other way (read API) does also work in a loop.


 


About InternetCloseHandle API.So what is the problem now with this API?So I also get always a (ERROR_INVALID_HANDLE (00000006) back but in eax is also value 1 = succes right?



ERROR_INVALID_HANDLE The handle that was passed to the API has been either invalidated or closed.

Ok listen,so I see the problem should be the InternetReadFile API so after using this API and then the


InternetCloseHandle = ERROR_INVALID_HANDLE for both handles of InternetOpenA & InternetOpenUrlA API.So if I don't use the InternetReadFile API and call direct the two InternetCloseHandle APIs then I get a success for both.So it seems that there should be any another API (InternetCloseFile maybe etc no idea) to use before or?


Below a quick exsample....InternetCloseHandle only works if I not call InternetReadFile....



Invoke InternetOpen,addr szAgent,INTERNET_OPEN_TYPE_PRECONFIG,0,0,0
mov InternetHandle,eax Invoke InternetOpenUrl,InternetHandle,addr szUrl,0,0,0,0
mov FileHandle,eax
;;-------------
Invoke InternetReadFile,FileHandle,SECTION,100000h,addr BytesRead
;;------------- invoke InternetCloseHandle, FileHandle
invoke InternetCloseHandle, InternetHandle

...but the question is why it fails?Do you know the reason and how to fix it etc?


 


greetz


Link to comment

Hi its me again so I forgot something to ask before about a new problem I have.


 


Question: How to sort lines into a alphabetical order before write the export text file?So I mean is there any simple way / API etc what can do this to prevent a manually letter / nummer / sings checking?So I have already logged all text datas into a memory block which I can write into text file now but its not in a alphabetical order.....


 


Exsample:



Hammer
Sichel
preis
Zanken
Ablesen

So this I get now into my tex file each line any word.But I wanna export it in that way..



Ablesen
Hammer
preis
Sichel
Zanken

...but how to handle this?Maybe you know also a simple solution for this (APIs / codes / whatever can help).Sorry for asking again but "correctness must be" :)


 


greetz


Link to comment

About InternetCloseHandle API

 

Have you checked the sample ? Does it always get ERROR_INVALID_HANDLE when calling InternetCloseHandle ?

I have checked the source code again,and found no problem.

 

post-66121-0-18642400-1415243743.jpg

 

 

 

MSDN remarks Returns TRUE if the handle is successfully closed, or FALSE otherwise.

The function terminates any pending operations on the handle and discards any outstanding data. If a thread is blocking a call to Wininet.dll, another thread in the application can call InternetCloseHandle on the Internet handle being used by the first thread to cancel the operation and unblock the first thread.

If there is a status callback registered for the handle being closed and the handle was created with a non-NULL context value, an INTERNET_STATUS_HANDLE_CLOSING callback will be made. This indication will be the last callback made from a handle and indicates that the handle is being destroyed.

If asynchronous requests are pending for the handle or any of its child handles, the handle cannot be closed immediately, but it will be invalidated. Any new requests attempted using the handle will return with an ERROR_INVALID_HANDLE notification. The asynchronous requests will complete with INTERNET_STATUS_REQUEST_COMPLETE. Applications must be prepared to receive any INTERNET_STATUS_REQUEST_COMPLETE indications on the handle before the final INTERNET_STATUS_HANDLE_CLOSING indication is made, which indicates that the handle is completely closed.

An application can call GetLastError to determine if requests are pending. If GetLastError returns ERROR_IO_PENDING, there were outstanding requests when the handle was closed.

 

About string sort. Here is my solution,and tests good.

 

EDIT: forgot one cmp

Sort & src upt.rar

 

 

Edited by White、、
Link to comment

Hi again,


 


yes I have checked your file you have send to me before "Example.exe" and there I get also 2 times a " LastErr ERROR_INVALID_HANDLE (00000006)" after stepping over InternetCloseHandle API..below the VAs of your file.Just load it into Olly and set a BP on it and then step and check what you get.Just check this in Olly to see whether you get the same trouble.



004015B9 53 PUSH EBX
004015BA FFD6 CALL ESI ; wininet.InternetCloseHandle
004015BC FF75 F4 PUSH DWORD PTR SS:[EBP-0xC]
004015BF FFD6 CALL ESI ; wininet.InternetCloseHandle

About your new sort file....so I still have again any problem with this file to get it run so Windows tells me that the file is not a valid 32 file.So it seems that I also for this new file need any system add-on which I don't have installed yet.So what do I need now again?


 


So can I use your source code also in WinASM?I don't think so or?


 


Thanks again so far


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...