How to access the internet?

[LCF-AT]

Ok I found a solution so I just need to use InvalidateRect API after a resize to prevent this kind of flicker issues (I got with goupbox). I also found out if you do use GetWindowRect API that you also need to sub the X cor of Width(right) and Y of Height(bottom) to get the right Rect datas so this info I didn't found on MSDN API description. Just only a info.

 

greetz

[LCF-AT]

Hi guys,

 

trying to play a little again with some internet APIs and this time I wanted to know how to read the Header datas but I don't get it work again and try to use HttpOpenRequest & HttpQueryInfo API.Do you have any examples for that maybe?

 

Thanks again

[LCF-AT]

Hi again,

 

so I am looking for different example codes (asm if possible) how to get http header datas.I tried to find any example tools but just find any online checkers for that like this...

http://www.tools4noobs.com/online_tools/get_http_headers/

 

...and one tool I found on DIC..

http://www.dreamincode.net/forums/topic/266444-masm-winsock-interacting-with-webpages/

...buts itsusing Winsock method / APIs but also this tool seems not to work for all sites as google.com for example so there I get this....

Sending:

------------------------

GET /? HTTP/1.0

Host: www.google.com

Connection: close

User-Agent: test
Received:

------------------------

HTTP/1.0 302 Found

Cache-Control: private

Content-Type: text/html; charset=UTF-8

Location: http://www.google.de/?&gfe_rd=cr&ei=....

Content-Length: 263

Date: Tue, 08 Sep 2015 13:29:15 GMT

Server: GFE/2.0

...so if I use any online checker then I get other results if you check with this..

http://www.tools4noobs.com/online_tools/get_http_headers/

http://www.webconfs.com/http-header-check.php

...there I get HTTP/1.0 200 OK and much other infos.

 

So do you have any examples codes for that (not in php or Java etc)?Would be nice if you got & post something about this what I can test.

 

Thank you

[LCF-AT]

One thing.

 

I tried also to check http status of any URL which gets redirected to main site and I tried to use this way...

http://www.cryer.co.uk/brian/delphi/wininet/example_isurlvalid.htm

...but isn't working so I still get status 200 back.

https://de.wikipedia.org/wiki/HTTP-Statuscode

Now I tried to use flag of INTERNET_FLAG_NO_AUTO_REDIRECT with InternetOpenUrl API and after this using HttpQueryInfo API but there I get status code 302 back also if the site is present!?!So what can I choose now to get / check the status code correctly?

 

In php its like this...

$fileUrl = "http://www.homepage.com/uperleft/calc";

$AgetHeaders = @get_headers($fileUrl);

if (preg_match("|200|", $AgetHeaders[0])) {

echo "\r\nOn      | calc site";

} else {

echo "\r\nOff     | calc site";

}

...if I can access this URL = Online and if not and I get redirected = Offline.So how I have to use the APIs above with the right flags to get same results?

 

greetz

[simple]

"parse http header" can mean a lot, i dont know about wininet, i dont like it because it doesn't have ssl & prefer to use normal sockets (then u can copy paste ur code to an ssl socket, unlike wininet where u have to rewrite everything)

 

1st - read rfc 2616 - http://tools.ietf.org/html/rfc2616#

 

yes, it sucks to read the rfc, what sucks even more is that many server/clients aren't rfc compliant & u have to write code for non rfc compliant clients/servers

 

i have sample codes in C that do this, but not masm, maybe later in this week or next week i can put it in masm if u dont figure it out, but i think u will

 

EDIT - i gave u some bad information but this should be correct -

 

for example, forum.tuts4you.com server will give header info w/this command (same as the other tools u posted do)

POST / HTTP/1.1\r\nHost: forum.tuts4you.com\r\n\r\n

in google's case, it will give u a 302 found but u need 200 OK

 

so in ur case u have this response -

HTTP/1.0 302 Found

Cache-Control: private

Content-Type: text/html; charset=UTF-8

Location: http://www.google.de/?&gfe_rd=cr&ei=....

Content-Length: 263

Date: Tue, 08 Sep 2015 13:29:15 GMT

Server: GFE/2.0

u need to create another request that looks like this

HEAD /?&gfe_rd=cr&ei=.... HTTP/1.1\r\nHost: www.google.de\r\n\r\n

then that will give u the 200 response just like the tools u posted do

 

also in response to ur post #54, i think the proper way to check for redirect is to use "Location:" value in the header, not the response code.

Edited September 8, 2015 by simple

[LCF-AT]

Hi simple,

 

thanks for your answer.

 

Yes the WinInet APIs working also strange and I get different results back for the SAME url (which is online now) I do test so that I get status 200 then 302 then again 200 then 302 and then just only 302!?!WTF!Uhhhhhm!But I also get the same if I use the tool from DIC what used WSock APIs.A few times I get 200 then 302 so what the problem in that case?

 

I also did test again google site with that tool from DIC I did post before and it seems to be right what you said so is this any kind of country restriction?If I use .de instead of com then its working.Hmm yes I can also not access google.com in browser of course.

 

Ok so that means if I want to read the header of any site (123.com) and I get results of 302 (redirect) then I get the redirect location in header back (like german version of site) and then I have to use this redirect location and do another request and get now status 200.Hmmm.Ok this step I do understand now so far a little but its more work and I need to call again InternetOpenUrl for example.Maybe its better not to use this API also not twice but what would be better?

 

Another thing.So what is the fastest way to read the header datas?I mean WinInet APIs where I use InternetOpenUrl (need most time) or HttpSendRequest need a while if I call them.I have seen that the WSock APIs are working much faster.

 

Ok listen simple,so if you if got any examples (WSock / WinInet etc) then it would be very nice if you could post some of them for me and you also don't need to translate all codes etc just the important API stuff to see how to use it correctly.

 

Thanks again (also for the RCF link so there's a lot to read) and till later.

[simple]

Well I'm not sure about the wininet I don't use it nor like it.

 

U have to parse all this data yourself lcf.

 

google returns 302 Found for a redirect, but www.yahoo.com returns 301 Redirect (the correct rfc compliant code) . this why i say its better to look for Location tag. u will probably need to learn MIME too.

 

also note that many servers will redirect to https... (ie u need openssl), many will use other tags besides Location, etc etc. read the rfc.

 

here the bin, u are a master reverser i'll leave it to u to rip the ASM from the functions u need. u will need to write ur own parsing codes.

int main()

{

        char *Server = "forum.tuts4you.com", *Header = NULL;

        int Socket, Port = 80;
        printf("[+] Connecting to %s:%i...\n", Server, Port);
        // server is the host, port is port, and 2 is timeout in seconds

        // read about non blocking sockets if u dont get it

	Socket = TcpConnectNonBlock(Server, Port, 2); 

	if (!Socket)

	{

		printf("[!] TcpConnectBlock() error %i\n", Socket);

		goto END;

	}
	printf("[+] Connected\n");
	Header = HttpGetHeader (Socket, "forum.tuts4you.com");
	if (Header != NULL)

	    printf("[+] Header - %s\n", Header);
CLEANUP:   

        closesocket(Socket);

	WSACleanup();

END:

    return 0;

}

Http.rar

Edited September 9, 2015 by simple

[LCF-AT]

Hi again,

 

thanks for your example file simple. I still do tests with WinInet & WinSock to find out what better is for me.Its seems that WinSock is better but a little harder to understand as WinInet.

 

One more question.So I tried to find a WS API xy similar as the API called HttpQueryInfo for WinInet where I can use it with specific flags...

HTTP_QUERY_VERSION              equ 18 

HTTP_QUERY_STATUS_CODE          equ 19 

HTTP_QUERY_STATUS_TEXT          equ 20 

HTTP_QUERY_RAW_HEADERS          equ 21 

HTTP_QUERY_RAW_HEADERS_CRLF     equ 22 

HTTP_QUERY_CONNECTION           equ 23

HTTP_QUERY_LOCATION             equ 33

etc

...to get a direct return of what I do request for instead to check the entire Header by strings etc.So what API can use for that?

 

greetz

[LCF-AT]

Hi again,

 

one more question about https.So do you also have a example for that how to handle https to get right header datas?So I just check yahoo.com for example and get as location the same address back just as https.

Send:

---------------------------

HEAD / HTTP/1.0

Host: yahoo.com

Connection: close

User-Agent: Testphase
Received

--------------------------

HTTP/1.0 301 Redirect

Date: Fri, 11 Sep 2015 15:52:17 GMT

Via: http/1.1 ........yahoo.com (ApacheTrafficServer)

Server: ATS

Location: https://www.yahoo.com/

Content-Type: text/html

Content-Language: en

Cache-Control: no-store, no-cache

Connection: keep-alive

Content-Length: 373

So how to solve this now to call the https location and getting status 200 + original site pagesource using GET command later?Maybe you know this too.

 

If I use any online checker then I get both results..

https://www.tools4noobs.com/online_tools/get_http_headers/
    HTTP/1.0 301 Redirect

    Date: Fri, 11 Sep 2015 15:35:38 GMT

    Via: http/1.1 ir25.fp.bf1.yahoo.com (ApacheTrafficServer)

    Server: ATS

    Location: https://www.yahoo.com/

    Content-Type: text/html

    Content-Language: en

    Cache-Control: no-store, no-cache

    Connection: keep-alive

    Content-Length: 371
    HTTP/1.0 200 OK

    Date: Fri, 11 Sep 2015 15:35:38 GMT

    P3P: policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"

    X-Frame-Options: DENY

    Strict-Transport-Security: max-age=2592000

    Set-Cookie: DNR=deleted; expires=Thu, 11-Sep-2014 15:35:37 GMT; path=/; domain=.www.yahoo.com

    Set-Cookie: DNR=deleted; expires=Thu, 11-Sep-2014 15:35:37 GMT; path=/; domain=.www.yahoo.com

    Set-Cookie: PH=deleted; expires=Thu, 11-Sep-2014 15:35:37 GMT; path=/; domain=.yahoo.com

    Vary: Accept-Encoding

    Content-Type: text/html; charset=utf-8

    Age: 0

    Via: http/1.1 ir42.fp.bf1.yahoo.com (ApacheTrafficServer)

    Server: ATS

    Cache-Control: no-store, no-cache, private, max-age=0

    Expires: -1

Thanks again.

[LCF-AT]

Hi again,

 

ok so today I made a another test code with Wsock instead of WinInet and its cool so Wsock is a lot faster than WinInet. The only problem thing is that stupid string handling if you got another location to check hmmmm!So this is really a bit bad to handle but its very fast going and its really a big advantage so that I think I keep working more with WSock in the future.

 

PS: Below a little picture piece of my app.Just checking 150 urls in just a few seconds.

 

greetz

[simple]

using winsock is like running naked throught he street, every can see ur private parts - no encryption!

 

to access https:// sites, like yahoo, u need to use openssl. pretty sure it has masm libs. besides, in a few years internet will have some type of mandatory encryption at all layers so might as well start now.

 

u actually need to parse 2 protocols, http & mime. 9/10 coders will buy libs to do that, all the good ones are not free (unless it's in php, javascript,etc). so if u dont want to pay then u have to learn to parse strings.

[SkyProud]

In my location(China), when visiting foreign websites, I often get some Google ads about VPN(Virtual private network) which claims - unblock any website, truly anonymous, fastest in the world.

[LCF-AT]

Hi,

 

great!More work and more to study!Ahhhmmm!

 

Is that the reason why WinSock is faster than WinInet (or is WinInet more secure)?What about WinInet....do I also run naked around with that too also if I just request http?

 

So you mean I have to install openssl and using that libs & APIs instead of WinSock APIs + sending all crypted instead naked (FKK style).

 

Have I do install any openssl package?So I mean if I do use that libs in my app does it then also work on other system who have not installed openssl?How to get the libs for ASM?On net I don't find again anything with assembly + openssl together just with C++ where I found a video but also there wasn't any libs inside and user did execute any commands in CMD window to make something?!?So what to do now?Can you give me any newbie advices etc (simple if possible)?

 

PS: Ah no I don't wanna buy any extra libs or else if they aren't free just to do the parsing stuff (http/s rtmp/* rtsp) etc so I think this I can handle by myself also if I don't like to work with strings on that way.

 

PS2: So I do use rarly VPN just sometimes to bypass any stupid Geo-Blockings or else.

 

greetz

[simple]

Is that the reason why WinSock is faster than WinInet (or is WinInet more secure)?What about WinInet....do I also run naked around with that too also if I just request http?

wininet is a naked runner too. Wininet is based on winsock so using directly winsock is like cutting out the middle man (ie its faster).

So you mean I have to install openssl and using that libs & APIs instead of WinSock APIs + sending all crypted instead naked (FKK style).

u use ssl socket on top of normal windows sockets so u need to know both. but using openssl is like having a 10 meter high wall on every side & above u instead of running naked!

 

id just do my whole project w/SSL sockets as almost every website should be SSL today. u need to read about normal winsock + openssl. im not 100% positive openssl offers masm headers but try, go on masm forum & ask there.

PS: Ah no I don't wanna buy any extra libs or else if they aren't free just to do the parsing stuff (http/s rtmp/* rtsp) etc so I think this I can handle by myself also if I don't like to work with strings on that way.

if u dont want to buy them u have to do it yourself, but this may be months of work for u depending on exact needs.

[SkyProud]

PS2: So I do use rarly VPN just sometimes to bypass any stupid Geo-Blockings or else.

It is important for some country domains in GeoIP.dat, and there is some implementations such as OpenVPN.

 

Yours sincerely.

[LCF-AT]

Hi again,

 

thanks again for the infos so far guys.

 

I did understand your declaration about the naked thing.So I checked the masm forum already but there wasn't postet anything about that (used google site search).Hhmmmm.Seems to be not so simple to handle that thing on any easy way now to get dressed during requesting any internet things.The other problem I see is that I need to create any certificate for me which I can use but how should this work on other systems?Does the user xy then also need to handle this too?I was checking the NET a little yesterday to find any alternative etc and found some other CMD tools as CURL (nice tool too but there are tons of versions to get) and for that you also have to create a certificate before you can request / handle ssl sites.

 

One more question.Lets say I have just to handle http sites and no https so is there also a tiny or other way to request infos without to be naked or anything like that etc?

 

greetz

[SkyProud]

I have used cURL, it is a command line tool for URL, and has many options, try the latest version if you are not sure.

 

The certificate is issued by the CA(certificate authority), OpenSSL and GnuTLS can handle this. I followed the step-by-step guidance of commands on a linux x64 machine. The process of OpenSSL seems to be easier to me. However, I figured out GnuTLS after reading some more documentation. Maybe it would be easier for European user. I remember it requires a CA cert first, then use the CA cert's signing to generate the private key, then use both the CA cert's signing and the private key to publish the public domain cert, such things like that. During the process you have to provide the password of the private key and the domain name, and the matchness of the domain name be checked, so there are many details in the whole process. Asking the site hosting provider for help can save a lot of work.

Edited September 14, 2015 by Tianjiao

[simple]

cURL & libcurl are buggy crap. They promote them like it's a full protocol lib but all it is is a buggy wrapper around openssl. To be clear, curl doesn't parse headers & has many, many bugs.

 

U don't need to worry about making a cert, u only verify it, if u want to. Verifying the cert is not manditory for an ssl connection but it isn't hard either, there's a lot of info on this. I'd worry about cert verify once u get everything else working.

 

No point in using curl cmd tool when openssl has their own that's much more stable & does the same thing.

 

Download this - http://slproweb.com/download/Win32OpenSSL_Light-1_0_0s.exe- install it - C:\OpenSSL-Win32\bin\openssl.exe - this is the cmd line tool - u can connect it to masm via CreateProcess():

 

C:\OpenSSL-Win32\bin\> openssl s_client -connect forum.tuts4you.com:443

...

this shows cert info, u can ignore it but it doesnt verify the cert because u need to point it to a .crt - u can ignore it

...

GET / HTTP/1.0

 

HTTP/1.1 302 Moved Permanently

.. rest of headers...

...

 

At least for http protocol, as long as u have no ssl u r naked. With SSL u are not, there is no in between.

[LCF-AT]

Hi again,

 

thanks for your answers again.So of course I try to find / use any easy solution so don't make it too compex for me and I am just a newbie on that internet field.

 

Ok I tried the openssl light now.First problem I got is anything with the path to the openssl.cfg so I got it in same folder but openssl seems not to read it from that location and tries to access...

WARNING: can't open config file: /usr/local/ssl/openssl.cnf

...I tried to setup this in cmd I did put in same bin folder...

openssl set OPENSSL_CONF=c:\OpenSSL-Win32\bin\openssl.cfg

...but dosen't work.Ahhm!Ok listen,so maybe you can explain the steps another time for me (for dummys if possible) + another infos I need to know + a tiny example + link of all commands I can use etc.Sorry for asking again but I still don't really understand the way.

 

Ok lets summary a litte.If I later use openssl CMD tool with any commands xy to read anything on net etc then it will do this with a encryption to be not naked.If I don't use it and do request anything from net raw (Wsock / etc APIs) then I am naked.So what does it really mean so what infos etc can be read of my system etc?Its just the communication from A - B and between is any BadBoy who can read everything rawly?Sorry again for asking but this makes me a litte confuse and I am not sure whether I need this or not,ok maybe its better to use it to be more safe etc but this I can't also use with my own apps without to have any libs right?So then I just could use the CMD tool itself with CreateProcess API + arguments.

 

So why are there never any simple solutions.

 

By the way,so what about any other apps where you got the feature to update (Firefox / Adobe etc) so they do it also rawly or?

 

So is there no main app to get which you could install on your system what you can enable (ssl ON / OFF) what does control all access to internet?So I mean similar as a Sandbox or something?

 

greetz

[SkyProud]

Thanks, verifying the domain cert only requires the signing of the CA, it would be much easier.

 

For the conf file, maybe you have to add the path of the folder of OpenSSL into the environment variables of the Windows system.

 

In Vista, Windows 7 and Windows 8 you can set path from command line  using ‘setx’ command.

setx path "%path%;c:\OpenSSL-Win32\bin\"

Edited September 15, 2015 by Tianjiao

[whoknows]

I have no clue, about low level coding... But allow me to say :

 

true, seems that the example using localhost.. just download a portable one @ http://www.usbwebserver.net/en/

 

put the file from pastebin to appdir/root/post.php

[LCF-AT]

Hi again,

 

I have a new small question and maybe you can help again.At the moment I try to write a Wsock code piece (just simple thing) which I wanna write as sniped routine so that I can use it always in any of my projects (you know what I mean) to prevent writing the same again & again.So I wrote now this another asm file what I can invoke with some paramters (like a API) and its working so far but the problem I have now is that I want to call my routine as own thread (CreateThread API).So how can I now use CreateThread API with my paramters of my API?

This is my call to my snpied with 2 paramters...
invoke PageSource,offset LINKTEST,offset HOSTTEST
...but for CreateThread I use this...
invoke  CreateThread, NULL, NULL, addr PageSource, 1, NULL, NULL

invoke  CloseHandle, eax

You know what I mean right?I wanna call my routine with 2 paramters but this I wanna let run as thread so that the app not hangs during checking but how to send now my 2 parameters?I am little confused now.

 

greetz

[simple]

U have to put your data into a struct, then pass the struct as a parameter

T struct
  SocketHandle       dword ?
  SocketTid          byte ?
T ends

local Socket:T
...
mov Socket.SocketHandle, Handle
mov Socket.SocketTid, Tid
invoke CreateThread, 0, 0, addr MyThread, addr Socket, 0, 0
...

U can also use global variables for this, but above way is better.

Almost every protocol is designed for request-receive conversations, so sending/receiving data in different threads will cause u problems, especially in HTTP. Although it's ok to create a socket in main thread for example, then pass it to a background thread for something like updating a GUI, it's probably best to keep creation - use - closing all in 1 thread so no accidents happen, just my opinion though.

[LCF-AT]

Hi simple,

yes something like this I wrote with a own struct (sorry forgot to send feedback about it).

checko STRUCT
  p_link      DWORD      ?
  p_host      DWORD      ?
  p_check     DWORD      ? 
  p_mem       DWORD      ? ; PageDump
  p_lvh       DWORD      ? ; LV Handle
  p_tabh      DWORD      ? ; tab Handle
checko ENDS

             .elseif wParam == IDM_NEWONLINE ; <---
                invoke RtlZeroMemory,offset tester2,sizeof tester2
		m2m tester2.p_link,offset LINKTEST
		m2m tester2.p_host,offset HOSTTEST
		m2m tester2.p_lvh,  LISTVIEW		
		m2m tester2.p_tabh, TABHANDLE		
		lea edi,tester2     
		invoke  CreateThread, NULL, NULL, addr PageSource,edi, NULL, NULL
		invoke  CloseHandle, eax

So its working so far and I can check all sites I wanna check.Thanks again for your answer simple.

greetz

[LCF-AT]

Hi again,

today I have another question about requesting some http stuff and how to do some of them manually.So what I wanna know is how to get my response of a request I did decoded instead to get the encoded response.If I do use Fiddler with same paramters like I did manually (wsock) then Fiddler does say that the response is encoded and need to decode before checking and right after I choose this option in Fiddler it does show the entire resonse decoded but I can't see what Fiddler did to decode it.In my received header I get this paramters.

HTTP/1.1 200 OK
Server: nginx/1.0.15
Date: Mon, 21 Dec 2015 17:50:56 GMT
Content-Type: text/html
Transfer-Encoding: chunked
X-Powered-By: PHP/5.3.3
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Encoding: gzip

3b2b
‹      í}ùzÛ¶òèßö

So I tried to add Transfer-Encoding: chunked & Content-Encoding: gzip into another GET request but it didn't work and get some infos about content lenght required but also if I add this...

Content-Lenght: 15147

...I don't get success.So what I have to do now correctly to get my encoded response decoded?Have I to do another requests and if yes what I have to add or can I decode my encoded response anyhow without to do another request?Would be nice if you could help and tell me how to do it.

greetz