Jump to content
Tuts 4 You

CMD tool works diffrent with aria2c!

LCF-AT

By LCF-AT
in General Discussions and Off Topic

 Share

Recommended Posts

LCF-AT

LCF-AT

Posted
Posted

Hi guys,

so I found out some strange thing when trying to use the system CMD.exe tool and execute it with specific commandlines.A while ago I found a cmd tool called aria2c you can use to download stuff etc.In some cases I get some error messages when I do execute a aria2c commandline manually.

Example: In extern file I have put some URLs into with filenames below... 

aria2c -j 1 -x 1 -i "Output.urls"

....now when I do execute this manually in CMD window I get this logged... 

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
3ebcba|OK  |   2.1MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag0
11ffe7|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag1
be8564|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag2
d0f719|OK  |   6.7MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag3
470ca8|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag4
71402c|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag5
2f7d79|OK  |   4.1MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag6
ac43e7|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag7
a9ad73|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag8
4c9fe1|OK  |    10MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag9
1a7b2d|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag10
444208|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag11
65cb14|OK  |    12MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag12
8e6b21|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag13
c5a280|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag14
053e3c|OK  |   8.3MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag15
76b757|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag16
740c59|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag17
a79b6d|OK  |    12MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag18
4d50ab|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag19
40a15a|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag20

....as you can see that aria does successfully download just every thrid URL (0,3,6,9,etc) and all other fail and the question is why?So above of this result list I can see the debug infos about all URLs and those with errors showing me all the same 2 error infos. 

[NOTICE] Download complete: C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag0  <--- success

[ERROR] CUID#8 - Download aborted. URI=https://....
Exception: [AbstractCommand.cc:351] errorCode=1 URI=https://....
  -> [SocketCore.cc:1018] errorCode=1 SSL/TLS handshake failure: Error: Die Sperrfunktion konnte die Sperrung 
nicht überprüfen, da der Sperrserver offline war.
(80092013) <---- this = fail 
                 
                 
[ERROR] CUID#9 - Download aborted. URI=https://...
Exception: [AbstractCommand.cc:351] errorCode=22 URI=https://....
  -> [HttpSkipResponseCommand.cc:239] errorCode=22 The response status is not successful. status=403  <---- this fail

In first place the download got success.In second place I get that "80092013" error and on third place I get a 403 response message.Pretty strange of course.I tried to find out what that error means and found this info on MSDN...

https://support.microsoft.com/en-us/topic/-0x80092013-crypt-e-revocation-offlinea-error-message-when-you-try-to-verify-a-certificate-that-has-multiple-chains-in-windows-server-2008-or-in-windows-vista-9636f0e3-5347-db2d-d8c6-6b3cf200c099

....something with used certificate or so.Not sure about that now.Now I do the same with adding a new command called "--check-certificate=false" to disable the check and now I get this results back.... 

aria2c --check-certificate=false -j 1 -x 1 -i "Output.urls"

.... 

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
d09113|OK  |   2.6MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag0
3997c8|OK  |   774KiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag1
1b6499|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag2
816d6f|OK  |   6.7MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag3
6e0db3|OK  |   1.4MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag4
e947b6|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag5
5ab864|OK  |    10MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag6
2c7d98|OK  |   2.3MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag7
33359e|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag8
78f763|OK  |    12MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag9
c5e119|OK  |   2.5MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag10
0c9f75|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag11
55a8b0|OK  |    12MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag12
e9b4a6|OK  |   2.9MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag13
ac5d6b|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag14
9b0a8d|OK  |    12MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag15
fb921b|OK  |   3.9MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag16
dbf0af|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag17
1da297|OK  |    16MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag18
589fea|OK  |   5.5MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag19
c8844b|ERR |       0B/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag20

....now I get the first 2 downloaded but not the third.So the with new commandline "--check-certificate=false" I could bypass this 80092013 error and at the end there is still the second error of 403.So normaly there is no reason to get a 403 response so I can access all URLs normaly.I just did wonder what it could be and how to find out the problem and bypass it.Just as info.In the example above I started the CMD tool manually by entering CMD into titlebar/path of Windows Explorer or via right click on start button etc.

Now a while ago I found some helpfully tool called HTTP Toolkit...

https://httptoolkit.tech/

....which you can run and get all traffic logged (similar like browser network tab).So with this tool you can run any browser type you want and also you can run a CMD instance called Fresh Terminal and this I did and tried the same commandlines (did CD to my folder before) above in that CMD window..... 

aria2c --check-certificate=false -j 1 -x 1 -i "Output.urls"

... 

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
fc2ff0|OK  |   2.8MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag0
f9f5e6|OK  |   1.7MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag1
97c382|OK  |   2.6MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag2
2ee936|OK  |   5.0MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag3
935bee|OK  |   6.5MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag4
f2b4ed|OK  |   6.8MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag5
e20a83|OK  |   9.8MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag6
e90f9d|OK  |    10MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag7
8e321a|OK  |    10MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag8
6025d0|OK  |   5.8MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag9
3e3141|OK  |    11MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag10
12a502|OK  |    12MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag11
e073fe|OK  |   9.0MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag12
819461|OK  |   8.5MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag13
d8cd7e|OK  |   8.6MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag14
e37bee|OK  |   8.3MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag15
bbe5df|OK  |    13MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag16
235a7d|OK  |    10MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag17
310715|OK  |    12MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag18
5d65d8|OK  |   7.5MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag19
69257f|OK  |    11MiB/s|C:/_First/ffmpeg-4.0.1-win32-static/bin 1/Output_17-02-2022_22-24-32axa.mp4.part-Frag20

....and there it works!No fail anymore and all downloads getting successfully downloaded.But why?So whats the dirrent of the started CMD tool by HTTP Toolkit?The same tool CMD getting started but I get thos diffrent results.I don't find any logic in that case and hope that maybe anyone of you has an idea and how to make the CMD work without to run it from the HTTP Toolkit.Strange is that I just get this error when using aria2c tool but not when using other tools like Curl & Co.

greetz

Link to comment
Share on other sites
kao

kao

Posted
Posted (edited)

It looks like your server is strictly limiting number of parallel connections to it. I'm guessing aria2c is not closing connection properly (or not sleeping long enough) before trying to download next file.
When you run it under the httptoolkit, httptoolkit is working as a proxy and slowing things down a bit. It would be easy to check in Wireshark packet capture.

 

Try adding "--enable-http-keep-alive=false" and see if it helps.

Or maybe you can try running a bat file like this? Put one URL in each of the  "Output.url1", "Output.url2", "Output.url3", "Output.url4" and then run: 

aria2c -j 1 -x 1 -i "Output.url1"
aria2c -j 1 -x 1 -i "Output.url2"
aria2c -j 1 -x 1 -i "Output.url3"
aria2c -j 1 -x 1 -i "Output.url4"

 

 

Edited by kao
  • Thanks 1
Link to comment
Share on other sites
LCF-AT

LCF-AT

Posted
  • Author
Posted

Hey kao,

thats the match!Thank you kao. :) The new commandline does the magic and all downloads are working at once.But what does this new commandline really mean? 

--enable-http-keep-alive [true|false]
    Enable HTTP/1.1 persistent connection. Default: true

Just using the HTTP/1.1 version instead of lower / higher version?On the other hand I also just used the commandline "-j 1" to just download only one by one and "-x 1" to use just one connection but this didn't work.In this case I just wanted to disable the accelerator.Hhmm.I also didn't found any timeout command for aria2 to set a custom timeout to stop a while (sleep) before working go on you know.

So I have one more question about that "--check-certificate=false" command. 

--check-certificate [true|false]
    Verify the peer using certificates specified in --ca-certificate option. Default: true

So what does it?Just checking the server certicficate whether its valid or so?When I disable it then its not checking = Not secure anymore = port 443 without encryption?Is it similar like sending requests just via http:80 and being naked?

greetz

Link to comment
Share on other sites
LCF-AT

LCF-AT

Posted
  • Author
Posted

Hi,

thanks for the info but this dosen't have any effect when using... 

aria2c --header="Keep-Alive: timeout=5, max=1000" --header="Connection: Keep-Alive" -j 1 -x 1 -i "Output.urls"

...just get same results.

So I have a another small question about ariac2 tool and using it to download file segments from a playlist.So when using aria2 normaly then it will download every single segment / chunk from a playlist on HDD / folder.So is there any aria2 command NOT to download every segment on disk and just to bundle a specific number of chunks / segments into one segment?

Example: In some cases I found playlists which using a segment duration of 20 second / 8 seconds / 5 seconds / 1 seconds.So the lower the second time is (1 second lowest I found) the more segments are present in the playlist and to download.In case of 1 second segments it can be many thousand of segments aria2 does download all on HDD whats really not a good idea.Now I was checking out the aria2 manual to find anything to manage that anyhow else but didn't found any command yet.

I would like to tell araia2 to download 30 segments (maybe just in memory if possible) then pause/timeout and bundle/merge them into one segment (put on HDD) and then continue the download like this till the end and merge all 30er chunk blocks into one etc.So my goal is it just to prevent tons of segments if possible.Any idea whether I can do that with aria2?

greetz

Link to comment
Share on other sites
kao

kao

Posted
Posted
On 2/18/2022 at 8:18 PM, LCF-AT said:

But what does this new commandline really mean?

It tells Aria to tell server that connection should be *closed* as soon as download is done. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Connection See "Connection: close"

Your latest experiment tries to keep connection *open* which is not what you need in this case.

 

8 hours ago, LCF-AT said:

So is there any aria2 command NOT to download every segment on disk and just to bundle a specific number of chunks / segments into one segment?

No. HTTP downloaders download stuff. Whatever server sends, they will store into a file and that's it. Any extra manipulation should be done by a different program.

 

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...