How to handle HLS?

[LCF-AT]

Hi guys,

at the moment I try to study a little bit how HLS streamings are working and wanna ask whether anyone of you has some knowledge and experience with that theme.I found some sites (mosty any IPTV) where you can find listet links of TV channel playlists in m3u8 format.Normaly I can use commandline tools like ffmpeg / livestreamer to play / download them but now I wanna know how to handle such links manually to play / download them.After a manually check of a playlist file I can find inside some informations and another links to other m3u8 files using diffrent quallity etc.As next I did download one of them and can find inside another infos about splitet ts file segments maybe 30 parts for example which I could also download manually.At the end I have many little video chunks which I can also play alone what means that they all have their own video headers and at last step I have to mux them all together to one video file with any tool.So I think I am right so far but to handle this processes manually is a little much work.Maybe I got it working if I try it to code something but at the end I would have maybe tons of chunk files and to put them together ... I cant do manually.As next I thought it would be better to use any libs / APIs who could do this for me similar as librtmp but there I just found some build tutorials for ffmpeg and languages C++ etc which are looking too complex for me etc you know.Of course I could use ffmpeg itself and execute it with CreateProcess API and commands but then I need to use the entire ffmpeg set with all files which are very big etc + commandline what is not so my thng.Just wanna ask whether you got some ideas / hints for me what I could do and use.Maybe there are also something more simple things to get I could use and dont it yet.

Thank you

[Extreme Coders]

Been dealing with hls recently while developing the ColorsDesktop app.

You can play hls streams directly in browser. The bitrate is selectable too. See http://www.flashls.org/latest/examples/chromeless/

With HLS you get a master a file (like index.m3u8).
Example of a master file:

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=81920,RESOLUTION=128x64
http://redacted.redacted.com/enc/fhls/p/1982551/sp/198255100/serveFlavor/entryId/0_xwca3s0a/v/2/pv/1/flavorId/0_ndg7mptw/name/a.mp4/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=332800,RESOLUTION=640x360
http://redacted.redacted.com/enc/fhls/p/1982551/sp/198255100/serveFlavor/entryId/0_xwca3s0a/v/2/pv/1/flavorId/0_4kypntem/name/a.mp4/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=480256,RESOLUTION=640x360
http:/redacted.redacted.com/enc/fhls/p/1982551/sp/198255100/serveFlavor/entryId/0_xwca3s0a/v/2/pv/1/flavorId/0_5xxo1ymt/name/a.mp4/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=685056,RESOLUTION=640x360
http://redacted.redacted.com/enc/fhls/p/1982551/sp/198255100/serveFlavor/entryId/0_xwca3s0a/v/2/pv/1/flavorId/0_9pbef17r/name/a.mp4/index.m3u8

The master file file in turn contains a list of links to other m3u8 files. Each such file list represents a particular bitrate.
Example of such a file

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-ALLOW-CACHE:YES
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-KEY:METHOD=AES-128,URI="encryption.key"
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:2.000,
seg-1-v1-a1.ts
#EXTINF:2.000,
seg-2-v1-a1.ts
#EXTINF:2.000,
seg-3-v1-a1.ts
#EXTINF:4.000,
seg-4-v1-a1.ts
#EXTINF:10.000,
#EXT-X-ENDLIST

.The video is divided up into segments numbered as seg-1-v1-a1-ts and so on. Additionally, each of the segments are encrypted with AES, so you would need to decrypt them too. Do all of this tasks is a lot of effort. You can directly use ffmpeg for this:

1. Run ffmpeg with the root m3u8 url to get a list of program id's

C:\>ffmpeg -i http://content.jwplatform.com/manifests/vM7nH0Kl.m3u8

Input #0, hls,applehttp, from 'http://content.jwplatform.com/manifests/vM7nH0Kl.
m3u8':
  Duration: 00:12:14.00, start: 0.000000, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 380000
    Stream #0:0: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 320x142 [SAR 852:857 DAR 1920:857], 24 fps, 24 tbr, 90k tbn, 48 tbc
    Metadata:
      variant_bitrate : 380000
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 107 kb/s
    Metadata:
      variant_bitrate : 380000
  Program 1
    Metadata:
      variant_bitrate : 3400000
    Stream #0:2: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1920x858[SAR 858:857 DAR 1920:857], 24 fps, 24 tbr, 90k tbn, 48 tbc
    Metadata:
      variant_bitrate : 3400000
    Stream #0:3: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 107 kb/s
    Metadata:
      variant_bitrate : 3400000
  Program 2
    Metadata:
      variant_bitrate : 1710000
    Stream #0:4: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1280x572[SAR 858:857 DAR 1920:857], 24 fps, 24 tbr, 90k tbn, 48 tbc
    Metadata:
      variant_bitrate : 1710000
    Stream #0:5: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 107 kb/s
    Metadata:
      variant_bitrate : 1710000
  Program 3
    Metadata:
      variant_bitrate : 670000
    Stream #0:6: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 640x286 [SAR 858:857 DAR 1920:857], 24 fps, 24 tbr, 90k tbn, 48 tbc
    Metadata:
      variant_bitrate : 670000
    Stream #0:7: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 107 kb/s
    Metadata:
      variant_bitrate : 670000
  Program 4
    Metadata:
      variant_bitrate : 120000
    Stream #0:8: Audio: aac (LC), 44100 Hz, stereo, fltp, 133 kb/s
    Metadata:
      variant_bitrate : 120000

2. Now using the program id, download the streams.

ffmpeg -i "<m3u8 url>" -map 0:p:<program id> -codec copy video.ts

C:\>ffmpeg -i http://content.jwplatform.com/manifests/vM7nH0Kl.m3u8 -map 0:p:3 -codec copy video.ts

That would automatically download, decrypt if necessary and assemble into a single ts file.

Edited August 29, 2016 by Extreme Coders

[LCF-AT]

Hi and thanks for the answer so far.

All clear so far but my question was more going into a direction without using the ready ffmpeg commandline tool itself and using some kind of lib / dll of ffmpeg maybe to handle all the steps in my app source and just using APIs / structs you know.Do you have some infos about that?

greetz

[Extreme Coders]

It is possible to use ffmpeg as a dll in a C/C++/C# ... application. 

Some links scoured from google:
http://www.codeproject.com/Tips/111468/FFmpeg-Tutorial
https://github.com/mpenkov/ffmpeg-tutorial

If possible, I will try to come up with some example code.

[LCF-AT]

Hi,

thanks for the links but they are nothing so far so maybe you find anything else later etc.

One more question about ffmpeg commands.Lets say I found a playlist I wanna play / DL and it has also diffrent ids / quality etc inside so is there are also a command to let choose ffmpeg to play the best or lowest stream without to declair manually if I use the master playlist link?I mean similar as for livestreamer where you just use "best" command you know.

greetz

[Techlord]

Hi @LCF-AT : Could you please give an example of a web site or link so that I can check and THEN post here IF my method is working ?

Lately, I have been also trying to download movies etc which play in MULTIPLE SEGMENTS and have been successful.

But I want to be sure my method works on the site (example site) which you give an example of, and if my method works will post here

Greetz

[LCF-AT]

Hi Techlord,

http://www.daserste.de/live/index.html

http://live-lh.daserste.de/i/daserste_de@91204/master.m3u8
----------------------------------------------------------------------
http://live-lh.daserste.de/i/daserste_de@91204/index_184_av-p.m3u8
http://live-lh.daserste.de/i/daserste_de@91204/index_2692_av-p.m3u8


Play & Pipe
ffmpeg -i http://live-lh.daserste.de/i/daserste_de@91204/master.m3u8 -c copy -f mpegts - | C:\Programme\VideoLAN\VLC\vlc.exe -
ffmpeg -i http://live-lh.daserste.de/i/daserste_de@91204/index_2692_av-p.m3u8 -c copy -f mpegts - | C:\Programme\VideoLAN\VLC\vlc.exe -

ffplay -i http://live-lh.daserste.de/i/daserste_de@91204/master.m3u8
ffplay -i http://live-lh.daserste.de/i/daserste_de@91204/index_2692_av-p.m3u8

Above a example site.If you play the master playlist with ffmpeg then it will start the best stream and with ffplay it will run the lowest stream.In this case I just wanna know which paramter / command I can use to force ffmepg / ffplay to play the best or lowest stream of a master playlist.Just dont want any kind of random choice etc.

greetz

[Techlord]

Hi @LCF-AT :

I have good news for you.

Using the method that I state here, you can :

1. Download the video to your hard disk in ONE click.

2. Choose the QUALITY of the video to download in that same click.

3. It does NOT  record from the screen. Meaning that if its a one hour video, it does NOT take one hour to save to disk. It only takes a FEW MINUTES to save to disk. That means its DEFINITELY NOT recording from the screen.

4, It has worked for me when I tried to download from the example site you gave above.

I downloaded the file named " Das Erste LIVE - Livestream – Erstes Deutsches Fernsehen - A.mp4"  successfully (No idea what that video means though, as its  in German )

5. Its all FULLY AUTOMATED. No need to download bit by bit and then manually join them.

Method :

1. Use any Firefox Browser version newer than v36. I used Firefox v45 in my screenshot below.

2. Go here and download the Video Downloadhelper v6 addon for the Firefox. Please note that the add-on version 6 has newer features and the OLD VERSIONS DO NOT WORK for this method (you need v5.2 or later of the add-on, to be exact) .

3. Install the add-on in the Firefox browser.

4. Navigate to the website having HLS streaming. I used the above example link that you gave.

5. Play the video.

6. Once the video starts playing, you'll see the tri-colour orb on the top-right corner light up. Click on it.

7. You will see the drop-down box containing the choices for the quality of the vido etc. Choose one of them and then choose where to save on your hard-disk. You can see that I chose the 2.7 Mbps version (BEST)  of the stream to download ...

8. Please see screenshot below that I have been successful with ! If the blue circle (orb) lights up in the dropdown box, that means that the download is in progress !

By the way @LCF-AT , it is my HABIT to give detailed instructions , just to avoid confusion. Please do not take offense

I know that you are very intelligent. Still I wanted to give the steps fully.

[/IMG]

NOTES :

1. It took a couple of tries before I was successful. SO please try it a few times if it does not download rightaway. The "2" in the RED CIRCLE (orb) in the dropdown box shows the errors that I had at first. THEN it started downloading the rather HUGE video file .

2. If any COMMERCIALS (advertisements) are playing before you reach the MAIN VIDEO, please WAIT till the MAIN VIDEO starts. Otherwise only the commercials would get downloaded and it'll crash

Good luck and HAPPY WATCHING !!!

Please click LIKE if you find this useful ...

 

Edited September 1, 2016 by Techlord

[LCF-AT]

Hi Techlord,

thanks for your description and your effort so far but I think you did understand me wrong.My main goal was it to find any kind of lib or dll of ffmpeg or else I could use for my own build apps to play / download HLS streams.If there are nothing like this I could use then I have to use ffmpeg cmd itself you know.If so then I was looking for a ffmpeg command to choose the best or lowest stream in a playlist automatically like I also can do with livestreamer cmd tool without to check the playlist again and to set a stream ID xy.

greetz

[Techlord]

On Monday, 29 August, 2016 at 11:15 AM, LCF-AT said:

Normaly I can use commandline tools like ffmpeg / livestreamer to play / download them but now I wanna know how to handle such links manually to play / download them

Oh , @LCF-AT : Sad to hear that this is not what you wanted...

I had got the impression that you wanted an AUTOMATED way to download and MUX all the tiny segments of a large streaming file on your computer, for OFFLINE viewing, without having to use any COMMAND-LINE utilities... Guess I got it wrong.

Greetz

 

Edited September 2, 2016 by Techlord