Recently, We were working on a project where we need to show a video with multiple audios and captions associated with it. And the user must be able to change the audio and captions at run-time.

We used HTML 5 <video> tag to display videos. We were passing video URL in source tag and all the caption files in the track tag and audio files into an audio tag.

It was working fine for the captions but audios were not working as Chrome and Firefox does not support this feature yet.

After some digging over the internet we found out Apple’s HLS is such an option where we can use multiple audios.

HTTP Live Streaming (also known as HLS) is an HTTP-based media streaming communications protocol implemented by Apple Inc. It resembles MPEG-DASH in that it works by breaking the overall stream into a sequence of small HTTP-based file downloads, each download loading one short chunk of an overall potentially unbounded transport stream.

We used HLS with videojs to display our videos and audio and caption track switching. We needed to convert our videos, audios into HLS stream. We used FFmpeg for that purpose.

ffmpeg is a very fast video and audio converter that can also grab from a live audio/video source. It can also convert between arbitrary sample rates and resize video on the fly with a high quality polyphase filter.

current_path – is path of our input file specified by the -i option.

-profile:v – specifies that input file is a video file, use “-profile:a” to input audio file.

-s – option is used to define stream size and bit rate.

-hls_time – set the target segment length in seconds. Default value is 2. Segment will be cut on the next key frame after this time has passed.

-hls_list_size – set the maximum number of playlist entries. If set to 0 the list file will contain all the segments. The default value is 5.

start_number – start the playlist sequence number from the specified number. The default value is 0.

out_path – is where we stored our playlist and all segment file.

The above piece of code will create a playlist with extension .m3u8 and many segment files having extension .ts. These segment files are mapped into a playlist file.

Now, by using this stream we can display our video. But our task is not done yet.

Here, we will upload an external audio track to video by following the same process of FFmpeg as we followed for video transcoding.

After audio file transcoding, we have another playlist file with many segment files. We created a main playlist file where we mapped both video and audio playlist files.

We can add any number of audio playlist and captions to HLS playlist. Though you can use caption directly with video tag by using <track>.

When we add an external audio stream with a video stream, original audio or video cannot be used. You can use the original audio stream by removing mapping of an external audio stream from the main playlist file.

Now, we created our main playlist file. To display our video first we need to create a video tag in our html file in the following way:

We were working on a Django project. Here we created as many track attribute as caption files.

In order to use video js and create audio switching and transcripting with captions see the below code:

We created a player here of our <video> tag.

audioTrackList defines the number of audio track our playlist contains.

textTrackList defines the number of captions our playlist contains.

We had a task where we need to display captions beside the video player and when user click on a caption, the player should play from there. We have done this using the following piece of code.

seek_track() takes the input and set the current time of the player to the input.

In above code track is “current caption track”. You can see all the captions using cues property.

track.cues_[i].startTime will return time of a single caption when it will be visible on our player.

track.cues_[i].text will return caption text. When we click on a caption cue it calls seek_track() and pass startTime as an argument and our player starts playing video from there.

Thanks for reading!.

Click here for more details…

At BoTree Technologies, we build enterprise applications with our Django team of 20+ engineers.

We also specialize in RPA, AI, Python, Ruby on Rails, JavaScript and ReactJS.

Consulting is free – let us help you grow!