Video Transcoding and Streaming Demo
1. Video Transcoding
Video transcoding deals with converting a previously compressed video signal into another one with different format,such as different bit rate, frame rate, frame size, or even compression standard. Due to the expansion and diversity of multimedia applications and present communication infrastructure comprised of different underlying networks and protocols, there has been a growing need for inter-network multimedia communications over heterogeneous networks. For example, in transmitting a video bitstream over a heterogeneous network, the connection from the video source to the end user may be established through links of different characteristics and capacities. In this case, the bandwidth required by the compressed video was usually adjusted by the source in order to match the available capacity of the most stringent link used in the connection. If real-time video is used, this can be implemented by the encoder adjusting its coding parameters. However, the visual quality has to be sacrificed because the encoded video bit rate has to match the “weakest link”. On the other hand, when pre-encoded video needs to be distributed to users with different connections, the implementation is difficult because the target transmission channel conditions are generally unknown when the video is originally encoded. Besides the problem of channel characteristics and capacities, different end devices used in today’s communication also introduce some problems. For example, people like to use some small handheld devices, such as cellular phones, handheld computers, etc., for video communication and Internet access. Most current handheld devices only have limited computing and display capabilities, which are not suitable for high quality video decoding and displaying. In this case, precoded high quality video may need to be converted into a lower quality one for displaying on handheld devices. As the number of different video compression standards (e.g., H.261, H.263, MPEG-1, MPEG-2, MPEG-4) increases, there is a growing need for conversion between videos coded by different standards. Furthermore, in the case where a video source needs to distribute the same video stream to several clients through channels with different capacities, the encoded video stream needs to be converted to specific bit rates for each outgoing channel. This problem also occurs in multipoint video conferencing, where multiplexing of multiple video streams may exceed the capacity of the transmission channel and require a bit rate conversion.
Video transcoding is a technique dedicated to solving these problems. Video transcoding comprises the necessary operations for the conversion of a compressed video stream from one format to another one for transmission over heterogeneous networks and end devices. The device or software that performs such a conversion is called Video Transcoder. Similar to source encoders, video transcoders can modulate the data they produce by adjusting a number of parameters, including quality, frame rate, and resolution. However, using of transcoders gives us a second chance to dynamically adjust the video format according to channel bandwidth and end devices. This is particularly useful when there are time variations in the channel characteristics.
In this work, I implement a demo system, which transcodes and transmits pre-encoded video stream according to different client profile and connections. The purpose of this system is to demonstrate the concept of video transcoding. In this system, some algorithms which have been proposed in other publications of the author and implemented in the transcoder for video spatial resolution downscaling and bit rate control. For the details of the algorithm, please read the related papers, which can be downloaded from the author’s web page.
This pager serves as an introduction of how the system is implemented and operated. The video encoding and decoding components are implemented based on the public domain software for H.263. The debug version of the whole system can be downloaded following the link at the end of this page. More details about video transcoding techniques can be found from a survey maintained by the author.
2. Concept Model and Features
Figure 1. Basic Modules of the Video Transcoding System
The concept modules and features of a video transcoding system are illustrated in Figure 1. In this system, pre-encoded, high quality, high bit-rate videos are stored at the video source. At the other side, different user clients maintain the clients profile, which includes the following parts:
1. Transmission Profile: transmission profile is responsible for monitoring the dynamic condition of the transmission channel, such as effective channel bandwidth, channel error rate, etc.
2. Device Profile: Device profile describes the capability of the device, such as screen size, processing power, etc.
3. User Profile: User profile describes the user preference.
The transcoding gateway can be implemented on the server where the pre-encoded video are stored, or on an intermediate node along the transmission path. The basic functions of the transcoding gateway includes:
1. Frame Size Conversion: change the frame size according to the client profile. Such as most handheld devices can only display small size video (QCIF, 176x144 pixel). When the pro-encoded video is in big video size, it needs to be transcoded to small frame size.
2. Bit Rate Adaptation: convert the video bit rate according to the channel conditions. Since the pre-encoded video is encoded at high quality and bit rate. For low bandwidth connections, the video bit rate needs to be converted to low bit rate.
3. Frame Rate Conversion: change the video frame rate. Since some handheld devices can only play video at a low frame rate, such as 5, or 10 fps (frames per second), the high quality video (30 fps) needs to be transcoded to low frame rate one. Frame rate conversion can also reduce the bit rate.
4. Error Resilience: change the error resilience of the video for unreliable channels. When the transmission channel is in high error rate, more error resilience features can be but into the video stream to protect it against channel errors.
5. Coding Syntax Conversion: change the compression standards, such as from MPEG1 to H.263, etc.
When the connection between the client and transcoding gateway is created, the client will transmit it profile to the transcoding gateway. Based on some negotiations, the transcoder will adjust the transcoding parameters adaptively.
Based on this concept module, I implemented the demo system. In the implemented system, the transcoding gateway is implemented together with the streaming server, where the pre-encoded videos are stored. The frame size conversion (downscale by 2, from 4cif to cif, and cif to qcif), frame rate conversion (from 30 fps to 30, 25, 20, 15, 10, 5 fps), and bit rate adaptation modules are implemented. Moreover, I also implemented the feature of color-to-grayscale conversion module, which will convert a color video into black-and-white video. The clients connect to the server through TCP/IP. The TCP is used as the transport protocol considering its reliability. The clients based on desktop and handheld devices (iPaq with WinCE PocketPC 2002) are implemented for decoding and displaying videos.
3. Server Design
3.1 System Diagram
3.2 Class Diagram
4. Client Design
4.1 Class Diagram
Figure 2. Server Screen Shot when 2 pre-encoded video are transcoded and streamed
Figure 2 shows a server screen shot when two videos are requested and streamed to two clients. At the lower left corner, a list shows information for all pre-encoded video on the server. The upper left part lists all information about every session, which includes the client IP address, socket ID, video that is requested, current frame number and frame bit rate. By pressing the “Transcoding” button, the transcoding parameters will be displayed. By pressing the “Channel” and “Error Control” buttons, the different channels can be simulated and different error control parameters can be applied. Please note that this part of function has not been implemented yet. In the session statistics window, the frame bit rate and PSNR will be plotted in two charts. For very session, a page will be inserted into the session statistics window. At the right side, the requested pre-encoded vide will be decoded and displayed.
Figure 3. Desktop Client decoding and playing a video stream
Figure 3 shows the screen shot of a desktop client displaying its requested video, which is in the size of CIF.
Figure 4. Handheld client decoding and playing a video stream
Figure 4 shows the screen shots of the client on iPaq. By changing the client’s profile, the requested video can be transcoded and displayed into different formats, such as different frame rate, frame size, color or grayscale.
Figure 5. Handheld Client Menu and Setting
Figure 5 shows the user interface on the iPaq client. The left picture is the menu of the client. By selecting “Connect…”, user will be able to connect to the server. The server IP address will be asked. After that, all the video on the server will be listed in the list box at the bottom of the window. By pressing “Setting…”, user will be able to set up the client profile, which includes frame size, color, frame rate, and channel bit rate as illustrated in the right picture. After setting the profile, by selecting a video from the list box and pressing “Play” the pre-encoded video will be transcoded and transmitted to the client. Client will decode and display the transcoded video in its window.
6. Software Download and Test
Executable and some video test sequences (server.zip)
6.2 Client on Desktop and iPaq
For the client on iPaq, please copy the profile.txt under the “My Documents” directory.
Since I created this page, I’ve got a lot of emails asking about the Pocket Client on iPaq. There might be some difficulties when you want to directly run it on your handheld, because of the compiling environment and configurations, etc. Therefore, I decide to put the source code for the PC client and Pocket Client here. You can download and build them. For Pocket Client, I used Microsoft eMbedded Visual C++ 3.0 and Microsoft Windows Platform SDK for Pocket PC 2002. You may need to get them first. When you build the application, you need to specify the target platform (ARM or x86) according to your handheld.
6.3 User Manual (OPManual.pdf)
6.4 Any bug report or comments are highly appreciated. Please feel free to send your comments to me.
· Public domain version of the H.263+ coder from UBC. http://spmg.ece.ubc.ca/h263plus/h263plus.html
· A Multithreaded Telnet Server - Chess Game Example developed by Lee Patterson from Code Guru, http://www.codeguru.com/network/ChessServer.shtml
· A Multiline Chart Control developed by Yuhe Zhao from Code Guru, http://www.codeguru.com/controls/multi_line_chart.shtml
· Playing an AVI file in the status bar, developed by Robin J Leatherbarrow from Code Guru, http://www.codeguru.com/statusbar/avi_statusbar.shtml
· DIBSectionLite, a DIBSection wrapper for Win32 and WinCE, developed by Chris Maunder from Code Project, http://www.codeproject.com/bitmap/dibsection.asp
· A Rollup Control developed by Johann Nadalutti from Code Project, http://www.codeproject.com/miscctrl/rollupctrl.asp