The following information represents the comprehensive notes from the ‘’Video Communication Course 2022 (Part1)’’. These notes provide the fundamental concepts, principles, and practical applications covered in the course. They serve as a valuable resource for students and professionals seeking to enhance their knowledge and skills in the field of video communication.
Part1 contains the following sections
- Temporal 時間 / Spatial 空間
- Aliasing 混疊
- Face Detection 臉部偵測
- Resilience 預防 / Concealment 隱瞞
- De-blocking Filter
- Digital Media Concepts
- Video in Demand
- Variable-Rate Transmission on Demand
Video Communication Flowchart
Temporal 時間 / Spatial 空間
- Temporal Coherence (時間同調)
- Spatial Coherence (空間同調)
If bandwidth is limited, more bandwidth needs more money. “Computation cost” is cheaper than “communication cost”.
- Computation cost (hardware cost, 硬體計算成本)
- Communication cost (transmission cost, 傳輸成本 e.g. 手機月租費)
How to reduce the usage of bandwidth (communication cost)?
- Temporal reduction
Use less frame within a second to reduce the usage of bandwidth. In other words, in the same unit of time (單位時間) we send less information.
e.g. Original video has 30 fps, we reduce it to 10 fps, but the image quality still the same as original like 1024*1024. - Spatial reduction
Compress the original video, it lower the usage of bandwidth but it also result in lower the video quality.
e.g. Original image size 30*30, we compress the video which also lead to shrink the image size to 10*10, then at encoder side we use 10*10 size to transfer, at decoder side we use another method (e.g. super-resolution) to recover it to original image size 30*30. The frame rate from encoder/decoder side are the same like 30 fps.
Aliasing 混疊
When we downsample an image (without any pre-processing), and then upsampling the image, this movement will cause “aliasing”.
How to check whether the video has been edit?
Add a random walking guy in the original video. We can check out the “shadow” from each object and determine which one is different from the others. Because shadow is hard to add into a video. You need to consider the light (optical) condition…etc, in order to let the shadow looks real in the original video, which is quite difficult to take everything in consideration.
Face detection 臉部偵測
When we’re on a video call, by using face detection we can use “face detection” to preserve more detail on face (use more bit to describe face). By doing so, when the bandwidth is limited, we use more bit to describe face (focus on the face, the face region will be more clear). Then we don’t have to waste data (bit) on the other region.
視訊的時候可以focus在臉上，臉的地方可以有更多detail (more detail, needs more bit)，當bandwidth是有限的時候，不用浪費data在其他臉以外的地方。
Resilience 預防 / Concealment 隱瞞
- error resilience 錯誤預防, error not happened yet, pre-processing
- error concealment 錯誤隱瞞, error already happened, post-processing
What’s the method to reduce below effect?
jittering, shaking, interlacing, blocking
- By using PDE-based method,
which can correct the displacement errors.
For example, “Bake & Shake”. (PDE — Partial Differential Equations) - By using Progressive scan,
which analyzing the two frames and deinterlace only parts are needed to. Result in high temporal and vertical resolution, which achieve smooth motions. For more… - By using Deblocking Filter,
which reduces the blocking artifacts caused by block-based encoding with strong quantization. It is applied by modifying samples along horizontal and vertical boundaries.
De-blocking Filter
If we implement de-blocking filter to image, it may also blurring the real edge from the image. There might be some “real edge” also be blurred at the same time. We can not distinguish the block is “real edge” (that we want to reserve) or “blocking effect” (that we want to remove).
Relative techniques:
- video synthesis 影片合成
- video matting 影片去背
- video dynamosaics (mosaics 馬賽克)
- Three-dimensional Waveform Coding
Subband coding 次頻帶編碼
Wavelet coding 小波編碼 - Motion-compensated Predictive Coding
DPCM (differential pulse code modulation 誤差訊號編碼)
It is a procedure of converting an analog into a digital signal in which an analog signal is sampled and then the difference between the actual sample value and its predicted value. For more… - OBMC (Overlapped block motion compensation 重疊分塊運動補償)
For more… - Model-based Coding
- Object-based coding
- Knowledge-based coding
- Video transcoding 影片轉碼
Spatial transcoding 空間轉碼
Temporal transcoding 時間轉碼
Digital Media Concepts
- LD (Low Definition)
320x240 pixels or 480x272 pixels(144p, 240p, or 360p)
e.g. VHS videotape - SD (Standard Definition)
640x480 pixels(480p)
e.g. DVD - HD (High Definition)
1280x720 pixels or 1920x1080 pixels(720p, 1080i, or 1080p)
e.g. High Definition TV - 4K (4K Ultra High Definition)
3840x2160 pixels or 4096x2160 pixels(4K)
e.g. 4KUHDTV - 8K (8K Ultra High Definition)
7680x4320 pixels(8K)
Video in Demand
Which means you can choose what you want to watch.
你可以選擇你自己想看什麼
e.g. Netflix
There are 100 movies in the database, and now there are 100 people who want to watch the movies they want to watch. Different algorithms may have different effects (running or lagging)
database中有100部電影，同時在現有100人要觀看各自想看的電影，不同的algorithm可能會有不同的效能(順跑 or LAG)
Variable-Rate Transmission on Demand
With digitalize, we can change our bit-rate to lower or higher to fit the bandwidth. Because the bandwidth has its limitation.
With analog, we can not change our bit-rate. It’s difficult for us to change the bit-rate, because it doesn’t have the flexibility like digital data.
因為資料可以數位化，我們可以改變我們的bit-rate降低或升高去符合頻寬，因為頻寬可能有限制。
如果資料是類比的形式，那我們很難去改變他的bit-rate，因為可能需要透過一些手段從物理層面去改變他。
Advantage of digitalize (數位化的優勢)
Easy to change data between different encoder like MPEG-1/MPEG-2.
容易在不同格式之間轉換，如MPEG-1/MPEG-2
Disadvantage of digitalize (數位化的缺點)
Data is large, it need space to store the information in somewhere, for example like hard disk (HD).
資料很龐大，他需要有地方可以儲存他，例如硬碟
Thank you for taking the time to read this article, and I sincerely hope that the information provided proves to be valuable to you. Whether you are a student, professional, or simply someone interested in video communication, it is my utmost wish that these notes enhance your understanding and contribute to your success in this field. Thank you once again, and best of luck on your journey in the world of video communication!
