Learning MediaCapture C# API for Webcam

If you have a USB camera, how can you build a simple C# camera application on Windows 10? There are three options: WIA (Windows Imaging Acquisition),DirectShow and MediaCapture. After trying some sample code that downloaded from CodeProject and GitHub, I got the conclusion: 1. WIA is not good because it does not support my webcam. 2. DirectShow can work well, but there is no C# API provided by Microsoft. You need to create a wrapper for C++ API. 3. MediaCaptureclass that designed for UWP apps provides C# APIs which provide low-level control over the capture pipeline and enable advanced capture scenarios. In this article, I want to share the source code of how to handle every preview frame that I learned from the sample provided by Microsoft.

Getting Preview Frames via C# Camera API

Although the reference page of MediaCapture class is informative, to quickly learn APIs, I prefer to get started with UWP sample code that provided by Microsoft. PressCtrl+F to search for camera, you will see following samples:

If you want to add custom effects or functionalities (OCR, barcode etc.) for the camera, a common way is to register a callback function for receiving every preview frame. After running all samples, I found the project CameraFrames is what I want.

How to build a UWP webcam app in C#

  1. To use webcam, double-click package.appxmanifest to check the corresponding capability:

2. Create an image element for rendering preview frames:

3. Initialize FrameRenderer with the image element:

_frameRenderer = new FrameRenderer(PreviewImage);

4. Initialize the media capture object:

// Create a new media capture object.
_mediaCapture = new MediaCapture();
var settings = new MediaCaptureInitializationSettings()
{
// Select the source we will be reading from.
SourceGroup = groupModel.SourceGroup,
// This media capture has exclusive control of the source.
SharingMode = MediaCaptureSharingMode.ExclusiveControl,
// Set to CPU to ensure frames always contain CPU SoftwareBitmap images,
// instead of preferring GPU D3DSurface images.
MemoryPreference = MediaCaptureMemoryPreference.Cpu,
// Capture only video. Audio device will not be initialized.
StreamingCaptureMode = StreamingCaptureMode.Video,
};
try
{
// Initialize MediaCapture with the specified group.
// This can raise an exception if the source no longer exists,
// or if the source could not be initialized.
await _mediaCapture.InitializeAsync(settings);
_logger.Log($”Successfully initialized MediaCapture for {groupModel.DisplayName}”);
}
catch (Exception exception)
{
_logger.Log(exception.Message);
DisposeMediaCapture();
}

5. Use DeviceWatcher to list all connected devices:

var deviceSelector = MediaFrameSourceGroup.GetDeviceSelector();
_watcher = DeviceInformation.CreateWatcher(deviceSelector);
_watcher.Added += Watcher_Added;
_watcher.Removed += Watcher_Removed;
_watcher.Updated += Watcher_Updated;
_watcher.Start();
private async void Watcher_Added(DeviceWatcher sender, DeviceInformation args)
{
await AddDeviceAsync(args.Id);
}
private async Task AddDeviceAsync(string id)
{
var group = await MediaFrameSourceGroup.FromIdAsync(id);
if (group != null)
{
await _dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
{
_sourceCollection.Add(new FrameSourceGroupModel(group));
});
}
}

6. Updates the current frame source to the one corresponding to the user’s selection:

_mediaCapture.FrameSources.TryGetValue(info.SourceInfo.Id, out _source);

7. Initialize MediaFrameReader and register a callback function:

if (_source != null)
{
_reader = await _mediaCapture.CreateFrameReaderAsync(_source);
_reader.FrameArrived += Reader_FrameArrived;
}

8. Start reading webcam frames:

MediaFrameReaderStartStatus result = await _reader.StartAsync();

9. Receive all frames and render them on image element:

private void Reader_FrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
{
// TryAcquireLatestFrame will return the latest frame that has not yet been acquired.
// This can return null if there is no such frame, or if the reader is not in the
// “Started” state. The latter can occur if a FrameArrived event was in flight
// when the reader was stopped.
using (var frame = sender.TryAcquireLatestFrame())
{
_frameRenderer.ProcessFrame(frame);
}
}
public void ProcessFrame(MediaFrameReference frame)
{
var softwareBitmap = FrameRenderer.ConvertToDisplayableImage(frame?.VideoMediaFrame);
if (softwareBitmap != null)
{
// Swap the processed frame to _backBuffer and trigger UI thread to render it
softwareBitmap = Interlocked.Exchange(ref _backBuffer, softwareBitmap);
// UI thread always reset _backBuffer before using it. Unused bitmap should be disposed.
softwareBitmap?.Dispose();
// Changes to xaml ImageElement must happen in UI thread through Dispatcher
var task = _imageElement.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
async () =>
{
// Don’t let two copies of this task run at the same time.
if (_taskRunning)
{
return;
}
_taskRunning = true;
// Keep draining frames from the backbuffer until the backbuffer is empty.
SoftwareBitmap latestBitmap;
while ((latestBitmap = Interlocked.Exchange(ref _backBuffer, null)) != null)
{
var imageSource = (SoftwareBitmapSource)_imageElement.Source;
await imageSource.SetBitmapAsync(latestBitmap);
latestBitmap.Dispose();
}
_taskRunning = false;
});
}
}

Reference

Basic photo, video, and audio capture with MediaCapture

Source Code

https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CameraFrames

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.