Easy Win: Adding Inking Support to a UWP Application

Published in

Windows Developer

7 min readNov 22, 2017

Ink Support During Text Input

Text input is the simplest and most basic form of ink support in an application. This scenario is handled by default via the Handwriting Input Panel in Windows itself. This operating system feature is enabled by default for all PCs that have pen hardware installed, and needs no custom development to support it.

The Handwriting Input Panel has evolved over time, and today is an excellent tool for basic data input on tablet PCs.

Getting Started with the InkCanvas Control

If your application needs more than just simple handwriting recognition and text input, developers have several options, the easiest to implement being the InkCanvas control. InkCanvas is a Universal Windows Platform (UWP) visual control that captures and displays ink strokes.

In a XAML-based UWP application, you only need to add an <InkCanvas> element to the application’s visual tree:

<Pagex:Class=”UwpApp1.MainPage”xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation"xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml"xmlns:local=”using:UwpApp1"xmlns:d=”http://schemas.microsoft.com/expression/blend/2008"xmlns:mc=”http://schemas.openxmlformats.org/markup-compatibility/2006"mc:Ignorable=”d”><Grid Background=”{ThemeResource ApplicationPageBackgroundThemeBrush}”><InkCanvas x:Name=”canvas”HorizontalAlignment=”Stretch”VerticalAlignment=”Stretch” /></Grid></Page>

While this is enough to capture and display ink strokes, by default it will only capture strokes made by a pen device. For most situations, a best practice is to also capture strokes made by a mouse or the user’s finger. Before the InkCanvas begins to capture ink strokes created by a mouse and/or touch gesture, the associated InkPresenter needs to be configured to recognize those input device types (the InkPresenter class is a critical component used internally by InkCanvas):

canvas.InkPresenter.InputDeviceTypes =CoreInputDeviceTypes.Mouse |CoreInputDeviceTypes.Pen |CoreInputDeviceTypes.Touch;

Developers can control the styling of the ink strokes themselves; for example, a wide-tipped marker can be simulated by designating the pen color, shape and size:

var attr = canvas.InkPresenter.CopyDefaultDrawingAttributes();attr.Color = Colors.Red;attr.Size = new Size(4, 18);canvas.InkPresenter.UpdateDefaultDrawingAttributes(attr);

A natural next step would likely be to allow the user to select the pen color. One way to implement that functionality could be a simple ComboBox, filled with colors, placed near the inking canvas:

<Grid Background=”{ThemeResource ApplicationPageBackgroundThemeBrush}”><Grid.RowDefinitions><RowDefinition Height=”Auto” /><RowDefinition Height=”*” /></Grid.RowDefinitions><ComboBox x:Name=”colorSelector”><ComboBoxItem Content=”Red” /></ComboBox><InkCanvas x:Name=”canvas”Grid.Row=”1" /></Grid>

The InkPresenter can be changed to use the appropriate color by handling the SelectionChanged event of the ComboBox:

colorSelector.Items.Clear();colorSelector.Items.Add(new ComboBoxItem { Content = “Red”, Tag = Colors.Red });colorSelector.Items.Add(new ComboBoxItem { Content = “Green”, Tag = Colors.Green });colorSelector.Items.Add(new ComboBoxItem { Content = “Blue”, Tag = Colors.Blue });colorSelector.Items.Add(new ComboBoxItem { Content = “Black”, Tag = Colors.Black });colorSelector.SelectionChanged += ColorChanged;colorSelector.SelectedIndex = 0;private void ColorChanged(object sender, SelectionChangedEventArgs e){var color = (Color) ((ComboBoxItem) colorSelector.SelectedItem).Tag;var attr = canvas.InkPresenter.CopyDefaultDrawingAttributes();attr.Color = color;canvas.InkPresenter.UpdateDefaultDrawingAttributes(attr);}

The InkPresenter — Behind the Curtains

The InkCanvas is only responsible for providing a region of UI to capture input events and render processed ink strokes, but it does not perform that processing. The processing of ink strokes (converting pen characteristics, coordinates, pressure, direction, speed and angle into electronic ink) is delegated to InkPresenter, which has no UI of its own. This arrangement may seem strange at first, but it offers the greatest degree of flexibility since it makes highly customized ink experiences possible.

Managing an Inking Workspace with the InkToolbar Control

The InkToolbar encapsulates most of the basic inking features a user would expect in an ink-enabled app. While it is possible to write the code for every tool needed in an inking workspace, the InkToolbar provides consistent UI for all inking tools and provides an easy mechanism for controlling all pen characteristics, not just color.

The InkToolbar control works in conjunction with InkCanvas controls; it provides little value unless attached to a target InkCanvas. Once attached to an InkCanvas, the InkToolbar provides quite a bit of functionality by default:

<Grid Background=”{ThemeResource ApplicationPageBackgroundThemeBrush}”><InkCanvas x:Name=”canvas”/><InkToolbar TargetInkCanvas=”{x:Bind canvas}” VerticalAlignment=”Top”/></Grid>

By default, the InkToolbar includes three different types of pens — ballpoint, pencil and highlighter — as well as an eraser and a ruler/protractor tool. These can be removed from the toolbar if needed, or placed in a customized order. Additional custom tools can be created, to support special pen shapes or devices that your app requires.

For example, it is simple to implement a 5-second self-destruct timer that switches on and off by adding a InkToolbarCustomToggleButton tool. This would have been immensely helpful to me as a young student who liked to doodle in class, but was always caught by the teacher when doing so.

The XAML to define a custom toolbar button is straightforward. Note that the default inking tools are still in place; our custom tool button is appended to the existing tools:

<InkToolbar TargetInkCanvas=”{x:Bind canvas}”VerticalAlignment=”Top”><InkToolbarCustomToggleButtonx:Name=”selfDestruct”ToolTipService.ToolTip=”Message Self-Destruction”><SymbolIcon Symbol=”ReportHacked”/></InkToolbarCustomToggleButton></InkToolbar>

The toolbar understands this is a button that should be toggled on or off when clicked, and we only need to write the code that implements the tool’s functionality:

var timer = new DispatcherTimer();timer.Interval = TimeSpan.FromSeconds(5);timer.Tick += (s, a) => {timer.Stop();canvas.InkPresenter.StrokeContainer.Clear();};canvas.InkPresenter.StrokesCollected += (s, a) => {if (selfDestruct.IsChecked.Value) {timer.Stop();timer.Start();}};selfDestruct.Click += (s, a) => {if (selfDestruct.IsChecked.Value){timer.Start();}else{timer.Stop();}};

Saving and Loading Collected Ink Strokes

Inking is great, but ultimately if an application is collecting ink strokes then it probably needs to save and/or load them from persistent storage as well. The self-destruct example above references the StrokeContainer object, which is the key to saving and loading ink stroke data. As the name implies, this object acts as an in-memory container for the ink strokes that have been collected so far. Since this is an in-memory buffer, calling the Clear() method is a quick and easy way to wipe all ink strokes from the associated InkCanvas. However, the stroke data is lost when the application is restarted, unless it is previously stored to a file or database.

There are two ways to work with stroke data that is currently being held in the StrokeContainer buffer. The first is by working directly with individual stroke elements. This may be appropriate if only a subset of strokes need to be stored, or if other processing needs to take place. The second option is to save or load strokes from an I/O stream using either the SaveAsync() or LoadAsync() operations. When using this option, the StrokeContainer currently supports storing stroke data into GIF image format, with the stroke data embedded into the file.

Recognizing Text and Shapes

So far, we have covered how to add Windows Ink support to a UWP app, how to add pen tools (including custom tools), and how to save/load ink stroke data. However, the inking story does not stop there; Windows Ink can find shapes, lines, and text — including text in many languages — as well as identifying semantic structures such as lists and text blocks. These stroke analysis tasks are performed by the InkAnalyzer class.

Analysis is typically performed after a short delay once a set of strokes have been collected in an InkCanvas. After analysis, if no additional strokes have been added or removed in the meantime, the results are scanned for recognized text or shapes. InkAnalyzer is intended to be reused between analysis events so that it only processes strokes that have changed, which improves performance of subsequent analysis events. Usage of InkAnalyzer involves a little more work than the other inking features previously looked at. For example, recognizing and extracting circles as they are drawn on an InkCanvas, entails significantly more code:

var brush = new SolidColorBrush(Colors.BlueViolet);
var analyzer = new InkAnalyzer();
var timer = new DispatcherTimer { Interval = TimeSpan.FromSeconds(1)};timer.Tick += async (s, a) => {
 timer.Stop();
 var strokes = canvas.InkPresenter.StrokeContainer.GetStrokes();
 if (strokes.Count == 0)
  return;analyzer.AddDataForStrokes(strokes);
 // only look for drawings
 foreach (var st in strokes)
  analyzer.SetStrokeDataKind(st.Id, InkAnalysisStrokeKind.Drawing); var results = await analyzer.AnalyzeAsync();
 if (results.Status != InkAnalysisStatus.Updated)
  return; var nodes =
   analyzer.AnalysisRoot.FindNodes(
    InkAnalysisNodeKind.InkDrawing);
 foreach (InkAnalysisInkDrawing node in nodes)
 {
  if (node.DrawingKind == InkAnalysisDrawingKind.Circle ||
      node.DrawingKind == InkAnalysisDrawingKind.Ellipse)
{
Ellipse ellipse = new Ellipse
{
Width = node.BoundingRect.Width,
Height = node.BoundingRect.Height,
Stroke = brush,
StrokeThickness = 2
};
Canvas.SetTop(ellipse, node.BoundingRect.Top);
Canvas.SetLeft(ellipse, node.BoundingRect.Left);
shapes.Children.Add(ellipse);
foreach (var id in node.GetStrokeIds())
{
var st = canvas.InkPresenter.StrokeContainer.GetStrokeById(id);
st.Selected = true;
}
analyzer.RemoveDataForStrokes(node.GetStrokeIds());
}
}
canvas.InkPresenter.StrokeContainer.DeleteSelected();
};
canvas.InkPresenter.StrokesCollected += (s, a) => {
timer.Stop();
timer.Start();
};

Digging Deeper

This has been an introduction to the fundamental concepts and UI controls in Windows Ink, but has barely scratched the surface of what’s possible, including advanced features such as wet or dry ink, custom drying, custom pens, and recognition of complex text constructs. The Windows 10 Fall Creators Update also recently introduced incremental rendering of ink strokes and the ability to work directly with an InkPresenter without an associated InkCanvas. The technology is rapidly evolving and each Windows release brings new improvements and features.

Go here to learn more about making your Apps more engaging.