CodeBetter.Com
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @CodeBetter

Peter's Gekko

public Blog MyNotepad : Imho { }

Recognizing handwritten text on a tablet PC

Way back on primary school (1963) my teacher tried to learn me to write. First with a pencil, later with a pen. It never was a big success, over the years I always had insufficient marks and nobody (including myself) could usually read what I had written down. Which made me a very early adopter for typewriters and later PC's with word-processors. Now, many years, cool keyboards and monster mice later I'm returning to the pen. That is, the tablet PC pen. Writing with a pen is far more comfortable and is a relief when your hands and wrists are sore of typing. And the tablet PC's handwriting does something most human had given up on: it does understand what I try to write down.

Handwriting recognition is integrated in every corner of the OS but imho it is only implemented right in one place: in the text input panel. While you scribble the input panel shows the recognition results in the tooltips. This is a tremendous feedback. When you seen something is not recognized as the intended word you can correct it on the fly in the scribbling. No need to erase anything, just scribble over the letters misunderstood. Just like you would do with pen and paper.

When you do a text recognition in a tool like Windows Journal an all text dialog will pop up and there is no longer the possibility to pen-correct your scribbling. I find that far less productive.

The basics of text recognition on the fly is not that difficult. to implement in your own tablet apps. The ink in a tablet is modeled in a collection of Stroke objects. An ink aware control has an Ink property which has a Strokes collection. The Strokes collection has an overriden ToString method which will recognize the strokes. (See this article for an introduction to the Tablet API).  When a stroke is added the control fires an OnStroke event. In the eventhandler you can do the recognition and present the result.

This demo application has an Inkpicture control and a label. The onStroke event is a one-liner.

private void inkPicture1_Stroke(object sender, Microsoft.Ink.InkCollectorStrokeEventArgs e)
{
   label1.Text = inkPicture1.Ink.Strokes.ToString();
}

Now this app has the basic feedback functionality.

Text recognition is expensive. What you will see is that the writing will lag behind as the text starts exceeding something like 10 words. After that you have to accept the recognized text and clear the picture or start using a more sophisticated algorithm.

Peter


Published Dec 15 2004, 11:59 AM by pvanooijen
Filed under:

Comments

Peter van Ooijen said:

It is. Excuse me for taking that for common knowledge.
# December 15, 2004 12:05 PM

Peter's Gekko said:

Tablet API : GetDefaultRecognizer() behavior
# December 22, 2004 1:17 AM

Tricia Beine said:

Hi Peter,
Yes my soon to be X Jamie Beine has one of these. While we were together I had an oppertunity to use his tablet PC. They are pretty handy. My son and Daughter have the writing tablet like to one in your picture. My son is very advanced for almost 6 and is very interested in technology! While my daughter is the artist. Somewhere down the road they will have their own team of web designers.(Grin)
Tricia Beine
# January 6, 2005 11:52 PM

Peter van Ooijen said:

hi Tricia,
It will be a different world with our kids just growing up with all this stuff which keeps amazing us every time. Well, different when it comes to the gadgets. I'm afraid we'll keep fightng each other. Take care.
Peter
# January 7, 2005 3:32 AM

TrackBack said:

# February 16, 2005 2:27 AM

Peter's Gekko said:

Recently I installed the Vista Beta on my Tablet PC. It works, given the occasional blue screen, quite...
# September 1, 2005 3:35 PM

Peter's Gekko said:

Recently I installed the Vista Beta on my Tablet PC.
It works, given the occasional blue screen, quite...
# September 1, 2005 5:02 PM

Leave a Comment

(required)  
(optional)
(required)  

Enter the numbers above:
Add
Check out Devlicio.us!