INK RECOGNITION
Diagram and annotation tools are yet to gain general acceptance. One of the reasons for this is the need for far more accurate recognition than is currently available. Recognition is important as it allows these tools to support more intelligent tasks such as editing, execution and conversion of these diagrams and annotations. However the ambiguity of hand drawn diagrams and annotations makes recognition problems hard to solve.
Typically recognisers are comprised of capturing stroke features and using algorithms to combine these features to identify the meaning of the ink. While many recognition algorithms have been developed to date, most have been informed by ad-hoc, heuristic-based assumptions about sketch properties. Thus, there is a critical need for more rigorous analysis of sketch recognition performance and tuning. As an initial step we are looking at the problem of separating drawing ink from writing ink - dividing and gesture recognition.
There is little ink data publicly available currently and very little support for obtaining and managing such data. We have developed DataManager, a tool that supports efficient data collection, labelling, automatic dataset generation (for data mining) and evaluation of sketch recognition algorithms. DataManager's evaluation framework has numerous recognisers already integrated; these recognisers are written in a range of languages including C#, C++ and Java.
Our toolkit is currently being used by our own research group. We have compiled a labeled dataset of sketched diagrams. Data was collected from 20 participants, with 3 diagrams each, totaling 7261 strokes. The data has been labeled with single stroke and group labels. Using our stroke feature library we have measured 100 features for each stroke in our dataset to use for analysis; the feature data file was constructed using our automatic dataset generator function. This is an open source project and we encourage other researchers to use and contribute to this project.
DataManager can be downloaded here. For any enquiries please email.
Screenshot of the prototype of the ink divider software
Publications
- Schmieder, P., R. Blagojevic, B. Plimmer (2009) Automatic Evaluation of Sketch Recognisers, 6th Eurographics Symposium on Sketch Based Interfaces and Modelling, New Orleans, USA, p 85-92.
- Blagojevic R., Schiemder P. and Plimmer B. (2009) Towards a Toolkit for the Development and Evaluation of Sketch Recognition Techniques, Proceedings of Intelligent User Interfaces (IUI'09) Sketch Recognition Workshop, Florida, USA.
- Blagojevic R., B. Plimmer, et al. (2008). A Data Collection Tool for Sketched Diagrams, Proceedings of Sketch Based Interfaces and Modeling, 73 - 80.