New programming tool turns sketches, handwriting into code

New programming tool turns sketches, handwriting into code
Credit: The 35th Annual ACM Symposium on User Interface Software and Technology (2022). DOI: 10.1145/3526113.3545619

Cornell University researchers have created an interface that allows users to handwrite and sketch within computer code—a challenge to conventional coding, which typically relies on typing.

The pen-based , called Notate, lets users of computational, digital notebooks open drawing canvases and handwrite diagrams within lines of traditional, digitized .

Powered by a , the interface bridges handwritten and textual programming contexts: notation in the handwritten diagram can reference textual code and vice versa. For instance, Notate recognizes handwritten programming symbols, like "n", and then links them up to their typewritten equivalents.

"A system like this would be great for , specifically with sketching plots and charts that then inter-operate with textual code," said Ian Arawjo, lead author of the paper and doctoral student in the field of information science.

"Our work shows that the current infrastructure of programming is actually holding us back. People are ready for this type of feature, but developers of interfaces for typing code need to take note of this and support images and graphical interfaces inside code."

Credit: The 35th Annual ACM Symposium on User Interface Software and Technology (2022). DOI: 10.1145/3526113.3545619

Arawjo also said the work demonstrates a new path forward by introducing artificial intelligence-powered, pen-based coding at a time when drawing tablets are becoming more widely used.

"Tools like Notate are important because they open us up to new ways to think about what programming is, and how different tools and representational practices can change that perspective," said Tapan Parikh, associate professor of information science and paper co-author.

Information on the tool was published in the proceedings of The 35th Annual ACM Symposium on User Interface Software and Technology.

More information: Ian Arawjo et al, Notational Programming for Notebook Environments: A Case Study with Quantum Circuits, The 35th Annual ACM Symposium on User Interface Software and Technology (2022). DOI: 10.1145/3526113.3545619

Provided by Cornell University
Citation: New programming tool turns sketches, handwriting into code (2022, November 28) retrieved 8 February 2023 from https://techxplore.com/news/2022-11-tool-code.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

New coding tool could aid computer programmers who are blind or have low vision

142 shares

Feedback to editors