Finger-specific key presses could speed up computer interaction

Finger-specific key presses could speed up computer interaction
Researchers evaluate and demonstrate finger, hand, and posture identification as keyboard shortcuts. By detecting the hand and finger used to press a key, and open or closed hand postures, a key press can have multiple command mappings.

If you're looking for a way to use a computer more efficiently, researchers at the University of Waterloo's David R. Cheriton School of Computer Science may have a solution for you.

Master's candidate Jingjie Zheng and Professor Daniel Vogel have recently filed a patent that would allow users to trigger different shortcut commands by pressing the same keyboard key with different fingers, hands, or hand postures.

"At its core, this provides a more nuanced way to press keys on a keyboard," explained Vogel. "There has been a lot of excitement recently about controlling a computer with voice, touch and gestures. Our work revisits the lowly keyboard and demonstrates how it can be taught some new tricks."

In a paper presented at the Association for Computing Machinery's 34th annual Conference on Human Factors in Computing Systems (CHI 2016), Zheng and Vogel, found that users can access more keyboard shortcut commands by using a built in computer vision algorithm that identifies fingers and hands on a keyboard captured by a built-in laptop camera and reflector.

These finger-aware shortcuts can help programmers, video editors, stock traders, architects, graphic designers, and those who use complicated programs for their work. They would immediately benefit from this kind of method to access program commands using a keyboard. This could also help PC gamers who are looking to increase the number of available commands.

Finger-specific key presses could speed up computer interaction
Trigger different shortcut commands by pressing the same key with different fingers, hands, or hand postures.
"Our goal is to provide people with as many as possible using just the keyboard that we use everyday. It's a simple idea, but we found that people like it a lot," said Zheng. "There's been research looking at how physical modifications to the keyboard can offer more availability and expressivity, but there's been one extra input dimension from your hands all the time!"

During their study, participants were shown the command and they were required to activate the corresponding shortcut after doing a text entry task or a trackpad pointing task. There was a cheat sheet that could be activated by the space key to help the participant memorize the mappings. The user error rate was 1.9%.

Explore further

Computer program lets users learn keyboard shortcuts with minimal effort

More information: Finger-Aware Shortcuts, DOI: 10.1145/2858036.2858355 ,
Provided by University of Waterloo
Citation: Finger-specific key presses could speed up computer interaction (2016, May 12) retrieved 26 March 2019 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors

User comments

May 12, 2016
The problem is that people hardly remember or know already existing common shortcuts for the software they use. Key shortcuts change depending on the context and software, which makes them difficult to remember and increase the learning curve if the software relies on keyboard input for its functionality. Adding a layer of hand-gymnastics on top is just going to add more cognitive load and the function will remain unused.

A good example is the row of function keys from F1-F12 that are hardly used anywhere or for anything because people don't know what they do.

While the program may be faster to operate once learned and mastered, the users actually prefer the easier to approach graphical user interface and ignore the keyboard shortcuts for most tasks.

The most intuitive paradigm is that the keyboard is for data input, and the mouse or trackpad, or touchscreen, is for data manipulation. Mixing the two creates confusion.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more