Finger-specific key presses could speed up computer interaction

If you're looking for a way to use a computer more efficiently, researchers at the University of Waterloo's David R. Cheriton School of Computer Science may have a solution for you.

Master's candidate Jingjie Zheng and Professor Daniel Vogel have recently filed a patent that would allow computer users to trigger different shortcut commands by pressing the same keyboard key with different fingers, hands, or hand postures.

"At its core, this provides a more nuanced way to press keys on a keyboard," explained Vogel. "There has been a lot of excitement recently about controlling a computer with voice, touch and gestures. Our work revisits the lowly keyboard and demonstrates how it can be taught some new tricks."

In a paper presented at the Association for Computing Machinery's 34th annual Conference on Human Factors in Computing Systems (CHI 2016), Zheng and Vogel, found that users can access more keyboard shortcut commands by using a built in computer vision algorithm that identifies fingers and hands on a keyboard captured by a built-in laptop camera and reflector.

These finger-aware shortcuts can help programmers, video editors, stock traders, architects, graphic designers, and those who use complicated programs for their work. They would immediately benefit from this kind of method to access program commands using a keyboard. This could also help PC gamers who are looking to increase the number of available commands.

"Our goal is to provide people with as many keyboard shortcuts as possible using just the keyboard that we use everyday. It's a simple idea, but we found that people like it a lot," said Zheng. "There's been research looking at how physical modifications to the keyboard can offer more availability and expressivity, but there's been one extra input dimension from your hands all the time!"

During their study, participants were shown the command and they were required to activate the corresponding keyboard shortcut after doing a text entry task or a trackpad pointing task. There was a cheat sheet that could be activated by the space key to help the participant memorize the mappings. The user error rate was 1.9%.