Study leads to a system that lets people use simple English to create complex machine learning-driven visualizations

data visualization
Credit: CC0 Public Domain

The ubiquity and sheer volume of data generated today give experts in virtually every domain ample information to track everything from financial trends, disaster evacuation routes, and street traffic, to animal migrations, weather patterns, and disease vectors. But using this data to build visualizations of complex predictive models using machine learning is a challenge to experts who lack the requisite computer science skills.

A team at the NYU Tandon School of Engineering's Visualization and Data Analytics (VIDA) lab, led by Claudio Silva, professor in the department of computer science and engineering, developed a framework called VisFlow, by which those who may not be experts in can create highly flexible data visualizations from almost any data. Furthermore, the team made it easier and more intuitive to edit these models by developing an extension of VisFlow called FlowSense, which allows users to synthesize data exploration pipelines through a natural language interface.

The research, "FlowSense: A Natural Language Interface for Visual Data Exploration with a Dataflow System" won the best-paper award at this year's IEEE Conference on Visual Analytics Science and Technology (VAST).

On Tuesday, October 22, Bowen Yu, who received his doctorate at NYU Tandon under Silva, will present the paper at the opening plenary session of the IEEE Visualization Conference (IEEE VIS) in Vancouver, British Columbia. The study is one of several papers spotlighting VIDA research that will be presented at IEEE VIS, the leading venue for research and a premier conference for computer graphics.

At the conference, collaborators with VIDA, which has established itself as a leading research center for data visualization, will present visualization modeling projects with applications in astronomy, medicine, and climate research developed at or with the center:

VisFlow, introduced in 2017 and funded in part by the Defense Advanced Research Projects Agency's Data Driven Discovery of Models program is a web-based framework that allows the user to use simple drag-and-drop actions to interact with data easily, letting users create visual data models based on time series, networks, geographical locations, and more, all of which can be formed into a compact and interactive visualization dashboard.

Yu said FlowSense takes these capabilities a step further. "Imagine if one could simply speak or type a sentence to activate a dataflow diagram," he said. "This capability would make non-experts more comfortable users, while providing experienced users with shortcuts. We believe that with natural language support we can mitigate the learning curve for a system like this and make dataflow more accessible" he said.

Silva, an IEEE Fellow who is affiliated with NYU's Courant Institute for Mathematical Sciences, Center for Data Science, Center for Urban Science and Progress, and Center for Advanced Technology in Telecommunications, added, "We're offering VisFlow and FlowSense as , free-to-all code-based frameworks on github, as a way to motivate further development for visualization purposes. There really is a lot more research that could be done in this area, and it is my hope that FlowSense will be a major stimulant for more collaborative work in making dataflow systems more flexible, easy to use, and popular among data analysts."

More information: Bowen Yu et al, FlowSense: A Natural Language Interface for Visual Data Exploration within a Dataflow System, IEEE Transactions on Visualization and Computer Graphics (2019). DOI: 10.1109/TVCG.2019.2934668

Citation: Study leads to a system that lets people use simple English to create complex machine learning-driven visualizations (2019, October 23) retrieved 18 July 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

New data science method makes charts easier to read at a glance


Feedback to editors