You are reading an article. Lots more of these in the archives. Jon Udell Meet Jon Arrow

Articles

16Comment Retweet

Visualization Trends For The Noosphere

Jan 30, 2009 In Web Culture By Jon Udell

Data visualization is an expressive medium. We use it to tell stories that help us make sense of the world. For five hundred years we wrote and illustrated those stories for the printed page. Toward the end of the last century, Edward Tufte showed us how much we had yet to learn about envisioning information using ink on paper. And his lessons will always inform our practice. But with data processing tools, networked software, and digital displays, we enter a new era of data visualization -- and not a moment too soon

The world we must make sense of now is one in which human actions have planetary effects. The good news is that we can, for the first time, begin to measure those effects. We’re instrumenting the atmosphere and the oceans, and torrents of data are arriving from our sensors. The bad news is that we’re not yet very skillful storytellers in the medium of data. That’s true both in the specialized realm of science, and more broadly at the intersection of science, public policy, and the media.

The biosphere is, of course, no more complex than it ever was. It just seems that way because we can know more about it. But the noosphere — the swarm of intelligence pulsing through the electronic networks that encircle the planet — is growing more complex every day. Our social, economic, and political lives are ruled by abstractions, and we’ve got to make sense of those too. If we had truly apprehended the global pool of money,and the dynamics of credit default swaps,could we have prevented (or at least blunted) the current crisis? We’ll never know. But if we can produce useful and relevant heads-up displays, maybe we don’t have to fly blindly into the future.

Painting pictures in the medium of data requires a daunting array of talents. For starters, somebody has to mix the paint. Although data is plentiful, it’s rarely available in a useful form. One of the visualizations presented here uses data from the Centers for Disease Control. You’d think it would be straightforward, in 2009, to acquire data on the incidence of obesity, state by state, for the past 20 years. Not so. The MIX Online team had to download hundreds of megabytes of data from the Centers for Disease Control, in SAS transport format, and then write an R script to parse it. Mixing the paint shouldn’t be so hard. Publishers need to work harder to deliver data in more useful packages. Meanwhile, data harvesters will continue to labor behind the scenes.

One of the first benefits of visualization, by the way, is to check the quality and consistency of data. Do the numbers even add up? Often they don’t. Most data repositories are seen by very few eyes, and touched by very few hands. When we make source data visible and tangible, more eyes and hands can help ensure that the numbers measure what we think they measure.

Given reasonably clean data, how do we interpret the numbers? And how do we empower others to make their own interpretations? Graphic artists, statisticians, interaction designers, programmers, and cinematographers all possess relevant skills. Only a handful of organizations can employ multidisciplinary teams that draw on all these domains of expertise.

To democratize visualization we’ll need a new generation of software. With personal and then web-based computing, we’ve seen it happen again and again: spreadsheets, desktop publishing, web multimedia, cloud-based services. Now, across a range of devices as well as in the cloud, we have the raw technologies to democratize the visualization — and collaborative analysis — of data.

Web standards are a key ingredient. We continue to find new and better ways to use HTML and JavaScript to transform static web pages into dynamic displays fed by live data. Meanwhile we’re pushing the rich-client envelope with Java, Flash, and now Silverlight. And we’re using technologies like Deep Zoom to make vast datasets seem to flow effortlessly through skinny network pipes.

The challenge is to combine all these ingredients into toolkits that don’t require developers, or even an ordinary users, to be expert in a dozen disciplines. What will those toolkits need to be? Honestly, we’re not sure. Watching as the industry solves these problems is half the fun. In the meantime, this issue of MIX Online explores the user experience side of this question.

Follow the Conversation

16 comments so far. You should leave one, too.

Ryan said on Jan 30, 2009

These visualizations are AWESOME! I was wondering if you are ever going to make a "top 10 list" visualization. Thanks for all your hard work!

Thomas Lewis said on Jan 30, 2009

Hi Ryan, glad you like them. We haven''t thought about a Top 10 List visualization but I really like the idea. The good news is that we will have the code available for our visualizations up soon and you would be able to tweak it and build one!

Rick Barraza said on Jan 30, 2009

Great post. I really like to see people pushing Deep Zoom into the region of Data Viz and agree completely that it could really solve some nice challenges currently in place with very deep dataset visualization over the ''interwebs''. If/When we get setPixel() and getPixel() type features in WPF/Silverlight, I think we''ll also be able to overcome a magnitude hurdle to start visualizing million+ data points in a single display as well. Love these conversations, keep up the great work!

Nishant Kothary said on Jan 30, 2009

@Rick +1 on set/getPixel(). The things we could do with those two little methods :)

@Ryan That''s a good idea. I take it you''ve also checked out some of the visualizations in Smashing Mag''s roundup? http://www.smashingmagazine.com/2007/08/02/data-visualization-modern-approaches/

David said on Feb 2, 2009

I''m curious as to why you chose Silverlight rather than Flash which has a larger installation base. Any chance you''ll post these as Flash files?

Joshua Allen said on Feb 3, 2009

Hi David,

Thanks for asking!

The first visualization we started on was the social timeline, which requires fairly sophisticated functionality which would have been impossible or impractical to implement in Flash. Flash is not really comparable to Silverlight -- a more apt comparison would be Adobe Air. This isn''t to say that Silverlight was the only option, but simply pointing out that we would require the user to install *some* runtime -- whether Adobe Air, Google Gears, or Silverlight. Since Silverlight was used to stream Olympics, Democrat Convention, and Obama''s Inaugural speech, the deployment is as good as any alternative platforms, and more importantly, it takes very little time to install Silverlight -- which is available for Macintosh and Linux. If the install wasn''t a significant adoption blocker for for-profit business like NBC, we reason that it''s not a significant barrier to people seeing our visualization.

In any case, we''ll be uploading source code tomorrow, which means that enterprising people could port to whatever runtime they like. And we have already made the data endpoints available via REST calls so you could apply *alternate* visualizations, which is IMO even more interesting.

Joe said on Feb 3, 2009

The synergy between the MixONLINE team and xplane is amazing. A more friendly visualization of data it is one of the most exciting trends that are evolving right now. I am really excited by these technology, I work with companies developing effective working teams, and the data we collect is huge and is sometimes difficult to grasp by our clients, to see its real potencial. I will push data visualition in my company, i see it as a competitive advantage. If you dont mind ill keep being a real fan of your site and your work. Thanks for all the resources, see you later. Greetings from Mexico.

Thomas Lewis said on Feb 3, 2009

Thanks Joe, we don''t mind you "being a real fan" of our site. Glad you like MIX Online! :)

David said on Feb 4, 2009

Joshua: Thanks for the info about your choice of Silverlight; I appreciate learning some of the thinking behind the decision.

Stefan Constantinescu said on Feb 18, 2009

I don''t think the problem is standards in the presentation layer, the problem is standards in the way the data is produced. Like you said, it was a pain to take that data from the CDC and interpret it into something useful.

Why?

Laura Edell Gibbons said on Mar 5, 2009

...yes, but will MIX Online have their own version of the Google Lunar plex, dedicated to the future challenges of tapping into the noosphere? Namely, to help solve the demands of extra-terrestrial computation and address the effects that the absence of gravity has with the normal ordering of data?

I grok your post. Well, post(s) actually. Please keep it up.

Patrick Foley said on Mar 18, 2009

One of the key types of data to visualize is geo data - One of the software companies I work with regularly has a great solution for this: http://www.idvsolutions.com.

One of my favorite features of their solution is the way they use MOSS to organize and collect data - when thinking about any type of data visualization, this is a great technique to consider. I spoke with them about it here: http://www.isvinnovation.com/Directory/Description.aspx?EventId=1176.

Jones said on Apr 27, 2009

Nice post...

ahmet maranki ahmet maranki said on May 1, 2010

I’m curious as to why you chose Silverlight rather than Flash which has a larger installation base. Any chance you’ll post these as Flash files?...

diyaliz diyaliz said on May 1, 2010

We haven’t thought about a Top 10 List visualization but I really like the idea...

Coach Bags said on May 24, 2012

Me and my friend were arguing about an problem similar to this! Now I realize that I had been correct. lol! Thank you for that information you article.