The program you see today is the result of rock-solid research by Edward Brent and Pawel Slusarz a few years back. Their seminal paper “Feeling the Beat: Intelligent Coding Advice from Metaknowledge in Qualitative Research” outlines their strategy for increasing the efficiency and quality of coding textual data by tapping into the great store of knowledge gathered by the coding process itself.

While computers offer a vast improvement over coding data by hand, the coding process isn’t eagerly anticipated by most researchers because it still requires vast manual effort:

In general, the work required for the coder for the second segment to be coded, or even the thousandth segment to be coded, is as great as it was for the first segment….The level of knowledge and the attention to detail required of the coder is as great or greater near the end of the project as it was at the beginning. As a result, coding is widely recognized to be a tedious, detailed, repetitive, even mind-numbing task.

Brent and Slusarz realized the metaknowledge gathered throughout the coding process could reduce the burden to researchers:

Between the time when the first segment is coded and the time when the last code of the project is entered, the researcher has amassed a vast database of coded data, typically including hundreds or even thousands of segments of text (or other materials)….The patterns of data that result have a clear structure, a “rhythm” if you will, that provides the underlying “beat” within which the “melody” of codes is experienced. Together, these diverse elements of knowledge about the data (the metaknowledge) provide a surprisingly informative set of cues as to what codes might be expected for any segment.

The authors go on to show how intelligent computational strategies—case-based reasoning, natural-language generation, semantic networks, and production rules—can take advantage of the knowledge implicit in coded information in qualitative databases to help code additional data.

If you’re interested in learning more about how Qualrus utilizes intelligent computational strategies to make coding easier, check out the full article: Feeling the Beat: Intelligent Coding Advice from Metaknowledge in Qualitative Research.

(Originally published in Social Science Computer Review by Sage Publications)

I’ve never seen someone so excited by data. I was excited too, since Rosling does more than just slap some graphs on a slide. He tells a story with the data.

Rosling explains:

Statistics has to go together with other information and with experience. That’s when it’s useful…People are not stupid. They will understand statistics and they will use it, if it’s made available in relevant and interesting ways.

As professor of global health at Sweden’s Karolinska Institute, Rosling explores data drawn mostly from the United Nations, intent on dispelling common myths about the so-called developing world. His arguments and observations are convincing, though his main appeal lies in the way he presents data.

Convinced that global data should be accessible and understandable, Rosling founded Gapminder with his son and daughter-in-law in 2005. Gapminder was purchased by Google in 2007.

Gapminder assembles and standardizes an impressive collection of international data, then presents them as animated, interactive graphs. By emphasizing statistical time series, Gapminder gives life to numbers on economy, energy, health, employment and more.

Who would have thought I could waste a whole morning generating charts on infant mortality, oil consumption, and family size? The interface is incredibly slick, the data are thought-provoking, and the animations are illuminating.

In short — you have to try it.

In our data-obsessed culture, we need more pioneers like Rosling to help us find comprehensive and engaging methods to present information. We’ve got numbers on everything, but information is easily misconstrued unless placed in the proper context, in relation to other relevant information.

Perhaps more importantly — let’s be honest here — numbers are boring to most people. A typical Excel table or graph isn’t capable of empowering and mobilizing people for broad social change.

Rosling shows us how global data — presented in the right way — naturally lead to an engaging look at the big picture. Presenting information should tell a story.

Rosling’s story is about how the world is getting more and more similar. He predicts countries like China and India will replace western nations (like the United States and the United Kingdom) as the leading world economy by 2048.

What’s your story?

But that’s only half the story.

It’s vital to remember that a user’s experience on your site is more than the sum of their clicks. Ideally, we’d all have the chance to sit in the same room as our users, watching their every move then interviewing them about their experience. But besides being somewhat creepy, such one-on-one assessment is rarely practical.

Fortunately, there are a host of web-based tools available designed to collect qualitative data from your users quickly at a reasonable price.

Here are some of my favorites:

Userfly

Price: $0/mo – $200/mo

After getting an account and adding one line of code to your site, Userfly records videos of user visits, letting you see mouse movements, click and form interaction. It’s a straight-forward way to see actual users interacting with your site.

There’s a great demo available to test it out, and the free account gives you 10 captures each month.

ClickTale

Price: $0/mo – $790/mo

ClickTale also records user visits and provides some great aggregate reports to help further understand user behavior. For instance, their form analytics report details where visitors might be giving up on your form. And the heatmaps metric shows which content visitors are actually viewing. What parts of the page do visitors skip? How far down do they scroll?

You also get enhanced information about link clicking behavior. Besides collecting click rates, ClickTale tells you how long users hovered over links and hover-to-click ratios.

FiveSecondTest

Price: Free

FiveSecondTest allows you you to submit your designs for testing. Other users view your design for five seconds, then jot down five things they remember.

While it’s not as comprehensive as some other services, FiveSecondTest provides a really quick overview of how others react to your web page. If you’re designing a landing page, it’s a great way to ensure users are seeing what you want them to see. The basic functionality is free, so it’s worth throwing your design up there and seeing what people think.

It’s just as easy to do a test yourself, so don’t forget to help another designer out and comment on their page.

Loop11

Price: $350 flat fee for each project (up to 1000 participants)

Loop11 combines user video recording with customized usability surveys. Users interact with your site, completing a series of predefined tasks while answering questions about their experience as they go. All interactions are captured, processed and made available in real-time reports.

The process is simple: create user tests for your website (tasks and questions), invite participants (through an invite link), and analyze your results. Loop11 is a good option for a fairly well-developed project, when you have some specific questions in mind.

UserTesting.com

Price: $29 flat fee for each video

UserTesting.com allows you to watch real people using your website, while they narrate their experience. After creating a user test, UserTesting.com finds testers that meet your criteria. Testers then use your website, recording their screen and speaking their thoughts as they browse. Then you get the video, along with a written summary.

Unlike Loop11, you don’t have to find your own testers, but you pay for each user video ($29 flat fee). If you think you’ll have trouble finding testers, or only need a handful of users (under 10), then UserTesting.com might be a good choice for you.

Last week, I got a chance to journey beyond my computer screen and help lead a focus group testing a project in development, the Peer Advising System (PAS). After some brief instructions, our potential users were let loose to explore the application. And I got to watch.

While we only tested about 20 users, I collected a truckload of useful feedback. Observing someone interact with our program for the first time allowed me to specifically see how they approached the program, how we could have provided more instruction, and the overall mood of the user experience.

Most of all, I enjoyed talking with the group of testers afterward. PAS is designed to help military personnel identify a friend that may be exhibiting symptoms linked to Post-Traumatic Stress Disorder (PTSD), and many of our users related stories of friends, cousins and fathers who needed help, but never got it.

It was touching to remember the purpose behind our project and to get affirmation that our application would help soldiers and marines stay healthy and safe.

We collected important quantitative data during the week too, but as Jocelyn Wyatts in an article for GOOD reminds us:

When evaluating the effectiveness of a program, quantitative data alone does not convey enough meaning, and typically leaves us with many questions. Numbers are, of course, necessary, but shouldn’t be relied on alone. Statistics should be complemented by deep stories of the impacts on an individual, family, or a community, and we should spend as much time thinking about how to effectively craft these stories as we do focusing on how to present the numbers.

There is something comforting about number-centric, black-and-white metrics. But that’s not the whole picture.

Effectively evaluating your theory, application or product relies on connecting people’s unique perspectives and individual experiences with the data about them.