The program you see today is the result of rock-solid research by Edward Brent and Pawel Slusarz a few years back. Their seminal paper “Feeling the Beat: Intelligent Coding Advice from Metaknowledge in Qualitative Research” outlines their strategy for increasing the efficiency and quality of coding textual data by tapping into the great store of knowledge gathered by the coding process itself.

While computers offer a vast improvement over coding data by hand, the coding process isn’t eagerly anticipated by most researchers because it still requires vast manual effort:

In general, the work required for the coder for the second segment to be coded, or even the thousandth segment to be coded, is as great as it was for the first segment….The level of knowledge and the attention to detail required of the coder is as great or greater near the end of the project as it was at the beginning. As a result, coding is widely recognized to be a tedious, detailed, repetitive, even mind-numbing task.

Brent and Slusarz realized the metaknowledge gathered throughout the coding process could reduce the burden to researchers:

Between the time when the first segment is coded and the time when the last code of the project is entered, the researcher has amassed a vast database of coded data, typically including hundreds or even thousands of segments of text (or other materials)….The patterns of data that result have a clear structure, a “rhythm” if you will, that provides the underlying “beat” within which the “melody” of codes is experienced. Together, these diverse elements of knowledge about the data (the metaknowledge) provide a surprisingly informative set of cues as to what codes might be expected for any segment.

The authors go on to show how intelligent computational strategies—case-based reasoning, natural-language generation, semantic networks, and production rules—can take advantage of the knowledge implicit in coded information in qualitative databases to help code additional data.

If you’re interested in learning more about how Qualrus utilizes intelligent computational strategies to make coding easier, check out the full article: Feeling the Beat: Intelligent Coding Advice from Metaknowledge in Qualitative Research.

(Originally published in Social Science Computer Review by Sage Publications)

Qualrus gives you the ability to attach memo fields to the project, each source, each segment, each code, each link, each view, and each script. Adding a memo to the project, source, segment or code are the most common actions by Qualrus users, so we’ll cover that functionally in this walk-through.

To add a memo to a project

1. Right-click the mouse on the name of your project found in the Project Overview pane of the Qualrus main screen (the folder icon in the upper left of the screen).
2. Select “Memo” from the popup menu. This opens a yellow popup window with a text field in which you can enter the memo.
3. When you finish entering your memo, simple close the popup window. The memo will save automatically.

To add a memo to a source

1. With your source open in the center pane of the Qualrus main screen, right-click the mouse anyway in your source.
2. Select “Memo > Source” from the popup menu. This opens a yellow popup window with a text field in which you can enter the memo.
3. When you finish entering your memo, simple close the popup window. The memo will save automatically.

To add a memo to a segment

1. Click on the name of a code in the right pane of the Qualrus main screen to highlight the segment, then right-click the highlighted text.
2. Select “Memo (segment)” from the popup menu. This opens a yellow popup window with a text field in which you can enter the memo.
3. When you finish entering your memo, simple close the popup window. The memo will save automatically.
4. A yellow icon will appear near the code names in the right pane of the Qualrus main screen to indicate a memo is attached to that segment.

To add a memo to a code

1. Right-click on the name of a code in the right pane of the Qualrus main screen.
2. Select “Code Memo” from the popup menu. This opens a yellow popup window with a text field in which you can enter the memo.
3. When you finish entering your memo, simple close the popup window. The memo will save automatically.
4. You can also add memos to codes in the Code Editor. With a code highlighted, click the “Write a memo” button (the last icon on the right).

Memos for all objects work in much the same way. If you open the memo field for the same object later, you will see your previous menu that can be edited or deleted as you like. If you delete an object, its memo will be deleted as well.

Searching and Retrieving Memos

The information in memos can also be searched, retrieved, viewed, reported, and edited later as desired. This permits researchers to easily search for theoretical memos to assist in theory building and to provide a clear audit trail for theoretical development.

Segment memos

Retrieving memos attached to segments can be done in QTools Search.

1. Build a search designed to find each “segment” from “Project” then click the “Add” button.
2. Select the “Memo” radio button and click submit to close the popup window.
3. Click Submit to launch the search of all segments in your project with memos.
4. You may also want to only search for memos containing certain text. You can add these search parameters next to the “Memo” radio button in the popup window. (For example, you could search for all memos containing the phrase “This section is not done yet” and then show the objects attached to those memos.)

Project, Source and Code memos

Memos attached to projects, sources or codes can be retrieved by following the same process you used to add the memo. Alternately, you can run a simple script to retrieve all memos attached to a certain type of object.

1. Go to “Editors > Scripts” and click “New” to run a new script.
2. Paste in the appropriate script and click “Submit”.
3. For Project memos: writeln(Project.Memo)
4. For Source memos: forEach source in Project.Sources write(source.Memo)
5. For Code memos: ForEach code in Project.Codes writeln(code.Memo)

That’s it. Enjoy!

Primarily, structure is imposed on your data through the coding process. Qualrus’s code editor, along with its powerful Boolean searches and built-in QTools provide quick access to filtered and sorted information.

But there’s another way Qualrus assists with your data analysis that most people ignore: visualization.

Graphical views introduce a welcome change of pace from text-only reports and provide fresh perspective on your developing theory. Analyzing visual representations of data can help advance your thinking on a particular concept or the project as a whole.

Many of us already think visually when trying to define the relationships between various concepts. Like a football coach drawing up a new play, graphical views get the idea in your head down on paper, where you (and your team) can assess, critique and tweak it.

The Views Editor in Qualrus provides a dead simple method of generating visual network views based on your coding scheme. You can open the Views Editor by going to Editors » Views, or by double-clicking “Views” on the Project Tree. Hit “New” to create and save a new graphical view.

As a project develops, the number of codes and links may increase to a level that is difficult to visualize, so Qualrus provides two types of network views: the Star View and the Network View.

The Star View is designed to focus on a single code at a time and display all of the direct links between that "in focus" code and any other codes. The Network View is similar to the Star View, except there is no central code, all codes are equal. Unlike the Star View, in the Network View the codes can be any number of links chained together.

In Star View, just choose a code from the drop down list to display direct links between that code and any other code in the knowledge base.

With the Network View, you can choose any number of codes to add to the View. Once the codes appear in your View, click a code then “Extend” to bring up a list of all the links attached to this code.

Views are automatically saved, but you may want to rename them using the “Rename” button. Also note that you can save your View as an image file by going to File » Save To » File.

That’s really all there is to it!  You can find more detailed instructions on pages 131-135 in the Qualrus Manual.

On your next project, don’t forget to regularly generate some Qualrus Views for a fresh perspective on your evolving theory.

Importing project elements also allows you to reuse work you’ve completed on previous projects. A coding scheme developed for a project involving the analysis of presidential inaugural addresses might turn out to be useful for another project classifying media election coverage.

Qualrus makes it easy to import components from other projects such as codes, scripts and link types, or completely merge two projects into one.

Step 1: Prepare your “donor” project

Your “donor” project contains the components you’re interested in merging with your master project. In order to correctly pull information from this project, you should make sure it is working properly.

The absolute best way to get your donor project ready to is open it up and export it. Go up to File > Export > All project files. This creates a complete working copy of your project and ensures it’s ready for importing.

Step 2: Open Master project and start the Import

With your donor project ready to go, close it and open your Master project (this is the project you ultimately want to contain the combined information).

Select File > Import > …from another project

You will see a window in which you can select the Qualrus project file (*.IQP) to merge with the current project. Select your donor project and click Open.

Step 3: Choose import options

The import window gives you full control over which components you want to import.

The defaults tab allows you to select the types of components you wish to import. The other tabs (Codes, Links, Sources, Segments, Scripts) allow you to designate specific components themselves.

In most cases, you’ll want to import everything, but this is up to you.

This import feature has a built-in validation checking routine to make sure that when important each component all of the other components required for it are also imported. For example, if we wish to import a link between two codes X and Y then we must have code X and code Y in the project.

Keep in mind that if you have an identical source in both your donor and master project that you want to be treated as a single source, the source must have an identical name in each project. Otherwise, Qualrus will treat them as two different sources.

Step 4: Don’t disable verification

When you click OK, Qualrus displays a warning that verification to assure consistency may take some time and gives us the option of turning off verification.

Verification is very important to assure that the knowledge base is consistent, so let’s leave it on by selecting “No” when asked if we want to turn it off.

(Modern computers can handle this without taking too much time.)

Step 5: Double-check and save

When the import is finished, you should see the revised project displayed in the main Qualrus window.

Depending on which components you imported, the project can include sources from the other project, all of its codes, and even the assignments of codes to specific segments.

Double check everything for accuracy and save your new merged project.
 

If you missed a step, or simply want to go back and review, the entire series is posted right here on our blog:

PHASE 1: Prepare your data for analysis
PHASE 2: Get to know and love your data
PHASE 3: Let the Coding Begin!
PHASE 4: Refine your coding scheme
PHASE 5: Report the Results

Questions? Comments? Send a note to support@ideaworks.com

By now you’ve put a lot of work into your project. After collecting your data, you prepared it for analysis, got acquainted with some general themes, then scrupulously marked and coded segments. In Phase 4, you took a step back and refined your coding scheme, checking your work for inconsistencies and redundancy.

At this point, your data are neatly organized and ready to be reported. As you write up your final document, it’s important to refer often to your data. It’s easy to get side-tracked while discussing a particularly interesting theme and overlook less remarkable results.

Qualrus can generate a number of reports designed to illuminate different aspects of your project. To ensure that you fairly and completely characterize your project data, keep these reports handy while writing your final manuscript.

QTools Statistics

The statistics page on the QTools form (Tools > QTools > Statistics) provides a quick summary of codes by frequency of occurrence for both individual codes and code pairs. When this page is opened Qualrus searches the database, counting codes and code pairs.

The scope of this search can be changed on the “Source” section of the form to search only the current source, selected sources, or all sources in a project. Changing the scope of the search in the Source box will re-compute the values. Selecting an item from the code or code pair list displays the corresponding segments in the segments lists. Selecting a segment displays the segment’s contents and codes.

Use visual diagrams

Remember when you created a hierarchy of codes back in Phase 4? It’s a good idea to conceptualize these relationships visually while you generate your report.

The lower right pane of the Code Editor window generates a visual representation of any code you select in the sidebar, including links to other codes.

Additionally, you can generate visual reports by going to Editors > Views. Click “New” to save a new view. You can choose to use the Star View or Network View.

The Star View is designed to focus on a single code at a time and display all of the direct links between that “in focus” code and any other codes. So we must select the code that is to be the focus code from the pulldown list of codes in the top middle of the window.

The Network View is similar to the Star View, except there is no central code, all codes are equal. Unlike the Star View, in the Network View the codes can be any number of links chained together.

Extend reporting with scripts

Qualrus scripts allow you to generate customized reports that are not built in to Qualrus. You can check out the Qualrus manual or Help files to learn more about building your own scripts.

Also, we have a few scripts available for free on our Support page. Just follow the directions on the Support page to save and run your scripts.

Writing your report

In your final report, remember to include:

• A description of the procedure you used to analyze data
• An organized list or table of your findings (use the parent-level codes you created in Phase 4)
• Representative quotes for each theme (which you identified in Phase 4)

Of course, there is more you should include based on the purpose of your project!

Now, you’re a master!

And that wraps up Phase 5, along with our entire series “5 Steps to Mastering Qualitative Data Analysis”. Hopefully, these tips have made you a more efficient, confident data analysis expert.

So you’re done taking a first pass at coding. Nice! The goal of coding is to separate the content from the fluff, while marking your meaningful content with appropriate themes.

The next step is to review the work we’ve done and look for redundant codes, groups of similar codes and categories that need to be split up.

Fortunately, Qualrus puts everything we need in one place: the Code Editor.

The Code Editor

Go up to “Editors > Codes” to access the Code Editor. The left window pane lists all the codes you’ve created in the project. Click on a code to highlight it. The upper right window contains four tabs with information about the selected code. For now, let’s look at the “Segments” tab.

The “Segments” tab lists every segment tagged with the selected code across your entire project. The segment appears in green, attached codes in blue and information about the segment’s source and position in black. You can click on the green text to jump directly to the segment in the original source.

Start by browsing through each code to get a sense of its segments. Look for any inconsistency in the coding, codes that can be combined, or codes that should be divided.

If you need to recode something, just jump directly to the segment and edit the segment’s coding using the coding form.

Identifying representative quotes

While you’re browsing each segment per code, it might be a good idea to identify segments that are particularly illuminating. You’ll probably want to include some sample quotes for each category of codes in your final report, so go ahead and mark those down now.

When you see a good quote segment, just click the green text, jump to the segment and add a code called “Quote”. At the end of your project, you can easily generate a report containing all these quotes and plug them right into your paper.

Linking codes

In most projects, codes represent different levels of information, where certain codes are a subset of another, more general code. Creating a simple hierarchy of codes can be accomplished using links.

Highlight a code you’d like to link, then click on the “Links” tab (upper right pane). The selected code will appear as the “Subject”. Choose “isa” from the Link drop down selection, then the other code to link as the “Object”. You will see the link appear on the code graph in the lower right window. You’ve basically told the program that the first code (subject) is a (or “isa”) child of the second code (object).

Go through your project and sort your codes into parent categories. You may have to create new codes to represent new categories.

Generalizing codes

While linking some codes, you may have noticed two codes that represent the same thing. To combine these two codes along with their connected segments, we’ll use the QTools Generalize tool. “Tools > QTools” in the main menu will bring up the QTools menu. Find the “Generalize” tab.

Select the two codes you’d like to combine in the drop down menus labeled “Select code pair” (upper left). At the bottom of the left column, click “Generalize X and Y”. Choose what type of replacement you would like to do and click OK.

Generalizing is a powerful feature that you cannot undo, so make sure to backup your project before using this tool.

QTools, coding video and more

At this point, all of your important content is contained in a segment. This allows you to use every tool in Qualrus to view, sort, refine and categorize your data.

Check out the other QTools available – especially Search, Coincidental Codes, Refine and Statistics. Most of them are pretty easy to understand and can help improve the accuracy of your coding.

For a more in-depth walkthru on this step, check out our Coding and Code Editor Training Video.

Now that you know and love your data, it’s time to begin coding.

Coding is the central process of qualitative research, where you begin giving meaning to your unstructured information. Your goal is to accurately classify data with appropriate, consistent themes.

We do this by designating certain parts of our source as a segment, and assigning one or more codes to that segment. A segment can include any length of text, but each segment should represent a coherent thought (we generally create segments 2-6 sentences long).

When reading through a particular passage, ask yourself: “What is the fundamental meaning of this section? What themes or ideas are discussed?”

Creating new segments in Qualrus

To create a new segment, highlight the text you would like to code. Then click the “Code” button on the top of the screen, or right-click the text and choose “Code”. This will bring up the Coding Form. Create a New Code by clicking the “New Code” icon in the top left corner of the Coding Form. Enter the name of your code.

Notice that your new code appears in the top right window of the Coding Form labeled “Codes in this segment”. Click OK to see your code applied to your source.

Applying existing codes

To apply a code that already exists in your project, open the Coding Form as before. All the codes available in your project are listed in the lower left window pane labeled “All codes”. Highlight a code and click (>) to add it to the segment.

Note that you can add as many codes as you would like to a segment. It’s generally a good idea to add appropriate codes to the same segment rather than creating a lot of small, overlapping segments with only one code attached each. Codes that appear together in the same segment will appear on Qualrus’ Coincidental Codes report which can give you some ideas of codes or themes that may be related.

Suggested codes

Qualrus is unique because it collects your coding patterns and begins suggesting codes for certain segments. This can make the coding process go faster and serve as a quality-control check on your work. Codes suggested by the program appear in the top left corner of the Coding Form, labeled “Suggested by the program”.

Separating content from “fluff”

People often ask us if it’s okay to leave certain sections of text uncoded. Yes! An essential function of the coding process is separating the meaningful content in your sources from the insignificant filler.

By creating a segment, you’re telling Qualrus that the selected text contains important content. Furthermore, you must specify the theme of the content by selecting a code (or many codes). Text that isn’t included in a segment will basically be ignored by the program. Discriminately marking only certain text as a segment goes a long way in making your data more manageable.

Don’t need to be perfect

Even with qualitative research software like Qualrus, coding can take a long time. Just remember that your codes don’t need to be perfect. After the coding is done we will go back and refine our coding scheme, correct errors and group codes into larger themes. For now, do your best to identify the meaningful content in your sources and assign descriptive codes.

In the left pane (Project overview) of the main Qualrus window, click the “+” sign next to “Sources”. This will reveal the list of every source associated with your project. Alternately, the drop-down menu immediately above the left pane displays this information as well.

Double-click a particular source to open it in the center pane. You can open multiple sources at once; each source shows up as a separate tab in the main window.

Now, open up all your sources and start reading. Your goal is to get a sense of the data as a whole.

Taking an objective look

You probably collected your data over a few days, weeks or even months. Now is your chance to take a step back, get in an objective mind-set and see the forest through the trees.

Keep a pen and paper handy and start jotting down major themes you run across as you read. It’s okay to generalize at this point. You’ll get more specific when we start coding in Phase 3.

You may be tempted to skip this step, but skimming through all your data will give you a good overall sense of your project’s major concepts and how those concepts fit together. When you start coding, you’ll be able to generate codes that makes sense in the context of your entire study.

In addition, reading through your data in this way keeps you close to your data, which leads to a more thorough analysis and a scrupulous final report.

Backing up your project

Last, let’s practice one of the most important things to learn in Qualrus: How to back up and export your project.

Exporting creates an exact copy of all your project files and sources files. We recommend you export your project often as a method of backing up your project. This technique can also be used if you’d like to work on your Qualrus project on different computers.

To export:

1. Open the most up to date version of your Qualrus project
2. Click File > Export > All project files
3. A box will pop up asking where you want to save the project. This will create a folder that has the same name as what you type in the box for the file name. This folder will include all of the Qualrus project files and all of the sources. Save this folder temporarily on the desktop or in a backup folder.
4. If you’re exporting your project in order to back it up, you may want to name each exported project something slightly different, like “My Project June 12″ so you can keep multiple backups.

All right, Phase 2 complete! Next time we’ll look at starting to code your project.

Last week, I got a chance to journey beyond my computer screen and help lead a focus group testing a project in development, the Peer Advising System (PAS). After some brief instructions, our potential users were let loose to explore the application. And I got to watch.

While we only tested about 20 users, I collected a truckload of useful feedback. Observing someone interact with our program for the first time allowed me to specifically see how they approached the program, how we could have provided more instruction, and the overall mood of the user experience.

Most of all, I enjoyed talking with the group of testers afterward. PAS is designed to help military personnel identify a friend that may be exhibiting symptoms linked to Post-Traumatic Stress Disorder (PTSD), and many of our users related stories of friends, cousins and fathers who needed help, but never got it.

It was touching to remember the purpose behind our project and to get affirmation that our application would help soldiers and marines stay healthy and safe.

We collected important quantitative data during the week too, but as Jocelyn Wyatts in an article for GOOD reminds us:

When evaluating the effectiveness of a program, quantitative data alone does not convey enough meaning, and typically leaves us with many questions. Numbers are, of course, necessary, but shouldn’t be relied on alone. Statistics should be complemented by deep stories of the impacts on an individual, family, or a community, and we should spend as much time thinking about how to effectively craft these stories as we do focusing on how to present the numbers.

There is something comforting about number-centric, black-and-white metrics. But that’s not the whole picture.

Effectively evaluating your theory, application or product relies on connecting people’s unique perspectives and individual experiences with the data about them.