Process
The Quantemo study for NetworkWorld was based on a proprietary process developed by OTOinsights. Methods of data collection, data analysis and participant selection were developed to maximize the study results. After Quantemo completed the study and provided specific observations and recommendations, NetworkWorld implemented those recommendations with a new and focused layout, navigation and design strategy.
Participants
Quantemo recruited 12 members of NetworkWorld's target audience — IT managers and executives who were also habitual visitors of NetworkWorld's online publication — to participate in the study. These participants were carefully chosen to fit the profile of a NetworkWorld.com reader, based on the company's previous market research, in order to bring the true NetworkWorld readership experience to the study.
Observation Methods
To analyze how visitors navigate the NetworkWorld.com interface, Quantemo used several quantitative and qualitative methods of observation. By applying scientific observation techniques, Quantemo is able to provide an objective view of the mental effort and emotional response produced when interacting with a web site. The NetworkWorld.com study included four distinct types of data channels:
- Physiological responses
Sensitive equipment monitored a variety of physiological reactions that measure emotional responses to stimuli. Several channels were monitored to gain information about each subject's emotional state throughout the study.
- Eye movements
Using specialized cameras to record eye movements on a screen, Quantemo gathered a great deal of information about what the participants saw on the NetworkWorld.com site, including areas of concentrated attention, what was noticed first, what objects required prolonged gazes to understand, how users visually search for items on a page, and much more.
- Video
Video cameras captured the facial expressions and body language of each participant throughout the study, which were cross-correlated with data from the eye recordings and physiological channels to provide a cohesive, point-by-point depiction of the user experience.
- Interview
Spontaneous and solicited verbal responses were recorded to provide additional information about each participant's experience. Feedback from participants augmented objective findings, allowing NetworkWorld and its visitors to benefit from Quantemo's comprehensive study methods and conclusions.
Data Analysis
After collecting enormous amounts of significant data, Quantemo analyzed each channel using a variety of methods. The goal was to examine this information from several perspectives — getting a mile-high view as well as putting it under a microscope — to gain a full understanding of the user experience with NetworkWorld.com. Once this understanding was achieved, Quantemo was able to make specific observations and recommendations to improve the NetworkWorld site for all users.
Data mining provided a statistical overview of the participants' experiences and allowed us to quickly pinpoint patterns of navigation and scanning choices as well as identifying points of frustration.
Quantemo does not rely exclusively on the analysis of computers, however. Interacting with a web site is a human experience, and this experience requires a human to understand habits and motivations, and especially to bring a true understanding of the causes of any frustration.
Therefore, Quantemo also reviewed physiological data, videotapes and eye movements, then cross-correlated them with the subjects' verbal comments and written questionnaire responses. With this degree of detailed analysis, the research lab was able to find even subtle patterns of behavior and could then determine their underlying causes. Additionally, while Quantemo seeks findings of statistical significance, at the same time observations for improvements based on even a single subject's experience are given appropriate weight. |
