There is a new kid on the block in the world of qualitative data-analysis tools – Reframer by Optimal Workshop. Time to give its Beta version a thorough review.
What is it?: Reframer can be utilised by UX professional to enter, analyse, report and share qualitative data via a web-based tool. Cool features include:
• Entering observations in real-time while researching and testing.
• Adding tags and levels of significance to these observations.
• Grouping observations into common themes.
Results are displayed on a dashboard highlighting:
1. Overview of the times certain tags were used for observations
2. The themes found in your findings.
3. How often each significance level was assigned.
Objective took it for a test run, see our dashboard below:
After using the tool, here is our feedback:
General Usability: The question that comes straight to mind is, how different is this tool compared to tools already at our disposal, such as MS Excel, where you can do similar things, and more? An advantage of Reframer compared to Excel would be its minimal functionality, which could make it easier to use for the less-experienced user, and faster for the experienced user. However, when first using the tool, I found it surprisingly hard to conceptualize the structure of the information – that is, where the pages, tools and interactions are positioned in relation to each other. It would help to have a clear overview of the available features and content structure to assist in navigation through the tool. Having this structure enables users to create a clear mental model of the site and see possible use flows, even before they use it.
Tags, Significance levels and Themes: After playing around for a while I found you could add and create tags during and after logging your data. However, you are only able to set significance level whilst logging your observations. For me, it would make more sense to set the significance level afterwards since you can then involve all stakeholders. Furthermore, the utilization of the themes and what you can do with them is not very clear at first sight. Here the tool would benefit of obvious on-site support, such as mouse-overs for “new” terms and a well-structured support page.
Data Overview: When you enter observations on the designated page the page layout does not provide a good overview of your logged data. The screen real estate has not been efficiently utilized with observations shown in a short list.
Below you find a screenshot of the observation-entering page, note the total amount of observations in one view is limited (at 1920×1080 resolution)
This lack of overview increases the risk of duplicates in your data and gives a lower feeling of control to the user. The impression I get is Reframer tries to obviate this overview issue by letting users add tags and significance levels on the go, and get the overview afterwards on the dashboard. I do not think this is neither the right moment for this in the user flow nor the right approach, as this could create unnecessary increase of workload on the researcher during a session. Personally, having more observations visible on the screen, would make it a better tool to use in-session.
Post-test analysis tool: When used as a post-test analysis tool it has one downside compared to similar methods, like affinity diagramming with post-it notes. It is missing the fun factor, like “playing” with your data (e.g. moving around your post-its to categorise them), which you preferably do in collaboration with you client(s) or colleague(s). Adding drag and drop functionality, for example for the tags, would add a lot more fun. Further being able to change the list view of the observations into a virtual affinity diagramming activity with movable ‘post-it notes’.
In conclusion: Reframer will be hard to use whilst in-field or in session, although it can be very useful as an analysis and reporting tool afterwards. It has strong graphics, and when used correctly the tags and themes can provide useful information, for example for consultants, to create reports with. However, it does need a big usability makeover to make it a potential success.