Melanie Tan Uy
navigation

Task Analysis and Usability Testing of an Enterprise Search Platform (2021)

💡
The client requests a search engine evaluation: they want to know if they can sunset their old digital platform to their new platform. They were worried about user adoption. What can we learn from the old and bring into the new tool? We brought further context and value by adding generative research: uncovering how their web search engine fit into the workflow of their subject matter specialists.
Back to Home
 
Outline

NDA, now what?

Is this really IBM (International Business Machines)?

 
No, not really. Different industries and different product lines. What is the same is that the company is a Fortune 500, multinational behemoth that distributes a wide range of specialized technical products and services to the private and government sectors.
  • Submits project bids regularly for government contracts
  • Currently in the midst of a company-wide digital transformation

Why choose IBM?

IBM is well known for its in-house researchers, scientists, and technical research outputs. Similarly, this company also has subject matter experts, general experts, and project data support groups. The likelihood of the need for robust information architecture is high for both companies.

What will I read?

No actual results or analysis will be presented as the product is in development or evaluated. Instead, I will give a project overview and learnings.
 

Overview

 

Problem

This was the client’s first opportunity to not only test but also uncover the needs and experiences of users using their old and new search platforms.
  • If they can ultimately replace all the disparate services under an all-in-one integrated product and sunset their old platforms
  • Find out what users like about the old platform
  • What they could improve with this new one

Project Summary

This project is the usability testing of their new integrated search engine that encompasses multiple local and shared enterprise databases and cloud drives. Since the product is already completed, this project tests the overall usability of the new platform.

Research Impact on Future Product Iteration

  • Contrary to client impressions, the old platform can be discarded for the new one. We found that users, including data fans sampled, were not resistant to the new product.
  • Users’ anxiety and mistrust are linked to the ease of discoverability and transparency of data sources which the new platform could introduce gradually.
  • We also found new opportunities to improve the findability of projects and personnel.

Timeline

  • October-December 2021. Three months end-to-end testing cycle but Thanksgiving Christmas holiday disruptions led to delays in recruitment and interview schedules
  • The interviews and analyses were compressed into two weeks

Role and Responsibilities

💡
This was slated as a moderated user testing protocol. However, to fully understand how the tool interacted with their workflow, I shifted the test into a semi-moderated exercise with the users explaining to me how and when they use the tool.
I was a contract user experience researcher for Daito Design, a digital product agency that excels at strategy and product design for enterprises.
  • Added general task mapping in the discussion guide
  • Conducted remote semi-structured interviews with 2 data managers, 3 generalists, and 2 subject matter experts
  • Conducted three-step analyses
  • Distilled insights for the final presentation

Team Composition

💡
I worked with designers, design interns, and project managers. Since the product was already completed, the project set-up involved communicating with the product owner and designers.
Overall Project Management
  • Client Project Leader - product owner, user recruitment, and coordination
  • Senior Project Manager - client touchstone
  • Junior Project Manager - day-to-day internal team operations
Remote Interview
  • Lone researcher with note-taker for 3 interviews; transcription was later automated using Descript
Deck Design
  • Researcher, 3 research support members working asynchronously
  • Senior Designer, a Junior designer
  • Head Design Director
 

Deliverables and Project Artefacts

💡
The results were simplified in the presentation to the client. A longer deck was submitted later with a full report written for internal use and data resource.
Client-Facing
  • Mid-project findings presentation
  • Final client presentation
  • Full detailed final deck submission
Internal
  • Select relevant quotes from interview transcriptions and label them in Excel
  • Secondary analysis, affinity mapping, concept categorisation, and general user journey on Miro
  • Filled in research templates
 

Process Overview

💡
I shifted the moderated usability testing into a semi-moderated exercise to understand the workflow and product fit of the employees at different points in their project cycle alongside product testing.
  1. Build a ground-up needs analysis from subject matter experts
  1. Contextual interviews to assess the search and task ecosystem of subject matter experts
  1. First, second, and third-level data analyses
  1. Presentation deck development and documentation
  1. Client presentation
 
 

Project Learnings

 
  • We severely underestimated the timeline and manpower requirement for the level of analyses that the research lead wanted for this project.
  • I recommend reducing the steps in analysis to adjust to changes in the recruitment delays.
  • This project was crucial in reshaping the future client proposals, project management, and manpower requirements for Daito Design.
 
Back to Top
Back to Main Page
 
badge