(Not So) New Skills for AI-Enabled Insights: Qualitative Research

AI and Qualitative Research are a Natural Fit

The resurgence in qualitative research that was sparked a few years ago by concerns related to panel quality and fraud is likely to intensify due to the rise of AI tools for Insights, which offer the possibility of making qualitative analysis radically more efficient. This is, to some extent, already a reality.  

The days when a junior team member attends an interview or group and furiously takes notes, which are then used in tandem with transcripts to create summarized results are coming to an end.  Transcripts and even raw recordings can now be uploaded to reliable 3rd party tools that leverage LLMs and AI to create summary outputs and respond to questions in a reliable manner.  

In addition, the ability to use trained, AI-based chatbots for qualitative interviewing tasks is also here, but not yet as advanced as the tools available for analysis and therefore not as widely accepted.  

The jury is still out on when, if ever, chatbots will rival human moderators, but even at this early stage they provide some obvious advantages in terms of scaling:

• Data Quality Screening- in asynchronous, “qual at scale” or even quantitative settings where it may not be possible to resolve the respondent’s identify fully, it is much more difficult for bad actors to game the system over the course of an interview or exercise that involves numerous lengthy text exchanges.  Chatbots make this easy and efficient to do.

• Recruitment- this can be time and labor intensive and is usually handled in a transactional manner by individuals or organizations that don’t have a deep or ongoing relationship with the recruited individuals. 

  • Aside from situations where a chatbot could be seen as impersonal or inappropriate (e.g., research on sensitive topics, highly valuable customer relationships, etc.) chatbots can be used to interact with potential research participants in various scenarios and direct them to a screener for qualification.  

  
  

• Interviewing- In many qualitative research scenarios the amount of research that gets done is often limited by available budget and timing concerns. 

  • For example, to conduct a certain number of in-depth interviews usually requires a certain amount of lead time for recruiting and qualification and then time to allow for the logistics of the moderator to actually conduct the interviews. For this reason, the cost per IDI is generally in the hundreds of dollars. 

  • While a chatbot interviewer will not be as good as an expert moderator, it has the advantage of being able to conduct interviews more efficiently (even simultaneously).  So, once a recruit is done, a chatbot could, for example, conduct 50 interviews in a day whereas a human moderator would need a week or two for the same number of interviews. This can radically reduce costs.

In addition, the ease with which scalable qualitative exercises can be launched and integrated with quantitative methodologies is likely to break down the barriers between these types of research.  This will lead to “surveys” that:

• Contain more qualitative exercises mixed in with quantitative measurement

• Allow researchers to deploy more creative approaches to asking questions

• Have higher data quality thresholds (i.e., they are harder to qualify for and fraudulently respond to)

Implications for Skill Development

If in the future, if the “heavy lifting” associated with qualitative, both in terms of interviewing and analysis, is handled by AI-enabled tools, where does the “human in the loop” fit in?

I would suggest that there are two and a half roles for humans in future qualitative scenarios.  I’ll explain what I mean by a “half role” below. 

First, the most obvious role for humans in a future AI-enabled qualitative scenario is design.  The ability to crystalize a set of client objectives and resulting decisions, and then map that back to a discussion guide or exercise is the “secret sauce” that will create separation among researchers and value for clients. 

The second key role for humans in the future of qualitative will be the skill with which one can use the AI tools themselves to realize the potential advantages.  Any new technology involves a learning curve and being able to maximize the value that can be derived from the tool will be critically important.

Now for the “half role” mentioned above.  Despite what I have said about the promise of chatbot interviewers, I believe that moderation skills will increase in value in the future for two reasons:

• They play a significant role in ensuring good design which, as mentioned above, will be the critical input in the future. 

• There will always be a role for traditional “human to human” interviewing.  Whether it is the sensitive nature of the topic, number and type of the interviews to be conducted or simply considerations of quality related to the available AI tools, I do not foresee actual conversations between humans being totally replaced by AI chatbots.

  
  

However, I do think that they will become less common and, due to the laws of scarcity, the perception of value for this type of interviewing will increase.

Think of it this way- 150 years ago, the furniture in your house would have been handmade, either by you or by a local furniture maker. Nowadays, our furniture tends to be made in a factory, which means that we have a wider selection, lower costs and that the effort needed to acquire something new can be as little as simply going to a store or website and paying for it.

Now, assuming that you accept that this is the state of the modern furniture market, answer this question: are there people still making furniture the old fashioned way, by hand, one piece at a time?

The answer is yes.  But the implications of this production method are dramatic. 

Handmade furniture costs much more than the kind produced in a factory.  It also takes longer to get. On the other hand, it may allow for customization, meaning that the customer can have more input into the design.  

While the specifics are different, this same distinction could be made in other domains such as fast food vs. fine dining restaurants or luxury vehicles like Ferrari or Lamborghini vs. Toyota or Volkswagen.  

To sum up, the rise of AI tools is likely not only to drive the continued resurgence of qualitative research, but it will open up new approaches that blend quantitative and qualitative methodologies to leverage the strengths of each in a way that ensures higher levels of data quality while being efficient and scalable.

Following the rule that “AI is just a tool to be used by humans” this means that traditional qualitative training and knowledge around design will still be of paramount importance in the near to medium term, along with familiarity and fluency with the various tools available.  

And, finally, there will still be room for the “fine furniture” makers among us in terms of qualitative moderation skills.  They will be rare, but for those who make the cut, they will be able to charge a premium, something that in the past I have called a “people premium.”