Building Indescribable Products: The Scope of Conversational AI

If you’re reading this, you’re probably pretty in tune with the massive advances in conversational AI platforms and natural language processing technology.  However, if I asked you to describe the capabilities of your favorite LLM-based virtual assistant (e.g., ChatGPT) to a layperson, what would you say?  Even as someone whose career revolves around this technology, my description would be either too specific (e.g., “you can ask X”) or too broad (e.g., “you can ask about anything published on the internet before 2021”).  This underscores a fundamental challenge in building conversational AI:  as AI chatbot technology gets better, the scope gets wider and fuzzier.  Unfortunately, developers rely heavily on this scope, frequently limiting the success of AI chatbot projects.  In this article, we explore this challenge further and discuss mitigation strategies based on collective decades of AI chatbot development at scale by Knowbl’s team.

Conversational AI Development’s Achilles Heel

From a very simplified perspective, traditional software development starts with high-level business or user needs, transitions into defining requirements and design specifications, and finally development begins.  While there are more iterative aspects and variants to this (e.g., agile), the high-level process for granular tasks follows this process (e.g., a small story in agile).  Two key assumptions that make this process successful are:

1. Efficient communication of business needs to the product engineering team.  If the engineering team doesn’t understand what the need is, it’s difficult for them to build a solution that actually solves the need.
2. Efficient communication of technical design to the stakeholders.  If the stakeholders don’t understand what the product engineering team wants to build, it’s difficult for them to provide feedback or meet the engineering team in the middle in the feature vs. buildability tradeoff space.

Indeed, a recent Forbes article explores the top 10 reasons why companies fail, many of the reasons likely boil down to breakdowns in communication between product development and stakeholders.  To mention a few:  complacency, not putting customers first, and not treating data as a key business asset can all be caused by not understanding the voice of the customer (i.e. inefficient communication with the most important stakeholder).

As practitioners of conversational AI in the enterprise space for several years, we’ve seen these assumptions around efficient communication seriously strained at no fault of the participants – this is a fundamental challenge in the conversational AI space due to the breadth of what intelligent virtual agents are expected to handle.  While the collective talent at Knowbl building intelligent virtual assistants could write an entire textbook on this challenge, we’ll scratch the surface with a couple key aspects and some mitigation strategies in the next section.

Moving Quickly Combats Necessarily Vague Scope

The business need is massive for conversational AI, but also necessarily high level and vague.  The enterprise purchaser of conversational AI isn’t going to provide a specific set of intents (topic areas that the AI chatbot can talk about) and sometimes not even a vague analysis of what will benefit end users most.  Knowbl’s massive enterprise partners support millions of inquiries about their brand on topics involving countless products and several business units, so extremely advanced analytics would be required to produce a well-defined AI-powered customer support scope.  Even if the brand is on top of their customer support analytics game, finding a single point of contact with this level of visibility is unrealistic and wrangling many points of contact for this information is impractical early in an engagement.

To mitigate the challenge of vague conversational AI scope, brands can use the following guidelines:

• Push for quick iterations.  Nothing can capture the voice of the customer better than real customer usage.  If brands don’t have enough analytics to find a precise definition of the requirements, they can consider narrowing down the need by scoping a broad experience and seeing how a (subset) of their real users interact with it.  In some sense, conversational AI can become one of the brand’s best tools for analyzing user behavior, creating a strong feedback loop for further development.
• Focus on a well-defined subset of the business.  Covering the full scope of an enterprise with a single AI chatbot experience is a massive undertaking.  Even covering the highest impact products or customer support inquiries can be a very long-term project.  When brands pick a specific product or business unit, it becomes possible to target users of those topics to demonstrate quick wins and the potential of a wider development effort.
• Consider channels where low-risk, incremental value is available.  There are a number of deployment strategies to leverage this approach: live chat that is handled by the AI only when the user asks something that the AI is very confident about, initial classification of IVR interactions, or even a secondary layer of defense on an existing chatbot channel.  Similar to the points above, the sooner something is in production, the sooner the voice of the customer is heard.
• Phase out the project.  Certain conversational experiences are a lot harder to build than others.  Since the exact needs of the customer might not be well known, focusing on the low hanging fruit frequently yields good outcomes.  This can include considerations like unauthenticated FAQs vs. deep transactional experience with complex chatbot integrations.
• Find a partner, not just a vendor.  Conversational AI vendors must be committed to solving the problem, not building a rigid solution.  Unless the brand knows exactly what their users want to do with their intelligent virtual assistant, a statement of work with a fixed number of intents or conversational flows will likely crumble as soon as real-world users interact with it.

Perhaps intuitive given the speed at which natural language processing is evolving, the recurring theme for all of these points is to start small, move quickly, and prove success before committing to a massive conversational AI experience.  Aside from the challenges of scoping and requirement definition, some other common barriers to short-term success to consider include vendor production readiness, pace of innovation, and past team successes.  A poor security review or a lack of commitment to speed or project success metrics can drag out an otherwise good idea to the point of obsolescence.

Accelerate with Knowbl

Worried about becoming a conversational AI laggard with your current solution?  Knowbl’s team is built from the ground up of researchers and practitioners for the enterprise conversational AI space to move quickly with brands striving to be leaders in this space.  We understand the challenges of moving quickly in an enterprise setting from defining conversational AI scope to security to true partnerships around project success.  Our team can be reached at https://knowbl.com/contact/ to understand how we can demonstrate quick success with your very own BrandGPT model.

Written by Parker Hill