Personal and professional history might not repeat, but it certainly rhymes. I’m thrilled to join the team at HumanFirst, and reconnect with a team of founders I not only trust, but deeply admire.
The skill of Prompt Engineering has been touted as the ultimate skill of the future. But, will prompt engineering be around in the near future? In this article I attempt to decompose how the future LLM interface might look like…considering it will be conversational.
Personal and professional history might not repeat, but it certainly rhymes. I’m thrilled to join the team at HumanFirst, and reconnect with a team of founders I not only trust, but deeply admire.
In previous articles, I argued that a data-centric approach should be taken when engineering training data for Natural Language Understanding (NLU). Building on this, this article will discuss the importance of creating and using intents when working with Large Language Models (LLMs).
Introduction
As with AI in general, NLU Models also demand a data-centric approach to NLU Design. Improving NLU performance demands that the focus shift from the NLU model to the training data.
NLU Design best practice needs to be adhered to, where existing conversational unstructured data is converted into structured NLU training data.
NLU Design should ideally not make use of synthetic or generated data but actual customer conversations.
The image below shows the process which can be followed for a data centric approach to intent detection, creation and use. Staring with Embeddings…
Embeddings
The first step is to use conversational or user-utterance data for creating embeddings, essentially clusters of semantically similar sentences.
These groupings each constitutes an intent, each group needs to be given a label which is the “intent name”.
Read the article below for a detailed description of step one.
Create Classifications
Once we have the groupings/clusters of training data we can start the process of creating classifications or intents. The terms “classes” and “intents” will be used interchangeably.
For the purposes of this article I’ll be making use of the Cohere LLM.
The classification can be done via the Cohere classify post endpoint:
https://api.cohere.ai/classify
The training body of text is classified into one of several classes/intents. The endpoint only needs a few examples to create a classifier leveraging a generative model.
The Colab notebook snippet below shows how to install the Cohere SDK, and how to create a client. You will need an API key which you can get for free by creating a login on the Cohere website.
For reasons of optimisation and speed, it is best to make use of a small model.
Once you have installed the SDK and created your Client, run this code to create the intents.
Each with training examples…from cohere.classify import Example
You will see the training utterances are all labelled using one of three classes or intents:
Shipping and handling policy
Start return or exchange
Track order
Extract Classification
The text below shows the queries, analogous to user utterances submitted to the conversational agent…
inputs=[" Am I still able to return my order?", "When can I expect my package?", "Do you ship overseas?",
As seen in the image,
(1) The input or query utterances can be submitted (2) The Classifications are extracted (3) the results printed
A portion from the Colab notebook showing steps 1, 2 and 3
The results:
Conclusion
Intents are often neglected and seen as an insignificant step in the creation of a conversational agent. Frameworks like Amelia, Oracle Digital Assistant and Yellow AI offer synthetically generated training phrases. This approach can run the danger of trivialising the intent creation process.
Synthetic training data can suffice as a bootstrap measure, but will not serve well in creating a longer term sustainable solution. NLU Design and Data Best Practice should be adhered to from the onset.
Also, these synthetic training phrases are based on often “thought up” intents and intent names which are most probably not aligned with existing user intents. And definitely not addressing the long-tail of intent distribution.
It is time we start considering intents as classifications of existing customer conversations; a process of intent driven development is required for successful digital assistant deployments.
I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI and language. Including NLU design, evaluation & optimisation. Data-centric prompt tuning & LLM observability, evaluation & fine-tuning.
Subscribe to HumanFirst Blog
Get the latest posts delivered right to your inbox
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
.jpeg)
.svg)
.svg)
%20(42%20x%2024%20in)%20(1).png)
.png)
.png)
%20(280%20x%20208%20px)%20(4032%20x%202304%20px)%20(1).png)
