A short Recap
The OpenAI announcement focused on several key aspects, including a dramatic decrease in the price of the hosted API, ChatGPT, which has been reduced by 90% since December 2022.
Additionally, the APIs hosted on Azure are expected to have more granular management and regional and geographic availability zones, providing further value.
The Whisper and ChatGPT APIs also make implementation and experimentation easier, and allow voice data to be included with text data.
ASR vendors must be feeling the pressure to stand out, and this can be achieved through superior customer service, fine-tuning, and support for minority languages.
Finally, OpenAI is allowing users to access a specific model version and update when needed, providing greater stability for production implementations, demonstrating the continuing maturation of the LLM environment.
Back to Chat Markup Langauge (ChatML)
The introduction of ChatML is highly significant and important due to its ability to protect against prompt injection attacks, to some degree at lest.
It segregates conversations into layers or roles, such as System, assistant, and user.
This is just version zero of the language, and more developments are expected.
Currently, the payload accommodated for in ChatML is only text, but OpenAI is planning on introducing other data types.
Users can still use the unsafe raw string format, but this approach is much more susceptible for injections. OpenAI is in a strong position to guide and manage the LLM landscape responsibly, setting the standard for creating applications.
ChatML makes explicit to the model the source of each piece of text, and particularly shows the boundary between human and AI text.
This gives an opportunity to mitigate and eventually solve injections, as the model can tell which instructions come from the developer, the user, or its own input.
~ OpenAI
ChatML Example Code
Below is a ChatML example JSON file with the roles defined of system, user & assistant.
And the working Python code snippet:
With the output below, notice the role which is defined, the model detail which is gpt-3.5-turbo-0301 and other detail.
In Closing
Creating a conversational interface based on Large Langauge Models (LLMs) presents a challenge due to the difficulty of sequencing prompt nodes into chains.
This is complicated by the unstructured nature of input, often in the form of natural language or conversation, making it difficult to manage the edges between nodes. ChatML can provide a standard target for data transformation and submission to a chain, thereby alleviating this issue.
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
.jpeg)
.svg)
.svg)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
