Back to blog
Articles
May 4, 2023
·
4 MIN READ

Example Code & Implementation Considerations For GPT 3.5 Turbo, ChatML & Whisper

May 4, 2023
|
4 MIN READ

Latest content

Customer Stories
4 min read

How Infobip Generated 220+ Knowledge Articles with Gen AI For Smarter Self-Service and Better NPS

Partnering with HumanFirst, Infobip generated over 220 knowledge articles, unlocked 30% of their agents' time, and improved containment by a projected 15%.
September 16, 2024
Articles
7 min read

Non-Technical AI Adoption: The Value of & Path Towards Workforce-Wide AI

Reviewing the state of employee experimentation and organizational adoption, and exploring the shifts in thinking, tooling, and training required for workforce-wide AI.
September 12, 2024
Articles
6 min read

AI for CIOs: From One-Off Use to Company-Wide Value

A maturity model for three stages of AI adoption, including strategies for company leaders to progress to the next stage.
September 12, 2024
Tutorials
4 min read

Building Prompts for Generators in Dialogflow CX

How to get started with generative features.
August 15, 2024
Announcements
3 min read

HumanFirst and Infobip Announce a Partnership to Equip Enterprise Teams with Data + Generative AI

With a one-click integration to Conversations, Infobip’s contact center solution, HumanFirst helps enterprise teams leverage LLMs to analyze 100% of their customer data.
August 8, 2024
Tutorials
4 min read

Two Field-Tested Prompts for CX Teams

Get deeper insights from unstructured customer data with generative AI.
August 7, 2024
Tutorials
5 min read

Optimizing RAG with Knowledge Base Maintenance

How to find gaps between knowledge base content and real user questions.
April 23, 2024
Tutorials
4 min read

Scaling Quality Assurance with HumanFirst and Google Cloud

How to use HumanFirst with Vertex AI to test, improve, and trust agent performance.
March 14, 2024
Customer Stories
4 min read

How Infobip Generated 220+ Knowledge Articles with Gen AI For Smarter Self-Service and Better NPS

Partnering with HumanFirst, Infobip generated over 220 knowledge articles, unlocked 30% of their agents' time, and improved containment by a projected 15%.
September 16, 2024
Articles
7 min read

Non-Technical AI Adoption: The Value of & Path Towards Workforce-Wide AI

Reviewing the state of employee experimentation and organizational adoption, and exploring the shifts in thinking, tooling, and training required for workforce-wide AI.
September 12, 2024
Articles
6 min read

AI for CIOs: From One-Off Use to Company-Wide Value

A maturity model for three stages of AI adoption, including strategies for company leaders to progress to the next stage.
September 12, 2024

Let your data drive.

Example Code & Implementation Considerations For GPT 3.5 Turbo, ChatML & Whisper

COBUS GREYLING
May 4, 2023
.
4 MIN READ

A while ago OpenAI has released the API for The LLM gpt-3.5-turbo, the same model used in ChatGPT. Additionally, the Whisper speech-to-text large-v2 model is available through an API for transcription.

GPT 3.5 Turbo & Chat Markup Language (ChatML)

The ChatGPT models are accessible via API, with gpt-3.5-turbo being used in the examples below. OpenAI also has a model named gpt-3.5-turbo-0301.

For up-to-date model information, the OpenAI model page is a great resource.

It is important to remember that OpenAI models are non-deterministic, meaning that the same input given at different times or even in a row can lead to different or varying results.

OpenAI states that setting the temperature to 0 will make the output mostly deterministic, but some variability may still exist. The ChatGPT web interface we are used to is very successful in managing conversational context.

ChatGPT web interface by OpenAI

It is important to note that the ChatGPT model (gpt-3.5-turbo) accessed through the API does not keep track of conversational context, as can be seen in the example below:

The ChatGPT model is able to maintain conversational context with a few-shot approach by buffering the prompts, similar to the way OpenAI managed context via the initial web interface.

An example of this few-shot learning prompt in action is shown below, with a very contextual and empathetic response from the ChatGPT model:

Chat Markup Language (ChatML) is an example JSON file that defines the roles of system, user, and assistant.

It is designed to protect against prompt injection attacks, which are the main security vulnerability and avenue of abuse for LLMs.

Below is a working example of a completion request sent to the gpt-3.5-turbo model using the ChatML file. The following Python code snippet can be run in a Colab Notebook:

Notice the role which is defined, the model detail which is gpt-3.5-turbo-0301, and other more in the output from the completion request below.

OpenAI Whisper large-v2 Model

Considering accessing the OpenAI Whisper AI via a Colab Notebook:

The result from uploading the MP3 audio file.

The lines of Python code to transcribe the audio:

And below is the output result…

I find it interesting that Whisper is able to detect the language of the recording before transcribing it.

According to the available Whisper models, languages and Word Error Rates (WER), Spanish has the best rate of 3, followed by Italian with a WER of 4, and English with a WER of 4.2. You can read more about it here.

I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.

Subscribe to HumanFirst Blog

Get the latest posts delivered right to your inbox