Back to blog
Articles
Articles
·
5 Min read

The OpenAI GPT-3.5 Turbo Model Has A 16k Context Window

|
5 Min read

Latest content

Customer Stories
4 min read

How Infobip Generated 220+ Knowledge Articles with Gen AI For Smarter Self-Service and Better NPS

Partnering with HumanFirst, Infobip generated over 220 knowledge articles, unlocked 30% of their agents' time, and improved containment by a projected 15%.
September 16, 2024
Articles
7 min read

Non-Technical AI Adoption: The Value of & Path Towards Workforce-Wide AI

Reviewing the state of employee experimentation and organizational adoption, and exploring the shifts in thinking, tooling, and training required for workforce-wide AI.
September 12, 2024
Articles
6 min read

AI for CIOs: From One-Off Use to Company-Wide Value

A maturity model for three stages of AI adoption, including strategies for company leaders to progress to the next stage.
September 12, 2024
Tutorials
4 min read

Building Prompts for Generators in Dialogflow CX

How to get started with generative features.
August 15, 2024
Announcements
3 min read

HumanFirst and Infobip Announce a Partnership to Equip Enterprise Teams with Data + Generative AI

With a one-click integration to Conversations, Infobip’s contact center solution, HumanFirst helps enterprise teams leverage LLMs to analyze 100% of their customer data.
August 8, 2024
Tutorials
4 min read

Two Field-Tested Prompts for CX Teams

Get deeper insights from unstructured customer data with generative AI.
August 7, 2024
Tutorials
5 min read

Optimizing RAG with Knowledge Base Maintenance

How to find gaps between knowledge base content and real user questions.
April 23, 2024
Tutorials
4 min read

Scaling Quality Assurance with HumanFirst and Google Cloud

How to use HumanFirst with Vertex AI to test, improve, and trust agent performance.
March 14, 2024
Customer Stories
4 min read

How Infobip Generated 220+ Knowledge Articles with Gen AI For Smarter Self-Service and Better NPS

Partnering with HumanFirst, Infobip generated over 220 knowledge articles, unlocked 30% of their agents' time, and improved containment by a projected 15%.
September 16, 2024
Articles
7 min read

Non-Technical AI Adoption: The Value of & Path Towards Workforce-Wide AI

Reviewing the state of employee experimentation and organizational adoption, and exploring the shifts in thinking, tooling, and training required for workforce-wide AI.
September 12, 2024
Articles
6 min read

AI for CIOs: From One-Off Use to Company-Wide Value

A maturity model for three stages of AI adoption, including strategies for company leaders to progress to the next stage.
September 12, 2024

Let your data drive.

Articles

The OpenAI GPT-3.5 Turbo Model Has A 16k Context Window

COBUS GREYLING
.
5 Min read

OpenAI has unveiled a new model, dubbed "gpt-3.5-turbo-16k," and I was able to submit a 14-page document to the model for summarisation.

A few days ago OpenAI made a new model available with the name gpt-3.5-turbo-16k. The astounding thing about this model is the size of its context window.

The image below shows the document submitted. The document consists of 14 pages and > 12,000 words, for summarisation, with success!

Something I found intriguing is that the gpt-3.5-turbo-16k model is a chat only model and is not supported by the completions endpoint. Below is the error message received:

Hence I had to opt for the chat endpoint and the ChatML notation for input.

Below a working Python application accessing the 16k model:

When I submitted a very large document which exceeded the context window, the error message received back is very informative in terms of what is possible from a context window perspective:

gpt-3.5-turbo-16k offers 4 times the context length of gpt-3.5-turbo at twice the price: $0.003 per 1K input tokens and $0.004 per 1K output tokens. 16k context means the model can now support ~20 pages of text in a single request. ~ OpenAI

The summary consists of 256 words, a one sentence brief summary and 12 key points.

Below is the response from the model, after processing the 14 page document

The usage detail at the bottom in terms token usage is helpful.

One of the ailments of LLMs which has been lamented for quite a while has been context window size of models. And that data has to be pre-processed to some degree to accommodate the limited context window size.

The sheer context which can be managed by this new OpenAI model is astounding, together with the speed and accuracy.

I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.

Subscribe to HumanFirst Blog

Get the latest posts delivered right to your inbox