Back to blog
Articles
May 29, 2023
·
5 MIN READ

Prompt Chaining

May 29, 2023
|
5 MIN READ

Latest content

Customer Stories
4 min read

How Infobip Generated 220+ Knowledge Articles with Gen AI For Smarter Self-Service and Better NPS

Partnering with HumanFirst, Infobip generated over 220 knowledge articles, unlocked 30% of their agents' time, and improved containment by a projected 15%.
September 16, 2024
Articles
7 min read

Non-Technical AI Adoption: The Value of & Path Towards Workforce-Wide AI

Reviewing the state of employee experimentation and organizational adoption, and exploring the shifts in thinking, tooling, and training required for workforce-wide AI.
September 12, 2024
Articles
6 min read

AI for CIOs: From One-Off Use to Company-Wide Value

A maturity model for three stages of AI adoption, including strategies for company leaders to progress to the next stage.
September 12, 2024
Tutorials
4 min read

Building Prompts for Generators in Dialogflow CX

How to get started with generative features.
August 15, 2024
Announcements
3 min read

HumanFirst and Infobip Announce a Partnership to Equip Enterprise Teams with Data + Generative AI

With a one-click integration to Conversations, Infobip’s contact center solution, HumanFirst helps enterprise teams leverage LLMs to analyze 100% of their customer data.
August 8, 2024
Tutorials
4 min read

Two Field-Tested Prompts for CX Teams

Get deeper insights from unstructured customer data with generative AI.
August 7, 2024
Tutorials
5 min read

Optimizing RAG with Knowledge Base Maintenance

How to find gaps between knowledge base content and real user questions.
April 23, 2024
Tutorials
4 min read

Scaling Quality Assurance with HumanFirst and Google Cloud

How to use HumanFirst with Vertex AI to test, improve, and trust agent performance.
March 14, 2024
Customer Stories
4 min read

How Infobip Generated 220+ Knowledge Articles with Gen AI For Smarter Self-Service and Better NPS

Partnering with HumanFirst, Infobip generated over 220 knowledge articles, unlocked 30% of their agents' time, and improved containment by a projected 15%.
September 16, 2024
Articles
7 min read

Non-Technical AI Adoption: The Value of & Path Towards Workforce-Wide AI

Reviewing the state of employee experimentation and organizational adoption, and exploring the shifts in thinking, tooling, and training required for workforce-wide AI.
September 12, 2024
Articles
6 min read

AI for CIOs: From One-Off Use to Company-Wide Value

A maturity model for three stages of AI adoption, including strategies for company leaders to progress to the next stage.
September 12, 2024

Let your data drive.

There has been a recent surge in Visual Programming tools which enable developers to chain large language model prompts into an application, thus creating a conversational user interface.

Introduction

A variety of chatbot development frameworks have incorporated Language Learning Models (LLMs) into their functionality as a means to supplement the existing chatbot development features. Unfortunately, the LLM implementations across these chatbot development frameworks have been largely similar in nature.

The initial implementation was followed by a second wave of conversational flow generation and conversation design assistants; which was spearheaded by Cognigy. And subsequently followed by Yellow AI and Kore AI.

LLMs are highly versatile with open-ended capabilities.

In this article, I want to dive deeper into the underlying principles behind designing and creating a native, visual programming interface for prompt chaining.

Native Prompt Chaining User Interface

Chaining Large Language Model prompts through a Visual Programming UI, the largest part of the functionality will be the GUI which facilitates the authoring process.

Below is such a GUI for prompt engineering, and prompt chain authoring. This design originated from research performed by the University of Washington and Google.

Source

The interface must offer granularity on multiple levels of the authoring process. Considering the chaining process, it is evident that between chains there are connections.

Any prompt chaining or LLM chaining tool must support data transformation between the steps of the chain. Considering that in most cases the LLM will return unstructured data, there will be a level of structuring and data transformation required.

A second element is the slight unpredictable nature of LLMs. A LLM can generate multiple responses to the same prompt. This gives rise to a challenge of cascading undesired responses in prompt chains. Where unexpected data is propagated throughout the chains.

Prompts can be customised at run-time without any specific model training required.

New tasks are easily be absorbed by the LLM Chained application.  This is achieved  by ingesting natural language instructions called prompts.

Granted, many real-world applications involve complex and often parallel multi-step tasks which pose a challenge for a single instance of chains.

Below a functionality stack below, a prompt chaining application based on Large Language Models (LLMs) & Generative AI should have three main components:

  • LLMs
  • Helpers
  • Communication
Adapted from source

LLMs

For many chains/nodes use-cases, the LLM output can be used unedited as supplied by the LLM.

In most cases this view will be presented to the user, or data transformation might be required for a next chain in the LLM App (aka Gen App).

The LLM output will determine to which chain to branch out to, hence these chain/node edges need to contain rules and conditions.

A last consideration is using the LLM as a classifier. LLMs can be implemented in a generative or predictive capacity. Generative AI is the most popular and most accessible side of LLMs. Predictive is more tricky and requires more fine-tuning or at least precise prompts.

Leveraging a LLM as a classifier (hence predictive) within a chain, makes the decision making nodes of the chaining application intelligent and flexible.

There will always be the temptation to quickly code/script decision making into the edges; also referred to as the chain-transition points. This is a quick-fix; but again introduces rigidity into the LLM Chaining application.

Helpers

Helpers assist in the evaluation of LLM output, considering aspects like politeness, empathy, etc. Helpers can also re-rank or resubmit prompts.

As granularity is a requirement for any production implementation, tweaking the prompt chained application will be important. A form of scripting will be required for processing and transformation of data.

Scripting encapsulates the development affordances given to the chain author to implement control measures, data transformation and more.

Communication

Communication takes place on a few levels, there is user data input in the form of unstructured conversational data.

Other user actions like asking the user to select the best response, or disambiguate based on a few possibilities.

And of course third-party systems (APIs) will be interrogated for any enterprise or production implementation.

The Design Of PromptChainer

The PromptChainer interface has a chain view visualiser where the chain structure with node-edges or “between-chains” can be created, edited, read and deleted [A].

The node viewer supports the implementing, improving and testing of each individual node or chain [B].

The editing of prompts for each node can be performed here.

PromptChainer also supports running the chain end-to-end [C] with options to clear the cache, view run logs, etc.

Source

Based on the quantitive research performed by the University of Washington and Google Research, discoveries where made on how users build and debug chains:

[1] Users want to build chains to not only mitigate LLM limitations, but also make their applications extensible and scale.

[2] Some users built one step of a chain at a time, while others sketched out abstract placeholders for all steps before filling them in.

[3] The interactions between multiple LLM prompts and chains can be complex, requiring both local and global debugging of prompts.

Considering the image below, here is a description for each node or chain:

[1] Define the input to a chain

[2,3] Use LLM output to filter and branch out inputs.

[4] Use the LLM output directly as the node output.

[5] Pre-implemented JavaScript functions for typical data transformation.

[6] Use the LLM output directly as the node output.

[7] Call external functions to connect professional services with LLMs.

[8] User-defined JS functions, in case pre-defined helpers are insufficient.

[9] Use the LLM output directly as the node output.

[10] Filter or re-rank LLM outputs by human-designed criteria, e.g., politeness. Enables external (end user) editing on intermediate data points.

Adapted from source

Conclusion

There are a few terms used currently for applications built on Foundation Models and Large Language Models. These terms are Gen Apps, Generative Apps, LLM Apps, Prompt Chaining, LLM Chaining and more.

Suffice to say, that there is a need to build on Large Models, and somehow harness the power of LLMs.

This need will be underpinned by the following principles:

  • Prompts
  • Leveraging LLM Predictive and Generative capabilities.
  • Chaining of Prompts; in both a parallel and series fashion.
  • A visual editor for programming chains/nodes and edges, between-nodes/chains.
  • LLM Response data transformation.
  • LLM Chained Applications will be conversational for both input and output, or the output can also encapsulate a RPA component.

I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.

Subscribe to HumanFirst Blog

Get the latest posts delivered right to your inbox