RAG Evaluation

Retrieval Augmented Generation (RAG) is a very popular framework or class of LLM Application. The basic principle of RAG is to leverage external data sources to give LLMs contextual reference. In the recent past, I wrote much on different RAG approaches and pipelines. But how can we evaluate, measure and quantify the performance of a RAG pipeline?

Rivet Is An Open-Source Visual AI Programming Environment

Rivet is suited for building complex agents with LLM Prompts, and it was Open Sourced recently.

What Is The Future Of Prompt Engineering?

The skill of Prompt Engineering has been touted as the ultimate skill of the future. But, will prompt engineering be around in the near future? In this article I attempt to decompose how the future LLM interface might look like…considering it will be conversational.

LLM Drift

A recent study coined the term LLM Drift. LLM Drift is definite changes in LLM responses and behaviour, over a relatively short period of time.

LangChain Hub

A few days ago LangChain launched the LangChain Hub…

Language Model Cascading & Probabilistic Programming Language

The term Language Model Cascading (LMC) was coined in July 2022, which seems like a lifetime ago considering the speed at which the LLM narrative arc develops…

Comparing LLM Performance Against Prompt Techniques & Domain Specific Datasets

This study from August 2023 considers 10 different prompt techniques, over six LLMs and six data types.

Does Submitting Long Context Solve All LLM Contextual Reference Challenges?

Large Language Models (LLMs) are known to hallucinate. Hallucination is when a LLM generates a highly succinct and highly plausible answer; but factually incorrect. Hallucination can be negated by injecting prompts with contextually relevant data which the LLM can reference.

How Does Large Language Models Use Long Contexts?

And how to manage the performance and cost of large context input to LLMs.

RAG & LLM Context Size

In this article I consider the growing context of various Large Language Models (LLMs) to what extent it can be used and how a principle like RAG applies.

Two LLM Based Autonomous Agents Debate Each Other

This working code example using the LangChain framework illustrates how two agents can debate each other after each agent has been assigned a persona and an objective. The agents have access to tools which they can leverage for their response.

GPTBot

OpenAI’s Web Crawler

Iterative Prompting Pre-Trained Large Language Models

As opposed to model fine-tuning, prompt engineering is a lightweight approach to fine-tuning by ensuring the prompt holds contextual information via an iterative process.

OpenAI Discontinued Their AI Classifier For Identifying AI-Written Text

A while ago I took human & AI generated text from various sources, including LLMs & submitted it to the OpenAI Classifier. The objective was to gauge the classifier’s ability to detect the origin of text content.

Emerging Large Language Model (LLM) Application Architecture

Due to the highly unstructured nature of Large Language Models (LLMs), there are thought and market shifts taking place on how to implement LLMs.

12 Prompt Engineering Techniques

Prompt Engineering can be described as an art form, creating input requests for Large Language Models (LLMs) that will lead to a envisaged output. Here are twelve different techniques in crafting a single or a sequence of prompts.

Prompt Tuning, Hard Prompts & Soft Prompts

Prompt Engineering is the method of accessing Large Language Models (LLMs), hence implementations like Pipelines, Agents, Prompt Chaining & more which are LLM based are all premised on some form of Prompt Engineering.

Plan-And-Solve Prompting

The notion of fine-tuning a Large Language Models (LLMs) for very specific generative use-cases is in most instances not feasible. However, due to the flexibility of LLMs, variations in Prompt Engineering can yield astounding results. This article covers a new prompt method which improves LLM results in accuracy and completeness.

These Are The Updates To ChainForge

ChainForge is an IDE for prompt engineering and a number of important improvements were made to the tool.

Eight Prompt Engineering Implementations

In essence the discipline of Prompt Engineering is very simple and accessible. But as the LLM landscape develops, prompts are becoming programable and incorporated into more complex Generative AI structures.

Flowise Now Has Custom Tools With OpenAI Function Calling

In the latest Flowise version, Custom Tools are introduced together with OpenAI Function Calling. In this article I cover a few practical implementations.

Prompt Chaining & Large Language Models

What are the underlying requirements driving the need for prompt chaining? What defines prompt chaining and what are the essentials of a robust prompt chaining development tool?

Six GPT Best Practices For Improved Results

Here are six best practices to improve your prompt engineering results. When interacting with LLMs, you must have a vision of what you want to achieve and mimic the initiation of that vision. The process of mimicking is referred to as prompt design, prompt engineering or casting.

Retrieval Augmented Generation (RAG) Safeguards Against LLM Hallucination

A contextual reference increases LLM response accuracy and negates hallucination. In this article are a few practical examples to illustrate how explicit and relevant context should be part of prompt engineering.

OpenAI GPT Chat Completions Accounts For 97% Of Usage

Build Your Own ChatGPT or HuggingChat

By making use of haystack and Open Assistant, you are able to create a HuggingChat or ChatGPT like application.

OpenAI Researched The Labor Market Impact Of GPT4

This research produced a list of occupations and their level of exposure with the advent of LLMs and Generative AI.

ChatGPT APIs & Managing Conversation Context Memory

Currently, ChatGPT is powered by the most advanced OpenAI language models. While OpenAI has made the APIs available to these models, it does not inherently manage conversation context & memory.

Practical Examples of OpenAI Function Calling

Three use-cases for OpenAI Function Calling with practical code examples.

LLM Prompt Injection Attacks & Testing Vulnerabilities With ChainForge

Using the ChainForge IDE to batch test and measure prompt injection detection.

Agents

Autonomous Agents in the context of Large Language Models

Prompt Chaining

There has been a recent surge in Visual Programming tools which enable developers to chain large language model prompts into an application, thus creating a conversational user interface.

Intent Creation & Extraction With Large Language Models

In previous articles, I argued that a data-centric approach should be taken when engineering training data for Natural Language Understanding (NLU). Building on this, this article will discuss the importance of creating and using intents when working with Large Language Models (LLMs).

Prior To Chatbot Deployment, It Is Essential that Intents are Ground-Truthed To Ensure Accuracy

Reaching targeted levels of intent recognition can be quickly achieved by verifying intents before deployment, rather than adopting a corrective strategy as a response.

Prompt Drift & Chaining

The notion to create workflows (chains) which leverage Large Language Models (LLMs) are necessary and needed. But there are a few considerations, one of which is Prompt Drift.

Generative AI & The New Category of LLM Powered Applications

New approaches and tools built upon the strength of Generative AI are emerging to create engaging conversational experiences.

The Anatomy Of Large Language Model (LLM) Powered Conversational Applications

In order to ensure the successful deployment of any application built on LLM API calls, it is essential to generate true business value from it.

Google Cloud Vertex AI & Generative AI

Google launched Vertex AI on 18 May 2021 at Google I/O & it seems like the product has faired well considering all the recent advances in LLMs and Generative AI

LLMs will not be taking the place of traditional chatbot NLU in the near future.

NLU pipelines are well-honed and excel in extremely precise tweaking of intents and entities with no significant expense and rapid iteration cycles.

Five Advantages of NLU

I gained an appreciation for the power of Natural Language Understanding (NLU) engines while experimenting with the predictive and classification capabilities of Large Language Models (LLMs).

Prompt Engineering, OpenAI & Modes

What role can prompt engineering play in preventing LLM hallucination, and what constitutes a good LLM prompt? Furthermore, how are OpenAI's models impacting this?

NLU & NLG Should Go Hand-In-Hand

Traditional NLU Can Be Leveraged By Following A Hybrid NLU & NLG Approach

How To Create A Custom Fine-Tuned Prediction Model Using Base GPT-3 models

LLMs can be divided into two categories: generative & predictive. The generative capabilities of LLMs have been the subject of much attention and discussion, and rightly so – they are incredibly impressive

Chat Markup Language (ChatML) Is Important For A Number Of Reasons

Here I will discuss why ChatML, introduced alongside OpenAI's ChatGPT and Whisper APIs on 1 March 2023, is an important development that should not be overlooked.

Users of the ChatGPT API Will Need To Keep track Of Context

ChatGPT is currently powered by gpt-3.5-turbo-0301, the most advanced OpenAI language model. Although OpenAI has made the API for this model accessible, the API does not automatically manage conversations contextually...

Example Code & Implementation Considerations For GPT 3.5 Turbo, ChatML & Whisper

A while ago OpenAI has released the API for The LLM gpt-3.5-turbo, the same model used in ChatGPT. Additionally, the Whisper speech-to-text large-v2 model is available through an API for transcription.

OpenAI Mode Specific Models

OpenAI has implemented modes in their playground and development interface, each one having its own dedicated Large Language Model (LLM).

HumanFirst & Cohere QuickStart Guide

To start using the HumanFirst [https://www.humanfirst.ai] / Cohere [https://www.cohere.ai] integration you will need a Cohere [https://cohere.ai] API key...in order to get your

The OpenAI GPT-3.5 Turbo Model Has A 16k Context Window

OpenAI has unveiled a new model, dubbed "gpt-3.5-turbo-16k," and I was able to submit a 14-page document to the model for summarisation.

LangChain Hub

A few days ago LangChain launched the LangChain Hub…