ChatGPT provides a powerful foundation model but knowing how to leverage and enhance it for your startup’s use case requires an understanding of its strengths and weaknesses, starting with a basic understanding of how the technology works. This article builds on my previous article Generative AI and building competitive advantage by providing deeper insight into the tech.

ChatGPT belongs to a category of machine learning Natural Language Processing models known as Large Language Model (LLMs). LLMs process vast amounts of text data and infer relationships between words within the text.

Over the past few years, LLMs like ChatGPT have made remarkable progress in generating human-like language with impressive proficiency. Unlike humans, who learn from their experiences and the surrounding world as they grow, LLMs solely rely on the surface form of language for learning. Their measure of success lies not in effective communication, but rather in replicating linguistic patterns with high probabilities.

The journey of Generative Pre-training Transformer (GPT) models began in 2018 with the release of GPT-1 by OpenAI. Subsequently, the models underwent further advancements with the introduction of GPT-2 in 2019, GPT-3 in 2020, and most recently, InstructGPT and ChatGPT in 2022. Advancements in computational efficiency have empowered GPT-3 to be trained on substantially larger datasets compared to GPT-2. This expanded training capacity equips GPT-3 with the ability to tackle a broader spectrum of tasks.

At its core, ChatGPT aims to generate a reasonable continuation of the text it’s got so far. Consider a sequence of words “The spiciest cuisine in the” and the goal is to predict the next word. To do this, imagine scanning billions of pages of text on the web and in digitised books, searching for every occurrence of this specific text sequence. The goal is to identify the word that follows this sequence and determine its frequency or percentage among all instances found. ChatGPT effectively does the above, except that it doesn’t look at literal text match, but looks for words that match in meaning. As a result, it produces a ranked list of words that might follow, together with probabilities.

When composing an essay or summarising a paper, ChatGPT repeatedly asks the question, “given the text so far, what should the next word be?” It then proceeds to append one word at a time to construct the desired output.

Inferring meaning with self-attention

A mechanism called “self-attention” enabled huge improvements in infusing meaning into LLMs and enables the processing of significantly larger datasets. By allowing the model to give varying weights to different parts of the input sequence, GPT is able to infer meaning and context.

Underpinning this mechanism, GPT works by converting tokens (pieces are text, which can be a word, grouping or words/text, sentence) into vectors that represent the importance of the token in the input sequence. GPT further enhanced its self-attention mechanism, enabling the model to capture sub-meanings and more complex relationships within the input data. This expansion empowers the model with a greater understanding of nuanced concepts and complex connections.

Reinforcement Learning From Human Feedback

Making the transition from classical language generation to recognising and responding to specific communicative intents is an important step to achieve better acceptance of user-facing Conversational AI systems.

One method for this is Reinforcement Learning from Human Feedback (RLHF). ChatGPT implemented the RLHF approach, incorporating human feedback into the training process to better align the output to user intent and taking contextual clues from user prompts. Human feedback process aim is to turn a sequence of text into a scalar reward that mirrors human preferences.

Human annotators evaluate the outputs of randomly selected prompts provided to the initial language model and rank them based on their quality. This ranking system creates a reward system for training, giving it a unique ability to score any given text based on how well it is perceived by humans. RLHF not only fosters more authentic and purposeful conversations but also positively bias the model towards ethical values while mitigating unethical, discriminatory or even dangerous outputs.

OpenAI created an environment to involve people in the training process and develop models that can learn human preferences in a given communicative situation. Its immense power comes from linear human annotation effort, which is prohibitively expensive in terms of labelled data, the known bottleneck for all supervised and reinforcement learning endeavours.

Injecting world knowledge into LLMs for specific use-cases

The knowledge of LLMs is constrained to concepts and facts that they explicitly encountered in the training data. Consequently, they may lack domain-specific knowledge necessary for specific commercial use cases.

Another important limitation, as of now, is the recency of the information. Since language models lack a notion of temporal context, they can’t work with dynamic information such as the current weather, stock prices or even today’s date.

You can solve the above problems by systematically injecting additional knowledge into the LLM. This new input can come from various sources, such as structured external databases (e.g. WikiData), company-specific data sources and APIs. Knowledge injection thereby helps you build domain intelligence, which is increasingly recognised as a crucial distinguishing factor for vertical AI products. By incorporating specific knowledge into the model, you can enhance its capabilities and provide a competitive edge in your respective industry.

Strive towards building a flexible architecture for your LLMs. Numerous approaches currently exist, and will continue to emerge in the future, for not only fine-tuning the base models but also tweaking their internal behaviours and learning mechanisms. If you have technical capabilities, leverage open-source models that will allow you to adapt and maximise the value-add of LLMs in your products or create new products.

Footnote: LLMs can generate remarkable outputs, but upon closer examination, it becomes apparent which tasks can be assigned to them and which will remain exclusive to human capabilities in the foreseeable future. It is important to assess the appropriate scope and limitations of LLMs, recognising the unique strengths and abilities that humans possess.