Rise of Large Language Models (LLMs)

Part 1 — From Language Models to the AI Systems Changing the World

Introduction

If the Transformer architecture was the invention that changed Artificial Intelligence, then Large Language Models (LLMs) became the first practical demonstration of its extraordinary capabilities.

Today, millions of people interact daily with AI systems capable of writing essays, generating software code, summarizing books, translating languages, answering technical questions, solving mathematical problems, and participating in natural conversations.

These systems appear remarkably intelligent. However, they are not intelligent in the human sense. They are the result of decades of progress in mathematics, computer science, linguistics, distributed computing, optimization, hardware engineering, and large-scale data processing.

The emergence of Large Language Models represents one of the fastest technological transformations in computing history. Within only a few years, language models evolved from experimental research systems into technologies that now influence education, healthcare, finance, software engineering, scientific research, legal analysis, content creation, customer service, cybersecurity, and industrial automation.

Understanding this transformation requires answering three important questions:

• What is a Language Model?
• Why did Language Models become “Large”?
• Why did Large Language Models suddenly become so powerful after 2018?

This lecture answers those questions by examining the technological journey that led from early statistical language models to today’s frontier AI systems.

What Is a Language Model?

Before understanding Large Language Models, we must first understand the simpler concept of a Language Model.

This definition surprises many students. People often assume AI understands language exactly as humans do. In reality, language models learn by identifying relationships among words, phrases, sentences, and larger linguistic structures through exposure to vast amounts of text.

The Next-Token Prediction Problem

Imagine reading: “The Earth revolves around the ______.” Most people immediately answer Sun. Your brain predicts the missing word using previous knowledge. Language models perform a similar task mathematically.

Given “The Earth revolves around the”, the model calculates probabilities for many possible next tokens. Example:

The model selects the most appropriate continuation. After generating one token, the process repeats.

This repeated prediction process allows LLMs to generate entire books, programs, conversations, and reports.

Why Earlier Language Models Were Limited

Before Transformers, language models primarily relied on statistical methods, N-gram models, RNNs, and LSTMs. These systems achieved useful results but struggled to understand long documents, maintain context over extended conversations, capture subtle relationships, scale efficiently, and learn broad world knowledge.

Most importantly, they required significant task-specific engineering. Researchers often built separate systems for translation, question answering, summarization, text classification, and speech recognition. Every application required different architectures. This fragmented approach limited progress.

The Transformer Changed the Definition of a Language Model

The publication of “Attention Is All You Need” (2017) fundamentally changed Natural Language Processing.

Instead of building many specialized language systems, researchers could train one enormous Transformer model on massive collections of text. That single model could later perform many different language tasks.

This idea introduced the concept of Pretraining. Rather than teaching the model one task, researchers allowed it to learn the statistical structure of human language itself.

What Does “Large” Actually Mean?

Large Language Models are “large” because they involve several dimensions simultaneously.

The Birth of GPT

One of the most influential milestones occurred in 2018 with GPT — Generative Pre-trained Transformer.

• Generative — generates new text rather than merely classifying existing text.
• Pre-trained — learns general language patterns from enormous text datasets before being adapted to specific applications.
• Transformer — built upon the Transformer architecture introduced in 2017.

GPT demonstrated that a single pre-trained model could perform many language tasks with little or no task-specific redesign. This idea fundamentally changed AI research.

The Scaling Hypothesis

As researchers built larger GPT models, performance did not simply improve gradually. Entirely new capabilities began to emerge. Increasing parameters, data, and computing power often produced qualitative improvements in reasoning, translation, programming, summarization, question answering, creative writing, and knowledge retrieval.

This became known as the Scaling Hypothesis. Researchers realized that scale itself had become an important design principle.

GPT Evolution

Researchers later discovered that pretraining alone was insufficient for helpful human interaction. Models were further adapted using supervised instruction tuning and preference optimization based on human feedback, dramatically improving helpfulness, accuracy, safety, and conversational ability.

Why LLMs Feel Intelligent

One of the most common misconceptions is that LLMs “think” like humans. They do not. Instead, they learn extraordinarily complex statistical representations of language, knowledge, and relationships from large-scale training data.

Their impressive performance comes from combining pattern recognition, context modeling, generalization, probability estimation, and massive computational scale. The result often resembles reasoning, although researchers continue to study the extent and nature of reasoning abilities in these systems.

Foundation Models

Traditional software is designed for one task. Foundation Models are trained once on broad data and then adapted to many downstream applications.

For example, one LLM can support customer support, programming assistance, medical documentation, educational tutoring, translation, legal document analysis, research assistance, and enterprise knowledge retrieval.

This flexibility fundamentally changed software development. Instead of writing thousands of specialized systems, developers increasingly build applications on top of a general-purpose AI model.

Part 2 — From GPT to ChatGPT: How Large Language Models Became AI Assistants

Introduction

By 2020, Large Language Models had already demonstrated remarkable capabilities. GPT-3 could write essays, answer questions, generate programming code, translate languages, summarize documents, and produce creative writing.

Yet despite these impressive achievements, interacting with GPT-3 was often frustrating. The model was extremely knowledgeable but unpredictable. Sometimes it produced excellent answers; sometimes it misunderstood instructions, ignored the user’s request, or generated convincing but incorrect information.

Researchers realized an important fact: building a powerful language model is not the same as building a useful AI assistant.

A language model predicts the next token. An assistant must understand user intent, follow instructions, remain helpful, avoid harmful outputs where appropriate, and maintain coherent conversations.

This distinction marked the beginning of the next major phase in AI development. Instead of making models only larger, researchers began making them more aligned with human expectations. This transformation ultimately led to systems such as ChatGPT and today’s generation of AI assistants.

Language Model vs AI Assistant

Many people use these terms interchangeably. They are related, but they are not identical.

Every conversational AI assistant is built upon a language model, but additional training and engineering transform that language model into an interactive assistant.

The Challenge of Human Instructions

Consider: “Explain quantum computing like I’m twelve years old.”

A traditional language model may produce a technically correct explanation filled with advanced terminology. An AI assistant recognizes the user’s intention and adjusts the explanation to match the requested audience.

This ability is not learned automatically through next-token prediction alone. It requires additional training focused on human interaction.

Instruction Tuning

The first major improvement after pretraining was Instruction Tuning. Instead of exposing the model only to raw text, researchers trained it on examples consisting of instructions and expected responses—thousands to millions of examples.

Through exposure to many such examples, the model learned not only language but also how humans ask for information and what kind of responses they expect. Instruction tuning dramatically improved usability.

Why Human Feedback Became Essential

Even after instruction tuning, researchers encountered another challenge. Suppose three responses are generated—one accurate but difficult to understand, one accurate and clearly organized, and one with minor factual errors. Which should the model prefer? Mathematics alone cannot answer this question. Human judgment becomes necessary.

Researchers began collecting feedback from human reviewers, asking which response is more helpful, clearer, more complete, better follows the instruction, and which output users would prefer.

Reinforcement Learning from Human Feedback (RLHF)

One of the most influential techniques in modern AI is Reinforcement Learning from Human Feedback (RLHF). Despite its name, the underlying idea is straightforward.

Instead of simply predicting words, the model gradually learns which kinds of responses humans consistently prefer—factually accurate, easy to understand, well organized, and relevant to the question. This process significantly improved conversational quality.

Conversation Instead of Completion

Earlier language models primarily completed text. For example, “The capital of Japan is…” predicts Tokyo.

Modern assistants maintain ongoing conversations, remembering previous messages, tracking context, and adapting responses as conversations evolve. Conversation management became one of the defining characteristics of AI assistants.

The Industry Responded

The release of ChatGPT accelerated AI development across the technology industry. Within a short period, many companies introduced their own conversational AI systems, including Google’s Gemini, Anthropic’s Claude, Meta’s Llama family, Microsoft’s Copilot, Mistral AI’s models, xAI’s Grok, and DeepSeek’s open-weight models.

Although these systems differ in architecture, training data, and design goals, they all build upon concepts introduced by the Transformer architecture and the broader evolution of Large Language Models. The result was one of the fastest periods of innovation in computing history.

The Rise of Multimodal AI

Early LLMs processed only text. Modern AI systems increasingly understand and generate text, images, audio, video, documents, computer code, charts, and structured data. This capability is known as Multimodal AI.

Instead of treating each data type separately, multimodal models learn relationships across different modalities. This represents one of the most active areas of AI research today.

The Emergence of AI Agents

As language models became more capable, researchers explored whether AI could complete entire tasks rather than only answer questions. This led to AI Agents.

Unlike traditional chatbots, agents can plan multiple steps, use external tools, search knowledge sources, execute software functions, interact with databases, write and run code, and monitor progress toward goals. A simple conversational model waits for instructions; an AI agent actively works toward completing an objective within the permissions it has been given.

Current Challenges

Despite remarkable progress, Large Language Models continue to face important challenges:

• Hallucinations (producing incorrect information confidently).
• Reasoning limitations in some domains.
• High computational costs.
• Dependence on high-quality training data.
• Bias inherited from training data.
• Privacy and security concerns.
• Energy consumption during training and deployment.
• Model evaluation and reliability.

Understanding these limitations is as important as understanding their strengths. Effective AI practitioners recognize both capabilities and constraints.

Why Large Language Models Changed Computing

Throughout computing history, software has traditionally been developed by explicitly programming behavior. Large Language Models introduced a different paradigm: developers train general-purpose models capable of adapting to many tasks through natural-language instructions.

This shift is comparable to earlier revolutions such as the personal computer, the graphical user interface, the internet, smartphones, and cloud computing. LLMs represent not merely another software tool, but a new computational interface through which humans increasingly interact with digital systems.

Recap: LLMs scaled next-token prediction into Foundation Models and assistants via instruction tuning and RLHF—next, close the History Track with evidence-based directions in Future of AI.