Chapter 1 defines the discipline the rest of the book serves. AI engineering is building applications on top of models that someone else trained: you select, adapt, evaluate and deploy a foundation model instead of collecting a dataset and training your own. The chapter explains how language models grew into foundation models, why a whole engineering discipline formed around them in roughly two years, what people actually build, how to decide whether a given application is worth building, and how the work differs from classical ML engineering.

An engineer should care for a practical reason. Most AI products fail not because the model is weak but because the team never decided what "good enough" meant, how humans and AI would split the work, or what the moat was. This chapter is the planning vocabulary that interviewers expect when they ask "how would you build an AI feature for X?"

In one minute

• A language model predicts tokens. Masked models (BERT) fill in blanks using context on both sides; autoregressive models (GPT, Claude, Llama) predict the next token and are what "LLM" means in practice.
• Self-supervision removed the labeling bottleneck: every text sequence is its own training signal, so the available training data became effectively the whole internet, and models scaled from 117M parameters (GPT-1, 2018) to hundreds of billions.
• A foundation model is a large, general-purpose, often multimodal model that many applications are built on. Scale unified previously siloed fields (NLP, vision, audio) and turned task-specific models into general ones.
• AI engineering exploded because of three forces: general-purpose capability, a flood of investment, and a low entry barrier (models as an API, plain-English interfaces).
• Eight use-case families cover almost everything built so far: coding, image and video, writing, education, conversational bots, information aggregation, data organization, workflow automation. Enterprises deploy internal-facing apps first.
• Planning questions: why build (existential threat, missed opportunity, hedge), what role AI plays (critical vs complementary, reactive vs proactive, dynamic vs static), what the moat is (technology, data, distribution), what metrics define success, and how to budget for the last mile from a demo to a product.
• The AI stack has three layers: application development, model development, infrastructure. Most new tooling is at the application layer; infrastructure needs barely changed.
• Versus ML engineering: you adapt rather than train, evaluation is harder because outputs are open-ended, the interface becomes a product surface, and the data work shifts from feature engineering to curation, dedup and retrieval. The workflow flips from data-first to product-first.

From language models to foundation models

Tokens and the completion machine

A language model encodes statistical knowledge of one or more languages: given some context, how likely is each possible continuation. Given "My favorite color is", a model of English should rate "blue" far above "car".

The unit it reasons over is the token: a character, a word or a word fragment such as tion, depending on the tokenizer. GPT-4 splits "I can't wait to build AI applications" into nine tokens, with "can't" becoming can and 't. The set of tokens a model knows is its vocabulary: about 32,000 for Mixtral 8x7B and 100,256 for GPT-4. For English text and GPT-4's tokenizer, a token averages roughly three quarters of a word, so 100 tokens is about 75 words.

Figure 1-1. An example of how GPT-4 tokenizes a phrase

Why tokens rather than words or characters?

1. Compared with characters, tokens carry meaning. "cooking" splits into "cook" and "ing", both of which recur elsewhere.
2. Compared with words, there are far fewer distinct tokens, so the vocabulary and the output softmax stay small.
3. Unknown words degrade gracefully. A coinage such as "chatgpting" becomes "chatgpt" plus "ing" instead of an out-of-vocabulary failure.

Tokenization has engineering consequences you will meet in later chapters: pricing and context limits are in tokens, non-English text often costs several times more tokens per sentence, and token boundaries explain odd failures in spelling, counting and arithmetic.

Masked versus autoregressive models

Type	How it predicts	Canonical example	Typical use
Masked language model	Predicts missing tokens anywhere in a sequence using context from both directions	BERT	Classification, sentiment, entity extraction, embeddings, code debugging
Autoregressive language model	Predicts the next token from the preceding tokens only	GPT-4, Claude, Gemini, Llama	Any generative task: chat, writing, code, translation, summarization

Figure 1-2. Autoregressive language model and masked language model

Autoregressive models dominate because generation is the more general capability. Unless stated otherwise, "language model" in this book means an autoregressive one. The mental model is a completion machine: give it text (a prompt) and it continues the text. Translation, summarization, coding and math are all expressible as completion, which is why one model can do all of them.

Self-supervision: the breakthrough that enabled scale

Classical ML is supervised: each training example needs a human-provided label, which is slow and expensive. Language modeling is self-supervised: the labels are the data itself. Every position in a sentence yields one training example whose target is simply the next token. The sentence "I love street food." produces six examples automatically.

Input (context)	Target (next token)
<BOS>	I
<BOS>, I	love
<BOS>, I, love	street
<BOS>, I, love, street	food
<BOS>, I, love, street, food	.
<BOS>, I, love, street, food, .	<EOS>

<BOS> and <EOS> mark the beginning and end of a sequence. Because text exists in near-unlimited quantities (books, web pages, code, forums), self-supervision turned the data bottleneck into a compute bottleneck, and compute scales with money. That is what let language models become large language models.

How large is large? GPT-1 (June 2018) had 117 million parameters and was considered big. GPT-2 (February 2019) had 1.5 billion. At the time of the book a model of 100 billion parameters counts as large, and the threshold keeps moving. The author's compression of the era: "If I could use only one word to describe AI post 2020, it would be scale."

From LLMs to foundation models

Text is one modality. Humans also perceive images, sound and more, so language models were extended to other data types. GPT-4V and Claude 3 read images alongside text; other models handle video, 3D assets, protein structures and audio. A model that handles more than one modality is a multimodal model.

Figure 1-3. A multimodal model can generate the next token using information from multiple modalities

The term foundation model (popularized by Stanford's 2021 report) captures two transitions:

1. From siloed research to unified models. NLP, computer vision and speech used to be separate fields with separate models. One foundation model now spans them.
2. From task-specific to general-purpose. A sentiment classifier could not translate. A foundation model does both, plus hundreds of tasks nobody explicitly trained it for. Benchmarks such as Super-NaturalInstructions, which collects over 1,600 task types, exist precisely to measure that breadth.

Figure 1-4. Tasks used by the Super-NaturalInstructions benchmark

The book uses foundation model for both large language models and large multimodal models. Because a foundation model is general, it is rarely optimal for your task out of the box. Three adaptation techniques, each a later chapter, close the gap.

Technique	What it changes	Data needed	Effort	Chapter
Prompt engineering	The instructions, examples and context in the input	A handful of examples	Low	5
RAG (retrieval-augmented generation)	What information the model sees at inference time by retrieving from an external store	A corpus to retrieve from	Medium	6
Finetuning	The model's weights	Thousands of task examples	High	7

Adapting an existing model is almost always cheaper than training one for your task: ten examples and a weekend rather than a million examples and six months.

From foundation models to AI engineering

Scaling up had two consequences. Models became more capable, enabling more applications. Training them became so expensive (data, compute, scarce talent) that only a few organizations can do it, so those organizations sell access: model as a service. Demand for AI applications rose while the barrier to building them fell, and AI engineering became one of the fastest-growing engineering disciplines. Three factors explain the timing.

Factor 1: general-purpose capability. One model writes, codes, translates, reasons and generates images. Applications that were research projects in 2019 became weekend prototypes, which expanded both the user base and the demand.

Factor 2: investment. Goldman Sachs Research projected AI investment approaching $100 billion in the US and $200 billion globally by 2025. FactSet found that about one in three S&P 500 companies mentioned AI on their Q2 2023 earnings calls, triple the year before. Capital follows that attention.

Figure 1-5. The number of S&P 500 companies that mention AI in their earnings calls

Factor 3: low entry barrier. Model-as-a-service means a single API call gives you a frontier model. AI also lowers the coding barrier twice over: it writes code for you, and it lets you instruct a system in plain English rather than a programming language. Open-source AI engineering projects (AutoGPT, Stable Diffusion web UI, LangChain, Ollama) collected more GitHub stars in two years than Bitcoin did in its lifetime and are on a trajectory to pass React and Vue.

Figure 1-6. Star history of popular AI engineering repositories

Foundation model use cases

The author surveyed 205 open-source AI applications with at least 500 GitHub stars and interviewed about 50 companies. The applications cluster into eight families. Most products combine several: a customer-support bot is a conversational bot that also aggregates information and automates workflow steps.

Figure 1-7. Distribution of use cases among open source AI applications

Category	Consumer examples	Enterprise examples
Coding	Code completion, code generation	Code review, test generation, documentation, migration
Image and video	Photo editing, design, video generation	Ad creative, presentations, localized marketing variants
Writing	Email, social posts, blog drafts	Copywriting, SEO, reports, memos
Education	Tutoring, essay feedback	Onboarding, upskilling
Conversational bots	General chatbot, AI companion	Customer support, product copilots
Information aggregation	Summarization, talk-to-your-docs	Market research, competitive intelligence
Data organization	Image search, personal knowledge base	Document processing, structured extraction
Workflow automation	Travel and event planning	Lead generation, invoicing, data entry

Enterprises are more willing to deploy internal-facing applications (knowledge management, internal copilots) than external-facing ones, because internal users tolerate mistakes, the data is already in-house, and a wrong answer to an employee is not a PR or legal incident.

Figure 1-8. Companies are more willing to deploy internal-facing applications

Coding

Coding is the most popular use case in every survey the author cites. GitHub Copilot's annual recurring revenue passed $100 million within two years of launch. Beyond autocomplete, AI handles structured-data extraction from web pages and PDFs, natural language to SQL or pandas, screenshot-to-code, language and framework migration, documentation, test generation and commit messages.

McKinsey's study found developers roughly twice as fast on documentation and substantially faster on code generation and refactoring (on the order of 25 to 50 percent), with little gain on highly complex tasks. Tool builders also report that AI is far stronger at frontend than backend work, where correctness depends on system state the model cannot see.

Figure 1-9. AI can help developers be significantly more productive

Image and video production

Generative models suit creative work because sampling many variations is cheap and "correct" is subjective. Midjourney reached about $200 million ARR at roughly 18 months old; in December 2023 half of the top ten free Graphics and Design apps on the Apple App Store had "AI" in their names. Enterprise uses: ad generation, promotional imagery, marketing variants by season and region, and fast A/B testing of creative assets.

Writing

Writing was the first mainstream use of LLMs. In the Noy and Zhang experiment (MIT, 2023), 453 college-educated professionals did writing tasks and half were given ChatGPT. Time fell about 40 percent and graded quality rose about 18 percent, and the gap between weaker and stronger writers narrowed, so the tool helped those with the least writing inclination most.

The same strength feeds a dark side. AI content farms produce SEO junk at scale, one site reportedly publishing 1,200 articles a day. NewsGuard found nearly 400 ads from 141 major brands running on AI-generated junk sites in June 2023.

Education

AI tutoring adapts to each learner, which is hard to do with humans at scale. Duolingo studied its four stages of course creation and found that lesson personalization benefits most from AI. Khan Academy ships an AI teaching assistant for students and a course assistant for teachers. If the threat is that AI replaces skills, the symmetric opportunity is that AI can tutor anyone in any skill.

Figure 1-10. Four stages of Duolingo course creation and AI impact

Conversational bots

For enterprises the dominant bot is customer support: it answers sooner than a human queue and cuts cost. Product copilots guide users through painful flows such as insurance claims, taxes or policy lookups. Beyond text, conversational interfaces include voice assistants and 3D characters (smart NPCs in games, retail and marketing avatars). AI companions are a large consumer category with their own safety and retention dynamics.

Information aggregation

Salesforce's 2023 Generative AI Snapshot found 74 percent of generative-AI users apply it to distill complex ideas and summarize information. When Instacart launched an internal prompt marketplace, one of the most used templates was "Fast Breakdown", which turns meeting notes, emails and Slack threads into facts, open questions and action items.

Data organization

Foundation models caption images and videos, match text queries to visuals, and extract structured fields from unstructured documents: receipts and tickets at the simple end, contracts, reports and charts at the hard end. The intelligent document processing (IDP) market is projected to reach about $12.8 billion by 2030 at roughly 33 percent annual growth.

Workflow automation

Agents are AI systems that plan and use tools to accomplish tasks. To book a restaurant an agent might search for the number, place a call and add the reservation to your calendar. Automation is the most ambitious category, because errors compound across steps and the system acts in the world rather than producing text for a human to check. Chapter 6 covers agent design and failure modes.

Planning AI applications

A demo takes a weekend; a profitable product takes much longer. Before building, decide why you are building and how.

Use-case evaluation

Start with why. The author frames three levels of urgency, from highest to lowest.

1. Existential threat. Competitors using AI will make you obsolete. Common in document processing, information aggregation, financial analysis, insurance and creative industries.
2. Missed opportunity. AI will not kill you, but it can cut acquisition cost, raise retention or improve internal operations, and skipping it leaves profit on the table.
3. Strategic hedging. You do not yet know where AI fits, but waiting risks being too late. Many companies have died by moving too slowly on a platform shift.

The role of AI and humans

Three dimensions determine requirements.

Dimension	Question	Implication
Critical vs complementary	Can the product work without the AI feature?	Critical AI must be accurate and reliable; complementary AI (Gmail smart compose, Grammarly) can be wrong more often
Reactive vs proactive	Does the AI respond to a request or anticipate a need?	Reactive features need low latency; proactive features can be precomputed but must clear a higher quality bar because nobody asked for them
Dynamic vs static	Does the system update from user feedback?	Dynamic features need per-user personalization and continual updates; static features ship one model for everyone, refreshed periodically

Microsoft's crawl, walk, run framing describes how the human role shrinks as quality rises:

1. Crawl. Human involvement is mandatory; AI drafts, humans decide.
2. Walk. AI interacts directly with internal employees.
3. Run. AI interacts directly with external users with increasing autonomy.

The staging can be driven by data. Start with AI suggesting responses to human support agents. If agents use 95 percent of suggestions verbatim for a class of simple requests, let customers receive AI answers directly for that class and keep humans on the rest.

Product defensibility

Competitive advantage in AI comes from technology, data or distribution. With everyone building on the same handful of foundation models, technology differentiation is thin. Distribution favors incumbents with existing user bases. Data is the nuanced one: a startup that ships first and captures usage data can keep improving a product that late entrants cannot match. The question to ask in planning is which of the three you will have in eighteen months, and whether a model provider could ship your product as a feature.

Setting expectations

Define success before building. The top-level metric is business impact. For a support chatbot that might be: the share of messages automated, how many more messages the team can handle, response-time reduction and labor cost saved.

Then set a usefulness threshold: how good must the system be before it is worth shipping? Track metrics in four groups.

Group	Examples
Quality	Accuracy, relevance, coherence, task completion
Latency	Time to first token (TTFT), time per output token (TPOT), total request latency
Cost	Cost per inference request, cost per resolved ticket
Other	Interpretability, fairness, safety

Thresholds change as the field moves. The cost of a fixed level of capability has fallen steeply year over year, which affects build-versus-buy and what counts as acceptable cost per request.

Figure 1-11. The cost of AI reasoning rapidly drops over time

Milestone planning and the last mile

Initial success misleads. LinkedIn's team reached 80 percent of their target experience in one month and then needed four more months to pass 95 percent. The first month made them badly underestimate the rest. The UltraChat authors put it the same way: 0 to 60 is easy, 60 to 100 is where the time goes.

Plan milestones around the hard part. The demo milestone is cheap; the budget belongs to evaluation, edge cases, guardrails and the feedback loop that gets you from 80 to 95 percent.

Maintenance

Building on foundation models means riding a fast-moving platform. Plan for several kinds of change.

Change	Example	Difficulty
Good but disruptive	Models get cheaper, context windows grow, outputs improve; your prompts and routing need re-tuning	Easy to moderate
Vendor risk	Provider deprecates a model, changes pricing or terms, has an outage	Moderate; mitigate with abstraction and fallbacks
Regulatory	New IP, privacy or compute rules (GDPR, export controls, the EU AI Act)	Hard
Fatal	A ruling makes the training data of your model a liability, or a provider exits	Potentially fatal

The best option today can be the worst tomorrow. Teams have built in-house models to save on API costs only to watch providers halve their prices three months later.

The AI engineering stack

Three layers

The stack has three layers. You can work at any of them, but most AI engineers live in the top one.

Layer	Responsibilities	Typical tools
Application development	Prompt engineering, context construction, evaluation, AI interface and UX	LangChain, LlamaIndex, evaluation frameworks, chat and agent UIs
Model development	Modeling and training, dataset engineering, finetuning, inference optimization	PyTorch, Hugging Face, vLLM, TensorRT-LLM, LoRA tooling
Infrastructure	Model serving, compute and data management, monitoring	GPU clusters, Kubernetes, vector databases, observability

Figure 1-14. Three layers of the AI engineering stack

The author tracked open-source repositories by category. In 2023 the fastest growth was in applications and application development; the infrastructure layer barely moved. That is expected: models and applications changed, but the core infrastructure needs (resource management, serving, monitoring) did not. Within the application layer, evaluation and agent tooling lead.

Figure 1-15. Cumulative count of repositories by category over time

Open source AI engineering repositories by category: evaluation and agent tooling lead the application layer

AI engineering versus ML engineering

Many companies list AI engineering and ML engineering under one umbrella, and the roles overlap. The differences are in what you do with the model, how you evaluate, what you ship and what data work looks like.

Figure 1-12. Many companies put AI engineering and ML engineering under the same umbrella

Dimension	Traditional ML engineering	AI engineering
Model source	You collect data and train a model	You use a model someone else trained and adapt it
Compute and latency	Small models, cheap inference	Huge models, GPU-bound inference, latency is a first-class constraint
Output type	Closed-ended (fraud or not, a price) and easy to score	Open-ended text, images, actions; much harder to evaluate
Starting point	Data, then model, then product	Product first, then data and model

Model adaptation splits into two families. Prompt-based techniques (prompt engineering, RAG, agents) change the input, not the weights: cheap to start, little data needed, and enough for many shipped products. Finetuning (full finetuning, parameter-efficient methods such as LoRA) changes the weights: more data and more engineering, but it can improve quality, latency and cost in ways prompting cannot.

Evaluation is harder for two reasons: outputs are open-ended, so there is no single reference answer, and models are general, so you must evaluate the tasks your users actually bring rather than one benchmark. Chapters 3 and 4 are entirely about this.

Interface becomes part of the engineering problem. Classical ML hid behind an existing product (a ranking score, a fraud flag). Foundation-model apps ship as standalone chat apps, browser extensions, Slack and Discord bots, IDE plugins and voice assistants, and the interface shapes what feedback you can collect.

Data work changes shape. The responsibilities inside model development shift as follows.

Category	Traditional ML	AI engineering
Modeling and training	Required: you design and train the model	Nice to have; you finetune or not at all
Dataset engineering	Feature engineering on mostly tabular data	Deduplication, tokenization, context retrieval, quality control on unstructured text and multimodal data
Inference optimization	Important	Critical, because the models are two or three orders of magnitude larger

Training vocabulary is worth getting right, because interviewers probe it.

Term	Meaning	Who does it	Resource intensity
Pre-training	Training from randomly initialized weights on raw text with self-supervision	Model developers	Extremely high; for InstructGPT, pre-training was up to 98 percent of total compute and data
Post-training	Further training (SFT, preference finetuning) to follow instructions and align with preferences	Model developers	Moderate
Finetuning	Continuing to train an already-trained model on your task data	Application developers	Low to moderate

Post-training and finetuning are the same operation performed by different parties: post-training when the model developer does it, finetuning when you do. Pre-training through finetuning is one spectrum with shared tooling.

Figure 1-13. AI engineering job listings

The full-stack shift

Because the model is available before the product, the workflow inverts. ML engineering went data, then model, then product. AI engineering goes product, then data, then model: build with an off-the-shelf model, ship to users, collect usage data, then decide whether finetuning or a custom model is justified. That rewards teams who iterate fast on the product surface.

Figure 1-16. The new AI engineering workflow rewards those who can iterate fast

The consequence is a pull toward full-stack and web engineering. Python still dominates model work, but JavaScript tooling (LangChain.js, Transformers.js, OpenAI's Node SDK, Vercel's AI SDK) grew quickly because the interface is where product differentiation happens. The strongest AI engineers combine solid software engineering with enough ML understanding to read a model card, design an evaluation and know when finetuning is worth it. Neither pure researchers nor pure frontend developers have that combination by default.

Design trade-offs: deciding what to build and how

The chapter's planning material compresses to a few decisions.

Decision	Options	Pick when
Build or wait	Existential threat, missed opportunity, hedge	Threat: build now and accept risk. Opportunity: pilot with measurable ROI. Hedge: small internal projects that teach the team
Where AI sits	Critical vs complementary	Complementary first if quality is uncertain; critical only once you can measure quality on your traffic
Human in the loop	Crawl, walk, run	Advance a stage only when acceptance-rate data for that request class supports it
Adaptation technique	Prompting, RAG, finetuning	Prompting until it stops working; RAG when the model lacks your facts; finetuning when you need behavior, format or latency prompting cannot give
Model sourcing	API vs open weights	API for speed and frontier quality; open weights for privacy, cost at scale and control (Chapter 4)
Moat	Technology, data, distribution	Be honest about which you have; usage data captured from a fast launch is the realistic startup moat

Practitioner checklist before Chapter 2:

• Classified the use case as existential, opportunity or hedge.
• Decided whether AI is critical or complementary, reactive or proactive, dynamic or static.
• Defined the human role and the data that would justify moving from crawl to walk to run.
• Written down business metrics and a usefulness threshold for quality, latency and cost.
• Budgeted the last mile, not just the demo.
• Assessed defensibility and vendor risk, and planned for model churn.
• Chosen the stack layer to start at (almost always application development).

Interview angle

• What is a foundation model and how is it different from a classical ML model? A foundation model is a large, self-supervised, general-purpose model (often multimodal) that many applications adapt rather than train. Classical models are trained per task on labeled data and produce closed-ended outputs. The shift moves engineering effort from training to adaptation and evaluation.
• Explain self-supervision and why it mattered for LLMs. The next-token objective makes every text sequence its own labeled dataset, so training data scales to the whole web without annotation. That removed the data bottleneck and made model size and compute the limiting factors, which money can buy.
• Masked vs autoregressive: when would you pick BERT over an LLM today? Masked models see both directions and are strong, cheap encoders for classification, extraction and embeddings. Autoregressive models generate. For a high-volume classification or retrieval task with labeled data, a finetuned encoder is often cheaper and faster; for anything generative or few-shot, use an autoregressive model.
• A PM wants an AI feature. How do you decide whether to build it? Ask why (threat, opportunity, hedge), where AI sits (critical or complementary), what the human role is, what business metric and usefulness threshold define success, and what the moat is. Then plan for the last mile: budget months, not a weekend, for 80 to 95 percent.
• What goes wrong when teams underestimate the last mile? The demo sets expectations at 80 percent quality; the remaining errors are the long tail that users notice. Without evaluation infrastructure, guardrails and feedback loops, quality plateaus and the project is cancelled after launch slips.
• Compare prompt-based adaptation and finetuning. Prompting and RAG keep weights frozen: fast, cheap, little data, easy to update, but bounded by context and the base model's behavior. Finetuning changes weights: more data and ops, but better format adherence, domain behavior, and often lower latency and cost per request because prompts shrink. Start with prompting; finetune when you have evidence it is needed.
• How is evaluating an LLM app different from evaluating a fraud model? Fraud is binary with ground truth. LLM outputs are open-ended, so you need references, rubrics, AI judges or comparative evaluation, and you must evaluate on your own traffic because the model is general. Evaluation becomes the hardest engineering task rather than a metric call.
• Design the rollout of a customer-support assistant. Crawl: AI drafts, agents edit, log acceptance rate per intent. Walk: expose AI directly to internal staff, add retrieval over the knowledge base, set TTFT and cost budgets. Run: for intents with 95 percent verbatim acceptance, let customers get AI answers with an escalate button; keep humans on everything else and monitor drift.
• Why does AI engineering pull toward full-stack work? The model exists before the product, so differentiation comes from the interface, feedback collection and iteration speed. Shipping chat, voice, browser-extension and IDE surfaces is web and product engineering, which is why JavaScript tooling grew alongside Python.

Pitfalls and misconceptions

• "Bigger model, better product." Product quality is dominated by task definition, context, evaluation and UX. A smaller model with good retrieval and prompts beats a frontier model with a vague prompt and no eval set.
• Confusing finetuning with post-training with pre-training. Pre-training is from scratch and costs the most by far. Post-training and finetuning are the same continued-training operation performed by the model developer and by you respectively.
• Treating the demo as proof of feasibility. The demo shows the easy 80 percent. Feasibility is proven when you have measured error rates on real traffic against a usefulness threshold.
• Shipping AI as a critical feature before measuring quality. Start complementary, where users tolerate errors, and promote the feature as evidence accumulates.
• Assuming technology is the moat. Everyone has the same models. Data flywheels and distribution are the durable advantages; a provider can turn your product into a feature.
• Building in-house to save API cost without modelling price trends. Provider prices have dropped repeatedly; a cost case that is marginal today becomes negative in a quarter.
• Ignoring tokenization. Token counts drive cost and context limits, differ sharply across languages, and explain many "the model cannot count letters" failures.
• Deploying external-facing first. Internal applications carry lower risk and faster feedback; most enterprises start there for good reasons.

Key terms

Term	Meaning
Language model	A model that assigns probabilities to token sequences, used to predict the next or missing token
Token	The unit a model reads and writes: a character, word or subword; vocabulary sizes run from tens of thousands to over 100,000
Masked language model	Predicts masked tokens using bidirectional context (BERT); suited to understanding tasks
Autoregressive language model	Predicts the next token from preceding tokens (GPT, Claude, Llama); the default meaning of LLM
Self-supervision	Training where labels come from the data itself, such as the next token in a sequence
Foundation model	A large general-purpose model, often multimodal, that applications are built on by adaptation
Multimodal model	A model that processes more than one data type, such as text and images
Model as a service	Access to a hosted model via API instead of training or hosting it yourself
Prompt engineering	Adapting a model by changing instructions, examples and context in the input
RAG	Retrieval-augmented generation: fetching relevant external data into the prompt at inference time
Finetuning	Further training an existing model on task data, updating its weights
Pre-training	Training a model from random initialization on large raw corpora
Post-training	Training after pre-training (SFT, preference finetuning) to make a model follow instructions and align with preferences
Critical vs complementary AI	Whether the product fails without the AI feature or merely gets worse
Crawl, walk, run	Staged automation: human-mandatory, internal-facing AI, external-facing AI
Usefulness threshold	The minimum quality, latency and cost at which the feature is worth shipping
Last mile problem	The disproportionate effort needed to move from demo quality to production quality
TTFT and TPOT	Time to first token and time per output token: the two latency metrics for streaming generation
Agent	An AI system that plans and uses tools to accomplish multi-step tasks

Further reading

• AI Engineering, Chapter 1 (Chip Huyen, O'Reilly, 2025). Companion resources: https://github.com/chiphuyen/aie-book
• Bommasani et al., "On the Opportunities and Risks of Foundation Models" (Stanford CRFM, 2021), the report that popularized the term: https://arxiv.org/abs/2108.07258
• Brown et al., "Language Models are Few-Shot Learners" (GPT-3, 2020), the paper that showed scale yields general-purpose behavior: https://arxiv.org/abs/2005.14165
• Devlin et al., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" (2018), the canonical masked language model: https://arxiv.org/abs/1810.04805
• Wang et al., "Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks" (2022): https://arxiv.org/abs/2204.07705
• Noy and Zhang, "Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence" (Science, 2023)
• Chip Huyen, "Building LLM applications for production" (2023): https://huyenchip.com/2023/04/11/llm-engineering.html
• OpenAI's tokenizer library, useful for seeing how text splits into tokens: https://github.com/openai/tiktoken