What the model actually does

A large language model (LLM) reads text as tokens. A token is a word or a piece of a word. At each step the model scores every possible next token and picks a likely one, with a little randomness. It adds that token to the text and repeats. A full answer is hundreds of these small guesses in a row.

The scores come from training. The model read a vast amount of text and adjusted billions of internal numbers, called parameters, to get better at guessing the next token. No neat table of facts comes out of that process. What the model knows is spread across those numbers as patterns.

Why this is not a search engine

A search engine finds a page and shows it to you. A plain LLM finds nothing. Ask it for a tax rate and it writes the answer that best fits the patterns it saw. For a common fact that appeared many times in training, the guess is usually right. For a rare or recent fact, it can be wrong and still read just as smoothly. The model has no built-in sense of which of its own claims are solid.

A model can look facts up only when the product gives it a way to, such as a search tool or a document placed in the prompt. Then it reads real text before it writes, and you can point to where the answer came from.

How builders use this

Ask one question of every AI feature: where does the answer come from? If it comes from the model's training, treat it as a draft that needs checking. If it comes from a source you supply, you can verify it and show it to the user.

The common mistake is to test a few famous facts, see correct answers and assume the model knows your domain. Test it on the rare, specific questions your users will actually ask. A useful habit is to ask the same question a few times. If the answers disagree, the model is guessing.

One request, one window

Every time your product calls a model, it sends one block of text. That block holds the instructions, the conversation so far, any documents and the user's new message. This is the context. The context window is the most text the model can take in one request, counted in tokens. The reply counts toward the limit as well. In English, a token is often a short word or part of a longer one. Text in Hindi or Tamil script often splits into more tokens for the same meaning, so it fills the window faster and costs more.

The model keeps no memory between requests. When a chat app seems to remember you, the app is resending earlier messages, or a summary of them, inside the window each time. Anything left out does not exist for the model on that turn.

Bigger windows still need choices

Modern models accept very long contexts, sometimes a whole book. That helps, but it does not remove the job of choosing. Every extra token adds cost and waiting time. Models can also miss a detail buried in a long, cluttered context, so more text is not always better.

When a conversation outgrows the window, apps drop the oldest messages or summarize them. Either way, some detail is lost. If that detail was an order number, the bot will ask for it again or, worse, guess.

How builders use this

For each task, write down what the model must see and in what order. Put stable instructions first and keep them short. Add only the documents and user facts that matter for this question.

The common mistake is assuming the model remembers yesterday's chat or a file the user shared last week. Unless your product passes it back in, it is gone.

Ask how your product handles a very long chat before users find out. Run one yourself and check whether the bot still knows what was said at the start.

Why models make things up

A hallucination is output that reads well but is false or unsupported. It follows from how LLMs work. The model is trained to produce likely text, and a confident, well-formed answer often looks more likely than "I don't know". When the real fact is missing from its context, it fills the gap with something that fits the pattern.

That is why hallucinations cluster around specifics such as names, numbers, dates, quotes and citations. A made-up court case has a plausible name and year, because real cases look like that. It also happens with questions built on a false premise, such as asking who wrote a book that does not exist.

Grounding and checks

Grounding means giving the model the source material and telling it to answer only from that. Retrieval-augmented generation (RAG) does this automatically. The system searches your documents, places the best passages in the context, and the model writes from them. Ask it to cite the passage it used and to say plainly when the answer is not there. If retrieval finds nothing relevant, the right output is a clear handoff to a person.

Grounding lowers the risk but does not remove it, since the model can still misread a passage. Match the safety net to the stakes. A wrong movie suggestion costs little. A legal or money answer needs a visible source and often a human review before it reaches the user.

The common mistake

Teams often test with questions they know the model handles well, then ship. Test instead with questions your documents cannot answer. A well-built product says it cannot find the answer. A risky one invents one.

Show users where each answer came from, with a link to the passage. People can then check what matters to them, and your team can log answers that lacked support and audit them later.

Two very different phases

Training is how a model gets made. A lab feeds it an enormous amount of text and runs thousands of specialized chips for weeks or months, adjusting the parameters to improve its predictions. Later rounds of fine-tuning teach it to follow instructions. The result is a fixed set of weights, the numbers that define the model.

Inference is using that finished model. Each time a user sends a message, the model runs to produce the reply, one token at a time. The weights do not change during inference. The model does not learn from your chat, unless the provider later uses such data in a new round of training. This is also why a model has a knowledge cutoff. Anything that happened after training ended is unknown to it, unless you supply it at inference time.

Where the product costs live

Most product teams never train a large model. They pay for inference, usually per token, for both the text they send in and the text that comes back. Long prompts and long answers add up fast. Bigger models usually cost more per token and respond more slowly.

Speed matters as much as cost. Users notice the wait before the first words appear, and streaming the reply word by word makes it feel shorter. A slightly smarter model that takes twice as long may lose to a faster one in a chat. For a nightly report, a slow and careful model is fine.

How builders use this

Estimate cost per task early: tokens in plus tokens out, multiplied by expected daily use. Pick the smallest model that passes your tests, and send only the hard cases to a larger one. The common mistake is to build the demo on the biggest model and only discover the running cost at launch.

Watch for features that call the model several times for one user action, such as an agent that plans and then checks its own work. Each call is another inference bill and another wait.

Two families of models

Predictive AI, often called classic machine learning, learns from labeled past examples to output a label or a number. Is this payment fraud? How many minutes until the rider arrives? The output comes from a fixed set or a numeric range, so you can measure accuracy directly. Credit scores and demand forecasts are both predictive.

Generative AI produces new content, such as an email draft or a block of code. LLMs are one kind of generative model, and image generators are another. Its output is open-ended, so there is rarely one right answer to compare against. Judging quality usually needs a rubric and some human review.

Picking the right tool

Use a predictive model when the answer is a decision or a score and you have past outcomes to learn from, like rows in a table. These models are usually cheaper to run and easier to test and audit. They do need labeled history, which some teams lack at the start.

Use a generative model when the output is language or media, or when the input is messy text that fixed rules cannot handle. An LLM can also classify through a prompt alone, which is handy for a quick prototype before you have training data. Expect a higher cost per call and less predictable output.

Most products use both

A food delivery app might use predictive models for delivery time and fraud checks, and generative ones to summarize reviews and run a support bot. The common mistake is reaching for an LLM for everything. If you have a large history of labeled outcomes and need a yes or no, a small predictive model is often the better fit.

When you scope a feature, ask two questions. Is the output a decision or a piece of content? Do we have past outcomes to learn from? The answers usually point to the right family, or to a mix of both.

Brief it like a new colleague

Imagine handing a task to a smart new joiner who knows nothing about your team. "Write a summary" would get you something generic. You would explain who will read it and what a good result looks like. A model needs the same briefing, because it cannot ask what you meant.

Vague prompts get average answers. The model fills each gap with the most typical choice, which is rarely the one your situation needs. A role line such as "You are a careful HR advisor" can help set the tone, but it cannot replace the actual details of the task.

What to spell out

Name the audience and the purpose. "For first-year students who have never seen a balance sheet" changes the output more than any adjective. Say what a great answer would let the reader do, such as decide or reply. Say what to include and what to leave out. Give the length and the format. If tone matters, describe it plainly, for example warm and brief.

Explain the reason behind a rule. "Keep it under 100 words because it appears in a phone notification" works better than a bare word count, since the model can apply the reason to cases you did not foresee. Put the main instruction where it cannot be missed. Mark where the instructions end and the material begins, for example with headings or tags.

The common mistake

People write prompts in their own shorthand. "Make it better" or "use the usual format" means something to you and nothing to the model. Before blaming the model, read your prompt as an outsider. If a new colleague could not do the job from it, the model probably cannot either.

A quick test: give your prompt to a classmate and ask what they would produce. If their answer differs from what you pictured, the prompt is unclear to the model too.

What few-shot means

A zero-shot prompt describes the task and nothing more. A few-shot prompt adds a handful of worked examples, each an input plus the exact output you want. The model picks up the pattern from these examples and applies it to the new input. Nothing is retrained, and the examples shape only that request. This is sometimes called in-context learning.

One example is called one-shot. Most tasks do well with two to five. Each example costs tokens on every call, and past a point extra examples add little.

What examples teach well

Examples are best at things that are tedious to describe: an output format, a tone of voice, the length of each part, or how to label tricky cases. If you want ticket summaries that open with the customer's goal in five words, show two and the model will usually follow. Examples also show what to leave out, which is hard to say in words.

Choose examples with care. Make them varied, so the model learns the real rule and not one surface detail. If every example is about refunds, the model may treat every ticket as a refund. Include an edge case, such as a message in Hinglish or one with two complaints. Wrap each example in clear markers, like tags, so the model does not mix them up with the real input.

The common mistake

Examples are strong signals, so their flaws get copied. A wrong label or an overly long answer in one example tends to show up in the outputs. Check examples as carefully as the instruction. When outputs go wrong, look at the examples before rewriting everything else.

Another mistake is using examples that are cleaner than real inputs. Pick them from real data, with personal details removed, so they look like what the model will face. Keep them with your test set, so any change to them gets tested too.

Context beats memory

For anything specific to your company or anything recent, the model's training is the wrong place to look. It may never have seen the fact, or it may have seen an old version. The fix is simple to state. Put the facts the answer should rest on inside the context, and tell the model to use them. Pasting the latest price list beats any instruction to "be accurate".

For a one-off task you can paste the document yourself. In a product, a retrieval step does it. The system searches your content for passages that match the question and adds the best few to the prompt before the model writes. This pattern is called retrieval-augmented generation, or RAG. Its quality depends on how documents are split into chunks and how well the search matches the meaning of a question.

Context engineering

Context engineering is the wider job of deciding everything the model sees on each request. That includes instructions, retrieved documents, user details, conversation history and tool results. Each piece should earn its place. Too little context and the model guesses. Too much and the key fact gets lost while cost and delay go up.

Write instructions that tie the answer to the material. Ask the model to answer only from the provided documents and to cite the passage it used. Tell it exactly what to say when the documents do not contain the answer. Label each document with its title and date, so the model can cite it and prefer the newest version.

The common mistake

Teams spend days polishing prompt wording while the retrieval step returns the wrong passages. If the right document never reaches the context, no wording can save the answer. When an answer is wrong, first look at what the model was actually shown.

Make this a routine. For a sample of real questions, read the passages that retrieval returned. If the right passage is missing or buried, fix the search or the documents before touching the prompt.

Why structure helps

Free text is fine for a person reading a reply. It becomes a problem when a program has to use the result, because a program needs the same keys in the same place every time. A sentence like "the customer seems mostly unhappy" cannot go into a spreadsheet column. A field like "sentiment": "negative" can.

Structure helps people too. A table with the same columns every time is far easier to scan and check than ten paragraphs written in slightly different ways. Structure also forces clear thinking. If you cannot name the fields you need, the task is not defined yet.

How to ask for it

For people, ask for headings or a table with named columns. For programs, ask for JSON with exact field names. List the allowed values for each field, so the model cannot invent a new category. Say what to do when information is missing, for example return null instead of guessing. If the shape is unusual, show one complete example of it.

Many model APIs now offer a structured output mode. You supply a schema, which is a formal description of the fields, and the API keeps the reply in that shape. Use it when it is available. Either way, validate the output in code before the next step uses it, and log any failures. Plan what happens when a reply fails, such as one retry and then a human queue.

The common mistake

Valid JSON is not the same as a correct answer. The format can be perfect while the sentiment is wrong. Check the values as well as the shape, on a sample of real inputs. It also helps to keep a short "reason" field, so a person can audit a decision later.

Another trap is a field so loose that it hides problems, such as a free-text "notes" field where the real answer ends up. Keep the fields that matter as tight as you can.

Start with a small test set

Collect 20 to 50 real inputs your feature will face. Include the common cases and the awkward ones, such as inputs the feature should refuse. For each, note what a good output must contain. This is your test set. It turns "the prompt feels better" into "the prompt passes 41 of 50".

Keep the test set in a shared sheet, one row per input, with the expected points beside it. Then anyone on the team can rerun it, not only the person who wrote the prompt.

Change one thing at a time

Run the current prompt on the whole set and score the outputs. Then change one thing, such as adding an example or tightening the format, and rerun everything. If you change several things together, you cannot tell which one helped, or which one broke a case that used to pass. Rerun the set when the model version changes too, since an update can shift behaviour.

Score with simple checks first. Some can run in code, like whether the output is valid JSON or under 80 words. Others need judgment, so write a short rubric and score those by hand. Later, a second model can apply the rubric at scale, once you have confirmed it agrees with your own scores.

Where evals begin

This habit is the start of evals, the tests that tell an AI team whether a change made things better or worse. Read real outputs and name the types of failure you see. Turn each type into a check, and add new cases whenever users find new failures. For a PM, this set becomes the definition of done for an AI feature.

The common mistake is judging a prompt by a few hand-picked tries. Outputs vary from run to run, and a change that fixes one example can quietly break five others. Only the full set tells you. Save each version's scores so you can show progress over time.

Two kinds of memory

A predictive model learns from training data: past examples with known outcomes, like loan applications marked repaid or defaulted. Whatever patterns sit in that data, good or bad, become the model's behaviour. If the history has gaps, the model has the same gaps. If past approvals leaned toward one group, the model can learn that lean too.

An LLM app usually works differently. The base model already knows language, so your data does its work at answer time. The app retrieves documents, such as policies or past tickets, and the model answers from them. Here your document collection sets the quality of the answers. Many teams also keep a set of real questions with known good answers, to test the app against.

What makes data good

Good data matches the real task and covers the cases users actually bring, including rare ones. It also has to be current. A refund bot grounded on last year's policy will give last year's answer with full confidence. A fraud model that saw few examples from small towns will be weaker there. Check where the data came from and whether it reflects the users you plan to serve.

Fine-tuning sits in between. You train an existing model further on your own examples to teach it a style or a narrow skill. It is slower to update than a document collection, so use it to shape how the model responds and use retrieval for the facts it should use.

The common mistake

Teams compare models for weeks and treat data as someone else's job. In practice, fixing a stale document or adding missing examples often helps more than switching models. Ask early what data this feature will learn from or read from, and who will keep it current.

List each feature's data sources in the spec, next to the model choice. A data source with no owner is a risk to flag early, before it shows up as a wrong answer in front of users.

Why labels disagree

A label is the answer you want a model to learn, like spam or not spam, or the right category for a support ticket. Labels for training predictive models, and for testing LLM apps, usually come from people. People read the same item differently. Is "where is my refund??" a refund request or a complaint? Without a rule, each labeler decides alone and the data fills with noise.

Some labeling is now done by LLMs, with people checking a sample. The same rulebook applies, because a model needs clear definitions as much as a person does.

Write the rulebook

Labeling guidelines define each label in plain words and show examples, including tricky edge cases. They also say what to do when an item fits two labels or none. A "not sure" option lets labelers flag hard cases instead of guessing. Keep the rulebook short enough that people actually use it, and update it when new cases appear.

Before labeling thousands of items, run a pilot. Give the same 100 items to two or three labelers and compare. The share of items where they agree, or a statistic like Cohen's kappa that corrects for chance agreement, shows whether the rules are clear. Low agreement usually points to unclear guidelines rather than careless people. Read the disagreements and fix the rules before you scale up.

How builders use this

Treat the rulebook as a product spec for your data. A PM often writes the first draft, because deciding what counts as "urgent" is a product decision. The common mistake is to hand labeling to a vendor with a one-line instruction, then find months later that different batches used the labels in different ways.

Keep a small gold set of items with agreed labels and mix some into every batch. It shows whether quality holds as the work scales. Version the rulebook too, since labels made under different rules mean different things.

What versioning means for data

Engineers version code so they can see every change and undo a bad one. AI systems need the same for data. A data version is a frozen, named snapshot. It could be the exact rows used to train a model, or the exact documents in a search index on a given date, along with the labeling guidelines in force at the time. Prompts deserve versions too, since a prompt change can shift behaviour as much as new data.

Lineage is the record that links these pieces. It tells you, for example, that model 7 was trained on snapshot 12 with a known version of the code. It also records where each source came from and how it was cleaned. Well-built pipelines record this automatically every time they run.

Why it pays off

Versions let you explain a result. If a user complains about an answer, you can see which documents the model had that day, even months later. They also make comparisons fair, since two models scored on different test data cannot be compared.

Most of all, versions let you roll back. If a new index or retrained model does worse, you switch to the last good version while you investigate. Without versions, a rollback means rebuilding from memory under pressure. Lineage also answers the audit question of where a system's data came from.

How builders use this

Set a simple rule: no model or index goes live without a recorded data version. Store snapshots instead of overwriting tables in place, and name them so a non-engineer can follow. The common mistake is "we just refreshed the data" with no record of what was there before, which turns every regression into guesswork.

Tools exist for this, from data version control systems to the metadata stores in ML platforms. For a small team, a dated folder per snapshot and a change log are a good start.

Four questions for every dataset

Data governance is the set of rules and roles that decide how data is collected and used. For an AI team it comes down to four questions per dataset. Who owns it and answers for its quality? What did people agree to when it was collected? How long do we keep it? Who can see it, including which models and vendors?

Write the answers where the team can find them, such as a one-page data card. If nobody can answer one of these questions, the dataset is not ready for an AI feature.

What India's DPDP Act asks

The Digital Personal Data Protection Act, 2023 sets the rules for handling digital personal data in India. A business that decides why and how personal data is processed is called a data fiduciary. It must give people a notice saying what data it collects and for what purpose. Consent must be free, specific, informed, unconditional and unambiguous, and people can withdraw it as easily as they gave it.

The Act requires reasonable security safeguards, and a breach must be reported to the Data Protection Board and to the affected people. Data must be erased once its purpose is served or consent is withdrawn, unless a law requires keeping it. The government notified the DPDP Rules in November 2025 with phased timelines, so check which duties apply on your date.

What this means for AI builders

Consent is tied to a specified purpose. Phone numbers collected to deliver orders may not cover training a marketing model. Before reusing personal data for AI, check the original notice and consent. Mask personal details the model does not need, and find out what your model vendor does with the data you send.

The common mistake is treating governance as paperwork for later. Adding consent and deletion to a live model and its training data is far harder than designing them in from the start.

What drift is

Drift is the growing gap between the data a model learned from and the data it sees in use. Data drift is when the inputs change, such as new kinds of users or new slang. Concept drift is when the link between inputs and the right answer changes. The same signals start to mean something else, as when fraudsters switch tactics. Some drift is seasonal, like festival shopping, and some is permanent, like a new way to pay.

LLM apps drift too. Documents in your index go out of date, and user questions move toward topics you never tested.

How to catch it

Drift is quiet. Nothing crashes while accuracy slides a little each week, so you have to look for it. Compare live inputs with the training data, for example the mix of cities or message lengths. Track outcomes where you can get them, like whether predicted delivery times matched actual ones. Some outcomes arrive late, such as whether a loan is repaid, so treat input changes as an early warning. Check results for important slices, not only the overall average.

Also watch the model's age. Google's ML guidance suggests tracking the time since a model was last retrained, with an alert once it passes a threshold. A model nobody has retrained in a year deserves a close look.

What to do about it

Plan retraining or index refreshes from day one, with an owner and a schedule. Keep a recent labeled sample to test against. When something big changes, like a festival season or a new rule, check the model on fresh data before trusting it. The common mistake is treating launch as the finish line.

Decide in advance who looks into an alert and what size of drop triggers retraining. Retraining is not always the fix. Sometimes an input pipeline changed format and needs repair. When your LLM provider updates its model, rerun your tests before trusting old results.

What problem first means

A product exists to change something for a person. Working problem first means you can name that person and describe their pain in their own words. You also know how often it hits them. Only then do you talk about features.

The opposite is a solution in search of a problem. A team gets excited about a technology, such as a chatbot, and then hunts for a place to put it. These products often demo well and then sit unused.

For a PM, the problem is also the brief. Engineers and designers often propose better solutions than yours when they know which pain they are solving. A feature list gives them nothing to improve on.

Writing a problem statement

Keep it short and free of any solution. Say who is affected, what goes wrong for them and what it costs. NN/g advises keeping a problem statement to one main problem and leaving solutions out.

A quick test: could two very different solutions answer it? "Users need an AI assistant" fails, because the answer is already baked in. "First-year students miss hostel fee deadlines because notices are scattered across WhatsApp groups" passes. A single notice board could fix it. So could SMS reminders.

Back the statement with evidence, such as user quotes or a count of support tickets. Five interviews where people describe the same pain without prompting beat a confident guess made in a meeting.

Why AI makes this harder

Models can do so many things that the list of possible features never ends. The problem statement is your filter. If a feature does not ease the pain you named, it waits.

Try this with your team. Ask everyone to write the problem in one sentence, alone. If the sentences differ a lot, you are not ready to build.

The common mistake is skipping this step because the solution feels obvious. Teams then argue about features with no shared way to judge them. A clear problem gives everyone the same yardstick.

Four questions before you build

Marty Cagan of SVPG sorts product risk into four questions. Value: will customers buy it or choose to use it? Usability: can they work out how to use it? Feasibility: can the team build it with the time and skills it has? Business viability: does it work for the rest of the business, such as legal and cost?

His advice is to tackle these risks early, in discovery, before engineers spend weeks in delivery. Each risk has a natural owner. The PM owns value and viability, the designer owns usability and the tech lead owns feasibility. The whole team still works on all four together.

How AI shifts the risks

Feasibility now includes a new question: is the model good enough, often enough? You only learn that by testing with real inputs. Viability now includes the running cost of every model call and the harm a wrong answer could do.

Viability also covers data. If the feature needs personal data, check consent and storage rules with legal early, before the design depends on it.

Usability changes too. People need to understand that the output can be wrong and know what to do about it. Value risk stays the biggest. A clever model does not make anyone want the feature.

Test each risk cheaply

Match the test to the risk. A landing page or a fake button tests value. A paper prototype tests usability. A one-day spike with real data tests feasibility. A short call with finance or legal tests viability.

Go after the biggest risk first. If nobody is sure people want the feature, tuning the model is wasted effort. If demand is clear but the model fails on many of your test inputs, feasibility comes first.

The common mistake is testing only feasibility, because building is what teams know best. A working prototype proves you can build it. It tells you nothing about whether anyone will care.

Why opinions mislead

Ask a friend "Would you use an app that splits hostel bills?" and they will probably say yes. Saying yes costs them nothing and keeps things pleasant. Hypothetical questions invite hypothetical answers.

Rob Fitzpatrick's book The Mom Test is built on this problem. Its goal is questions so grounded in real life that even your mother, who wants to be kind, could not mislead you. Compliments and guesses about the future feel like data. Past behaviour is far stronger evidence, because it already happened and cost the person something.

Questions that work

Talk about their life, not your idea. YC partner Eric Migicovsky suggests questions like these. What is the hardest part of doing this? Tell me about the last time it happened. What have you already tried? What do you dislike about those fixes?

Then dig into specifics. What did you do next? How much time or money did it cost you? If they have never tried to fix the problem, it may not hurt much.

Also ask who else is involved. A student may feel the pain while a parent pays for the fix. Find out who decides and who pays, because a product needs both to say yes.

Running the interview

Listen far more than you talk. Bring a partner to take notes so you can focus on the person. Write down exact quotes instead of your summary of them. Do not pitch your idea during the interview. Once people see it, they start being polite again.

After five or six interviews, look for patterns. One person's complaint is an anecdote. The same story from most of them is a signal worth building on.

The common mistake is asking "Would you use this?" or "How much would you pay?" These questions feel efficient. The answers are guesses about a future the person has not lived, so you cannot trust them.

Output, outcome, impact

Output is what the team makes, such as features and releases. An outcome is a change in what people do, such as more users finishing setup or fewer customers raising tickets. Impact is the business result that follows, such as revenue or retention.

Josh Seiden, author of Outcomes Over Output, describes outcomes as changes in customer behaviour that drive business results. That framing helps because behaviour is something a team can observe and move within weeks. Impact takes longer and has many causes.

Think of it as a chain. Output leads to an outcome, and the outcome leads to impact. If you cannot explain how a feature changes behaviour, you cannot explain how it will help the business.

Working on outcomes

Start each piece of work with the behaviour you want to change. "Launch AI search" becomes "more shoppers find what they want in their first search". Then pick a metric for that behaviour and set a target.

Now each feature is a bet. If the first version does not move the metric, you change the approach instead of moving to the next item on the list. SVPG calls teams that work this way empowered product teams. Feature teams, by contrast, are judged on shipping what they were told to build.

Good outcome metrics sit close to the feature and move quickly. "Share of new sellers who list a second item within a week" responds in days. Annual revenue moves for too many reasons to judge one feature.

The common mistake

Teams report how many features shipped or how many people clicked a new button. A click shows curiosity. It does not show value.

For AI features, watch whether people keep the output. Do they send the suggested reply, or delete it and type their own? Acceptance and edit rates often tell you more than usage counts.

If the real outcome takes months to show, pick an early sign that predicts it, such as repeat use in the first week. Then confirm later that the early sign and the real outcome moved together.

What goes in a short PRD

A PRD, or product requirements document, explains what you are building and why. Long ones go unread, so aim for one or two pages. Some teams call the short version a one-pager. Five parts carry most of the value.

Problem: the pain, in the user's words, with evidence. Users: who has it and in what situation. Scope: what this release will do. Success metric: the outcome you will measure, with a target. Out of scope: what you agreed not to do this time.

For an AI feature, add a short quality section. Include a few examples of good and bad output, and say what should happen when the model is unsure.

How PMs use it

Write the first draft before the kickoff, then invite engineers and designers to poke holes in it. Share it where the team already works so people can comment. Keep open questions in a list at the end so nobody pretends they are settled. Atlassian's PRD template has sections for open questions and for what you are not doing.

For AI work, the PRD is also where the team agrees on risk. Note what the model must never do, such as inventing a company name or showing another student's data, and how you will check for it.

Keep the document alive. When a decision changes, update it and note the date. A PRD is the team's shared memory, and it should be more reliable than anyone's recollection of a meeting.

The common mistakes

The first mistake is writing a solution spec with no problem section. Without the problem, nobody can judge trade-offs when time runs short.

The second is skipping out of scope. Without it, every review adds one more small thing and the release slips. Naming what you will not do is often the most useful line in the document.

The third is a success metric with no number. "Improve engagement" can never fail. "Half of applicants send a tailored letter by the end of placement season" can, so it tells you whether the feature worked.