How AI Coding Assistants Read Your Prompt: Tokens, Context Windows and Non-Determinism

Learning outcome

You can explain, in terms an interviewer accepts, what actually happens between typing a request into Copilot Chat, Cursor, Claude Code or a chat window and getting C# back. You know what a token and a context window are, why the same prompt can give different answers, why the model "forgets" things, and why it can write a confident call to an API that does not exist. You can turn each of those facts into a concrete prompting habit.

This course targets .NET 8 (LTS) and C# 12, and every example also builds on .NET 10. Note that Microsoft support for .NET 8 ends on 10 November 2026, so new projects should start on .NET 10; the prompting techniques are the same.

Intuition

A large language model (LLM) is a very well-read autocomplete. It does not look up your repository, run your code or check NuGet unless a tool does that for it. It receives one long piece of text (the system instructions, the files the tool attached, the chat history and your message) and predicts the most plausible continuation, one token at a time.

Everything useful in prompting follows from that sentence:

• If a fact is not in the text it receives, the model fills the gap with what is typical. Typical .NET code is often older than yours (Startup.cs, Newtonsoft.Json, DateTime.Now).
• "Plausible" is not "correct". A method called string.IsNullOrEmptyOrWhiteSpace sounds right, so the model can produce it.
• The text it receives has a size limit, and the tool decides what goes in. You can influence that choice.

flowchart LR
    S["system / tool instructions"] --> W["context window (one long token sequence)"]
    R["repo instructions file (.github/copilot-instructions.md, AGENTS.md)"] --> W
    F["attached files, open editor tabs, search results"] --> W
    H["chat history"] --> W
    U["your prompt"] --> W
    W --> M["model predicts next token, again and again"]
    M --> O["answer: text, code, or a tool call"]

Deep dive

Tokens

Models read and write tokens, pieces of text that are often a short word or part of a word. A useful rule of thumb for English prose is about four characters per token; code, with its symbols and indentation, often costs more tokens per character. Tokens matter for three reasons:

• Cost and speed. API pricing and latency scale with input and output tokens.
• Limits. Every model has a maximum context window (the total tokens it can consider at once) and usually a separate cap on output tokens. A long generated file can be cut off mid-method when the output cap is hit; that is why "continue" sometimes produces a broken join.
• Attention. Even within the limit, a model uses some parts of a long context better than others. Burying the one important constraint in the middle of 4,000 lines of pasted code is a reliable way to have it ignored.

The context window is assembled for you

In an IDE assistant you rarely send the whole prompt yourself. The tool builds it: a hidden system prompt from the vendor, your repository instructions file, the current file and selection, other open tabs, results of a code search, and the conversation so far. In agent mode it also adds tool results such as terminal output and test failures.

Two consequences:

1. What you do not see still counts. Ten open tabs from an unrelated project can pull the answer the wrong way. Close them, or attach the exact files with the tool's file or #-reference feature.
2. Long chats degrade. Old turns are summarized or dropped when the window fills. A constraint you stated 40 messages ago may be gone. Start a new chat for a new task, and restate key constraints.

Why the same prompt gives different answers

Generation samples from a probability distribution. Settings such as temperature control how adventurous that sampling is. Chat products typically use non-zero values, and even at temperature 0 providers do not promise byte-identical output. For you this means:

• One good answer does not prove a prompt is good. One bad answer does not prove it is bad.
• For code you will ship, the test suite and the compiler decide, not the model's confidence.
• When you build prompts into an application (Module 4), you evaluate them over many cases, not one.

Training data has a date

A model's knowledge comes from training data with a cut-off, and the mix is weighted toward what was common on the internet. Recent APIs (for example newer TimeProvider usage, Microsoft.Extensions.AI, or a library's latest major version) may be missing or mixed up with older ones. Tools reduce this by searching docs or your codebase, but you should still:

• state the target framework and package versions in the prompt;
• ask the assistant to say when it is unsure whether a member exists;
• compile before you trust.

Hallucination is a feature of the method, not a bug you can prompt away

A hallucination is fluent output that is not grounded in fact: a NuGet package that does not exist, an overload with the wrong parameters, a configuration key nobody reads. Better prompts and grounding (giving the real interface, letting an agent search the repo) reduce it a lot. They do not remove it. Lesson 12 builds a verification routine around this.

What the model is good and bad at in .NET work

Worked example

The same question asked two ways in a fresh chat.

Before

How do I get the current time in C#?

Typical output:

You can use DateTime.Now:

var now = DateTime.Now;
Console.WriteLine(now);

Nothing is wrong with that as trivia, but it is the wrong answer for a server: local time, difficult to control in tests, ambiguous across time zones.

After

Context: ASP.NET Core on .NET 8, C# 12. Services are unit tested with xUnit.
I need "now" inside an OrderService to stamp CreatedAtUtc, and tests must control time.
Show the registration and the service constructor. If an API you mention is not in .NET 8, say so.

Typical output:

// Program.cs
builder.Services.AddSingleton(TimeProvider.System);

// OrderService.cs
public sealed class OrderService(TimeProvider clock)
{
    public DateTimeOffset Stamp() => clock.GetUtcNow();
}

The assistant also tends to mention that TimeProvider was added in .NET 8 and that tests can use FakeTimeProvider from the Microsoft.Extensions.TimeProvider.Testing package. Both statements are true. The difference came entirely from context: platform, version, purpose and the testing requirement.

Failure modes

• Treating the assistant as a search engine with a database of facts. It predicts text; check facts that matter.
• Ultra-long chats. Constraints scroll out of the window and quality quietly drops.
• Unrelated open files. The IDE tool attaches them and the answer drifts to that code style.
• Accepting the first answer because it compiled. Compiling proves the API exists, not that the logic is right.
• Pasting secrets. Anything in the prompt leaves your machine and may be logged under the vendor's retention policy. Check your company's approved tools and data rules.

Interview angle

"How do you use AI assistants day to day, and how do you stay safe?" A strong answer: "I treat the model as a fast, well-read pair programmer that predicts plausible code from whatever context it gets. So I give it the facts it can't know: target framework, packages, the real interfaces and the constraint that matters. I keep tasks small and chats short, and nothing is done until it compiles, the tests pass and I've reviewed it like a colleague's PR. I never paste secrets or customer data, and I only use tools the company has approved."

Practice questions

1. Why can an assistant produce a call to a method that does not exist?

Answer: It generates the most plausible continuation of the text it received. A method name that fits common patterns is plausible even when it is not part of the real API, unless the context or a tool grounds it.

2. What is a context window, and why should you start a new chat for a new task?

Answer: The maximum number of tokens the model can consider at once, including hidden instructions, files and history. Long chats get summarized or truncated, so earlier constraints can be lost, and old unrelated turns can bias the answer.

3. You ran a prompt once and the output was excellent. Is the prompt good?

Answer: Not proven. Output is sampled and can vary between runs and model versions. Judge a prompt over several runs or cases, and judge code by compiling and testing it.

4. Name two pieces of context that most change C# output quality.

Answer: Target framework and package versions, and the real types or interfaces involved (plus the app type: web API, worker, console).

5. Why does code cost more tokens than prose of the same length?

Answer: Symbols, indentation and identifiers split into more, shorter tokens than common English words.

Revision checklist

• An LLM predicts tokens from the context it receives; it does not know your repo unless a tool adds it.
• Context = tool system prompt + repo instructions + attached files + history + your message.
• Output varies between runs; tests and the compiler are the judge.
• Training data has a cut-off; state versions and ask it to flag uncertainty.
• Hallucination is reduced by grounding, never eliminated.
• Short chats, one task each, restated constraints.

Related on TechInterviewNotes

• Next lesson: Anatomy of a Strong Developer Prompt
• Course: AI/LLM Integration in .NET covers calling models from your own .NET code in depth.

References

• GitHub Docs: Adding repository custom instructions for GitHub Copilot
• Microsoft Learn: TimeProvider class
• .NET and .NET Core support policy