Docker Image Layers, the Build Cache and .dockerignore for .NET

Learning outcome

You can read an image as a stack of layers, predict which Dockerfile steps the build cache will reuse, and order a .NET Dockerfile so that a one-line code change rebuilds in seconds instead of minutes. You know what the build context is, why .dockerignore matters for speed and for secrets, and how to prove all of it with docker history and --progress=plain.

Intuition

Think of an image as a stack of transparent sheets. Each Dockerfile instruction that changes files (RUN, COPY, ADD) lays a new sheet on top, and the stack is what the container sees. The builder remembers every sheet it has made, keyed by "the sheet below plus this instruction plus its inputs". If nothing in that key changed, it reuses the sheet. As soon as one sheet must be redrawn, every sheet above it must be redrawn too.

So the whole art is ordering: put what rarely changes (base image, NuGet restore) at the bottom and what changes on every commit (your C# source) at the top.

flowchart TB
    A["FROM sdk:10.0 (shared, pulled once)"] --> B["COPY *.csproj (changes when packages change)"]
    B --> C["RUN dotnet restore (slow: network)"]
    C --> D["COPY source (changes every commit)"]
    D --> E["RUN dotnet publish (fast-ish: compile only)"]
    style C fill:#e8f4e8
    style D fill:#fdeaea

Deep dive

Layers are content-addressed diffs

• Every layer is a tarball of file changes (added, modified, deleted) relative to the layer below, identified by a SHA-256 digest.
• Layers are shared. Ten images built FROM mcr.microsoft.com/dotnet/aspnet:10.0 store the runtime layers once on disk and push or pull them once per registry.
• Instructions such as ENV, EXPOSE, USER and ENTRYPOINT only change the image config and show up in docker history as 0-byte entries. WORKDIR also creates the directory if it does not exist.
• Deleting a file in a later layer does not shrink the image. The file still exists in the earlier layer; the new layer only hides it with a "whiteout". This is why a secret copied in one step and removed in the next is still in the image (Lesson 7), and why apt-get install and the cache cleanup belong in the same RUN.

How the cache decides

For each instruction, BuildKit (the default builder since Docker Engine 23) computes a cache key from:

• the parent layer;
• the instruction text (for RUN, the exact command string);
• for COPY/ADD, a checksum of the contents and metadata of the copied files (timestamps are ignored).

If the key matches a stored result, the step prints CACHED. If it misses, the step runs and every later step in that stage misses too. Two consequences matter:

1. RUN dotnet restore is cached as long as the files copied before it are unchanged. Copy only the .csproj (and Directory.Packages.props, nuget.config, global.json if you have them) before restoring, then the rest of the source.
2. RUN apt-get update is cached forever because its command text never changes. That is why you combine apt-get update && apt-get install in one RUN, and why CI periodically builds with --pull (fresh base image) or --no-cache.

Multi-project solutions

For a solution with several projects, copy every .csproj into its folder before restoring:

COPY *.sln Directory.Build.props Directory.Packages.props ./
COPY src/Shop.Api/Shop.Api.csproj src/Shop.Api/
COPY src/Shop.Core/Shop.Core.csproj src/Shop.Core/
RUN dotnet restore src/Shop.Api/Shop.Api.csproj
COPY src/ src/
RUN dotnet publish src/Shop.Api/Shop.Api.csproj -c Release -o /app/publish --no-restore

Listing projects by hand is tedious but explicit. Generators and the dotnet publish /t:PublishContainer route (Lesson 6) avoid this.

The build context and .dockerignore

docker build . sends the build context (the directory .) to the builder. Only files in the context can be copied. .dockerignore excludes paths from the context entirely, which:

• keeps bin/, obj/, .git/, node_modules/ and test output out of uploads (speed);
• stops a host's obj/project.assets.json, which holds host-specific paths, from leaking into COPY . . and confusing the container's restore;
• keeps .env files, secrets/ and user files out of any layer (safety);
• prevents irrelevant edits, such as a README change, from invalidating COPY . ..

The .dockerignore used in this course:

**/bin/
**/obj/
**/.vs/
**/.vscode/
**/*.user
**/node_modules/
.git/
.env
**/secrets/
Dockerfile*
compose*.yaml

Cache mounts for NuGet

Restore output lives in a layer, so when the .csproj changes, every package is downloaded again. BuildKit cache mounts keep the NuGet folder between builds without putting it in the image:

RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \
    dotnet restore Shop.Api/Shop.Api.csproj

Mount the same NuGet cache in subsequent build/publish steps that consume the packages; a cache mount is not committed into the image layer. It helps on a long-lived builder. On ephemeral CI runners, layer-cache export (--cache-to/--cache-from, Lesson 17) is separate from preserving cache-mount contents; the GitHub Actions cache does not preserve those mounts by default.

Code example: watching the cache work

The Dockerfile from Lesson 3 is built repeatedly while the inputs change. A small script summarises docker build --progress=plain into one line per step: CACHED, or how long the step took. NuGet restore is slow on the network of the machine these were captured on, which makes the difference easy to see.

# Layers of the image built in this lesson (newest first)
$ docker history shop-api:1.0 --format '{{.Size}}\t{{.CreatedBy}}' | cut -c1-95
0B	ENTRYPOINT ["dotnet" "Shop.Api.dll"]
0B	EXPOSE [8080/tcp]
0B	USER 1654
4.14MB	COPY /app/publish . # buildkit
0B	WORKDIR /app
27.1MB	COPY /dotnet /usr/share/dotnet # buildkit
0B	ENV ASPNET_VERSION=10.0.13
24B	RUN /bin/sh -c ln -s /usr/share/dotnet/dotne…
82.8MB	COPY /dotnet /usr/share/dotnet # buildkit
0B	ENV DOTNET_VERSION=10.0.13
9.64kB	RUN /bin/sh -c groupadd         --gid=$APP_U…
42MB	RUN /bin/sh -c apt-get update     && apt-get…
0B	ENV APP_UID=1654 ASPNETCORE_HTTP_PORTS=8080 …
0B	/bin/sh -c #(nop)  CMD ["/bin/bash"]
78.2MB	/bin/sh -c #(nop) ADD file:7121eb4a5ba391efb…
0B	/bin/sh -c #(nop)  LABEL org.opencontainers.…
0B	/bin/sh -c #(nop)  ARG LAUNCHPAD_BUILD_ARCH
0B	/bin/sh -c #(nop)  ARG RELEASE

# 1) Rebuild with nothing changed
$ docker build -t shop-api:1.0 .
build context sent: 157B
 DONE 0.0s  [build 1/6] FROM mcr.microsoft.com/dotnet/sdk:10.0
    CACHED  [build 2/6] WORKDIR /src
    CACHED  [build 3/6] COPY Shop.Api/Shop.Api.csproj Shop.Api/
    CACHED  [build 4/6] RUN dotnet restore Shop.Api/Shop.Api.csproj
    CACHED  [build 5/6] COPY Shop.Api/ Shop.Api/
    CACHED  [build 6/6] RUN dotnet publish Shop.Api/Shop.Api.csproj -c Release -o /app/pub
 DONE 0.0s  [final 1/3] FROM mcr.microsoft.com/dotnet/aspnet:10.0
    CACHED  [final 2/3] WORKDIR /app
    CACHED  [final 3/3] COPY --from=build /app/publish .
total 0.3s

# 2) Change one line of C# and rebuild
$ docker build -t shop-api:1.1 .
build context sent: 5.82kB
 DONE 0.0s  [build 1/6] FROM mcr.microsoft.com/dotnet/sdk:10.0
    CACHED  [build 2/6] WORKDIR /src
    CACHED  [build 3/6] COPY Shop.Api/Shop.Api.csproj Shop.Api/
    CACHED  [build 4/6] RUN dotnet restore Shop.Api/Shop.Api.csproj
 DONE 0.0s  [build 5/6] COPY Shop.Api/ Shop.Api/
 DONE 3.3s  [build 6/6] RUN dotnet publish Shop.Api/Shop.Api.csproj -c Release -o /app/pub
 DONE 0.0s  [final 1/3] FROM mcr.microsoft.com/dotnet/aspnet:10.0
    CACHED  [final 2/3] WORKDIR /app
 DONE 0.0s  [final 3/3] COPY --from=build /app/publish .
total 3.7s

# 3) Add a NuGet package (the .csproj changes) and rebuild
$ docker build -t shop-api:1.2 .
build context sent: 862B
 DONE 0.0s  [build 1/6] FROM mcr.microsoft.com/dotnet/sdk:10.0
    CACHED  [build 2/6] WORKDIR /src
 DONE 0.0s  [build 3/6] COPY Shop.Api/Shop.Api.csproj Shop.Api/
DONE 121.6s  [build 4/6] RUN dotnet restore Shop.Api/Shop.Api.csproj
 DONE 0.0s  [build 5/6] COPY Shop.Api/ Shop.Api/
 DONE 3.2s  [build 6/6] RUN dotnet publish Shop.Api/Shop.Api.csproj -c Release -o /app/pub
 DONE 0.0s  [final 1/3] FROM mcr.microsoft.com/dotnet/aspnet:10.0
    CACHED  [final 2/3] WORKDIR /app
 DONE 0.0s  [final 3/3] COPY --from=build /app/publish .
total 125.2s

# 4) The naive single-stage Dockerfile after the same kind of one-line C# change
$ docker build -f Dockerfile.naive -t shop-api:naive .
build context sent: 5.86kB
 DONE 0.0s  [1/5] FROM mcr.microsoft.com/dotnet/sdk:10.0
    CACHED  [2/5] WORKDIR /src
 DONE 0.0s  [3/5] COPY . .
DONE 124.2s  [4/5] RUN dotnet publish Shop.Api/Shop.Api.csproj -c Release -o /app
 DONE 0.0s  [5/5] WORKDIR /app
total 124.6s

# 5) Build context with and without .dockerignore (bin/ and obj/ exist locally)
$ du -sh Shop.Api/bin Shop.Api/obj
4.0M	Shop.Api/bin
340K	Shop.Api/obj
$ docker build -f Dockerfile.naive -t shop-api:noignore .   # .dockerignore temporarily removed
build context sent: 4.40MB
$ docker build -f Dockerfile.naive -t shop-api:naive .      # .dockerignore restored
build context sent: 307B
$ docker image ls shop-api --format 'table {{.Tag}}\t{{.Size}}'
TAG        SIZE
naive      960MB
noignore   961MB
1.0        234MB

What to notice:

• History: the app itself is the 4.14 MB COPY /app/publish layer. The rest of the 234 MB is the Ubuntu base (78 MB), native dependencies (42 MB) and the .NET and ASP.NET Core runtimes (83 + 27 MB). Those layers are shared by every .NET 10 image on the machine.
• 1) No change: every step CACHED, 0.3 s.
• 2) One C# line: restore stays cached; only COPY source and publish run. 3.7 s.
• 3) New package: the .csproj changed, so restore and everything after it run again. 125 s, almost all of it in restore.
• 4) Naive Dockerfile (COPY . . before restore): the same one-line change makes dotnet publish restore from scratch. 125 s instead of 3.7 s on every commit. The image is also 960 MB, because the SDK ships in it (Lesson 3 fixes that).
• 5) Context: without .dockerignore the local bin/ and obj/ folders go into the context (4.4 MB here; hundreds of MB in real repositories with test results and node_modules). BuildKit sends the context incrementally, so a rebuild sends only changed files (307 B).

Failure modes

• COPY . . before restore. Every commit re-downloads all packages. Copy project files first.
• A missing .dockerignore. Slow uploads, host obj/ in the image, and secrets or .git history baked into layers.
• Cleaning up in a later layer. RUN rm -rf /var/lib/apt/lists/* in its own step saves nothing. Clean up in the same RUN that created the files.
• Stale base images. A cached FROM keeps an old, unpatched base forever on that machine. CI should build with --pull and rebuild regularly.
• Busting the cache by accident. ARG BUILD_DATE or a COPY of a file that changes on every build (a version stamp) early in the Dockerfile invalidates everything below it. Put such steps last.
• Trusting --no-cache as a fix. It hides a bad layer order. Use it to diagnose a suspected stale step, not as a habit.

Interview angle

"How do you make Docker builds for a .NET app fast?" Strong answers mention, in order: copy the project files and restore before copying the source, so restore is cached; multi-stage builds so the runtime image stays small; a .dockerignore for bin, obj and .git; BuildKit cache mounts, or registry/GHA cache export in CI; and --pull on a schedule for security patches. Show that you understand why: a cache miss invalidates every later step in the stage.

A senior follow-up is "Why is my image still big after I deleted the files?" Layers are immutable diffs, and a deletion in a later layer only hides the file. Remove files in the same RUN, or copy only what you need into a fresh stage.

Practice questions

1. You change README.md and the restore step re-runs. Why, and how do you fix it?

Answer: COPY . . (or a broad copy) sits before restore and includes the README, so its checksum changed. Copy only the project files before restore, and exclude docs in .dockerignore.

2. Does RUN rm secret.txt after COPY secret.txt . remove the secret from the image?

Answer: No. The file remains in the earlier layer and anyone with the image can extract it. Use BuildKit secret mounts instead (Lesson 7).

3. Why combine apt-get update and apt-get install in one RUN?

Answer: A separate apt-get update layer is cached forever (same command text), so a later install can use stale indexes or fail. One RUN also lets you delete the lists in the same layer.

4. What decides whether a COPY step is cached?

Answer: The parent layer plus a checksum of the copied files' contents and metadata (not timestamps).

5. Two images use the same aspnet:10.0 base. How much disk do the base layers use?

Answer: Once. Layers are content-addressed and shared between images, and registries deduplicate them too.

Revision checklist

• A layer is an immutable, content-addressed diff; deleting later does not shrink the image.
• A cache miss invalidates every later step in the stage.
• Order: base -> project files -> restore -> source -> publish.
• .dockerignore: bin/, obj/, .git/, .env, secrets/.
• docker history for layer sizes; --progress=plain for cache hits.

Related on TechInterviewNotes

• Previous lesson: Images vs Containers
• Next lesson: Writing a Dockerfile for ASP.NET Core: Multi-Stage Builds

References

• Docker docs: Docker build cache
• Docker docs: Optimize cache usage in builds
• Docker docs: Build context and .dockerignore
• Docker docs: Image layers