Module 1 · Incremental source generators · Lesson 3 of 3
Testing the source a generator emits
Watch
Why an IDE check is not enough
Seeing a generated member in the IDE is a useful smoke check, but it is not a repeatable test of valid and invalid input. Test the generator independently, then keep an application build as a separate integration check.
Test the generator with a driver
The direct-driver approach starts with a small CSharpCompilation and a CSharpGeneratorDriver. It lets the test examine generated sources, generator diagnostics and the updated application compilation separately.
Create input source that uses the marker. Our generator emits the marker in post-initialization, so do not declare a duplicate in the test input. Adapt its IIncrementalGenerator with AsSourceGenerator when constructing this driver. GeneratorDriver is immutable: assign the driver returned by RunGenerators or RunGeneratorsAndUpdateCompilation before inspecting GetRunResult.
Check three separate contracts.
Contract 1: emitted source
First, the emitted source. Inspect each generator result's GeneratedSources, using HintName and SourceText instead of an IDE-dependent physical path. Compare the complete expected declaration and source count, not just a substring. Our successful fixture expects a marker file and one generated class. Also assert that the generator result's Exception is null.
Contract 2: diagnostics
Second, the diagnostics. Give the driver a non-partial target. Assert TINSG001, Error severity, and a span selecting its type name. Check that no partial declaration for that invalid target was emitted. The marker file can still be present.
Contract 3: compilation correctness
Third, compilation correctness. RunGeneratorsAndUpdateCompilation supplies an outputCompilation containing generated source; check its compiler diagnostics. The successful fixture below also requires Emit to succeed. RunGenerators alone returns a driver, not an updated compilation. Feed the original or edited user compilation into later runs, not the prior output with generated trees already attached.
Three contracts to test
- Emitted source
- Compare HintName and full SourceText, plus the source count Demo.Parcel.GeneratedName.g.cs matches the expected text
- Diagnostics
- Assert ID, severity and span for invalid input TINSG001, Error, span selects the type name
- Compilation
- Check outputCompilation diagnostics and Emit RunGeneratorsAndUpdateCompilation, then Emit succeeds
Do not use an IDE's obj directory as the unit-test oracle. Keep the driver result as the source/diagnostic contract and a real application build as a separate check.
Then test reuse and change propagation
Enable tracked steps when constructing the driver, before its first run. Give the model a tracking name. A Cached result reused prior output; Unchanged means the step ran but returned an equal value. New, Modified and Removed describe other transitions. A non-cached step alone does not establish an allocation bug: inspect the input change and comparison boundary.
The checks below reuse the returned driver for unchanged input, an unrelated added file, a target rename, and replacement of the marked input with an unmarked class. That last fixture checks removal of stale generated members; it does not delete only the attribute while leaving the old consumer call in place. A separate fresh driver checks reproducibility. None of those tests, alone, is a benchmark or proof that every internal callback was skipped.
Complete console test harness
Add a sibling Tin.Tests folder to the previous lesson's two-project layout. The test project intentionally uses an ordinary reference so its process can construct NameGenerator. Both the tests and generator pin the same Roslyn package.
Tin.Tests/Tin.Tests.csproj:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>12.0</LangVersion>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.14.0" />
<ProjectReference Include="../Tin.Generator/Tin.Generator.csproj" />
</ItemGroup>
</Project>Tin.Tests/Program.cs:
using System;
using System.IO;
using System.Linq;
using System.Text;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Tin.Generators;
var parseOptions = new CSharpParseOptions(LanguageVersion.CSharp12);
var paths = ((string?)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES") ??
throw new InvalidOperationException("Run this harness on .NET 10."))
.Split(Path.PathSeparator);
var references = paths.Select(path => MetadataReference.CreateFromFile(path)).ToArray();
CSharpCompilation Input(string text) => CSharpCompilation.Create("Fixture",
new[] { CSharpSyntaxTree.ParseText(text, parseOptions, "Input.cs", Encoding.UTF8) },
references, new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));
GeneratorDriver NewDriver() => CSharpGeneratorDriver.Create(
generators: new[] { new NameGenerator().AsSourceGenerator() },
parseOptions: parseOptions,
driverOptions: new GeneratorDriverOptions(
IncrementalGeneratorOutputKind.None, trackIncrementalGeneratorSteps: true));
static void Require(bool condition, string message)
{
if (!condition) throw new InvalidOperationException(message);
}
static string[] Sources(GeneratorDriver driver) => driver.GetRunResult().Results.Single()
.GeneratedSources.OrderBy(item => item.HintName, StringComparer.Ordinal)
.Select(item => item.HintName + "\n" + item.SourceText.ToString()).ToArray();
static void NoCrash(GeneratorDriver driver) => Require(
driver.GetRunResult().Results.Single().Exception is null, "Generator crashed");
const string valid = """
namespace Demo
{
[Tin.GenerateName]
public partial class Parcel { }
public static class Use { public static string Read() => Parcel.GeneratedName(); }
}
""";
var input = Input(valid);
GeneratorDriver driver = NewDriver();
driver = driver.RunGeneratorsAndUpdateCompilation(input, out var output, out var diagnostics);
NoCrash(driver);
Require(diagnostics.Length == 0, "Unexpected generator diagnostic");
Require(!output.GetDiagnostics().Any(item => item.Severity == DiagnosticSeverity.Error),
"Generated program does not compile");
using var emittedAssembly = new MemoryStream();
var emission = output.Emit(emittedAssembly);
Require(emission.Success, "Emission failed: " +
string.Join(Environment.NewLine, emission.Diagnostics));
var generated = driver.GetRunResult().Results.Single().GeneratedSources;
Require(generated.Length == 2, "Expected the marker attribute and one generated class");
const string expected = "namespace @Demo\n{\npartial class @Parcel\n{\n" +
" public static string GeneratedName() => \"Demo.Parcel\";\n}\n}\n";
Require(generated.Single(item => item.HintName == "Demo.Parcel.GeneratedName.g.cs")
.SourceText.ToString() == expected, "Wrong generated text");
Console.WriteLine("PASS: expected source and output compilation");
var firstSources = Sources(driver);
// Feed user input again, never the previous output containing generated trees.
driver = driver.RunGeneratorsAndUpdateCompilation(input, out _, out diagnostics);
Require(diagnostics.Length == 0 && firstSources.SequenceEqual(Sources(driver)),
"Unchanged input changed output");
var reasons = driver.GetRunResult().Results.Single().TrackedSteps["TargetModel"]
.SelectMany(step => step.Outputs).Select(item => item.Reason).ToArray();
Require(reasons.Length == 1 && reasons.All(reason => reason is
IncrementalStepRunReason.Cached or IncrementalStepRunReason.Unchanged),
"Unchanged target model was not reused");
Console.WriteLine("PASS: unchanged input reuses the target model");
var unrelated = input.AddSyntaxTrees(CSharpSyntaxTree.ParseText(
"internal class Unrelated { }", parseOptions, "Other.cs", Encoding.UTF8));
driver = driver.RunGeneratorsAndUpdateCompilation(unrelated, out output, out diagnostics);
NoCrash(driver);
Require(diagnostics.Length == 0 && firstSources.SequenceEqual(Sources(driver)),
"Unrelated input changed source");
Require(!output.GetDiagnostics().Any(item => item.Severity == DiagnosticSeverity.Error),
"Unrelated edit broke compilation");
Console.WriteLine("PASS: unrelated file leaves generated source unchanged");
var renamed = Input(valid.Replace("Parcel", "Shipment"));
driver = driver.RunGeneratorsAndUpdateCompilation(renamed, out output, out diagnostics);
Require(diagnostics.Length == 0 && Sources(driver).Any(text =>
text.StartsWith("Demo.Shipment.GeneratedName.g.cs\n", StringComparison.Ordinal)),
"Changed target was not generated");
Require(Sources(driver).All(text => !text.Contains("Parcel", StringComparison.Ordinal)),
"Stale generated target survived rename");
Require(!output.GetDiagnostics().Any(item => item.Severity == DiagnosticSeverity.Error),
"Renamed output does not compile");
Console.WriteLine("PASS: target rename replaces the generated source");
driver = driver.RunGeneratorsAndUpdateCompilation(Input("internal class Unmarked { }"),
out output, out diagnostics);
Require(diagnostics.Length == 0 && driver.GetRunResult().Results.Single()
.GeneratedSources.Single().HintName == "GenerateNameAttribute.g.cs",
"Removed target left stale source");
Console.WriteLine("PASS: removing the marker removes generated members");
var invalid = Input("[Tin.GenerateName] public class Broken { }");
var negative = NewDriver().RunGeneratorsAndUpdateCompilation(invalid, out _, out diagnostics);
NoCrash(negative);
Require(diagnostics.Length == 1 && diagnostics[0].Id == "TINSG001" &&
diagnostics[0].Severity == DiagnosticSeverity.Error, "Wrong invalid-target diagnostic");
Require(diagnostics[0].Location.SourceTree == invalid.SyntaxTrees.Single() &&
invalid.SyntaxTrees.Single().GetText().ToString(diagnostics[0].Location.SourceSpan) == "Broken",
"Diagnostic does not identify the target name");
Require(negative.GetRunResult().Results.Single().GeneratedSources.Single().HintName ==
"GenerateNameAttribute.g.cs", "Invalid target emitted a partial class");
Console.WriteLine("PASS: non-partial target reports TINSG001 at its name");
string[] unsupported =
{
"[Tin.GenerateName] public partial class GeneratedName { }",
"[Tin.GenerateName] public partial class Generic<T> { }",
"[Tin.GenerateName] partial class Mixed { } class Mixed { }",
"class Outer { [Tin.GenerateName] public partial class Inner { } }",
"[Tin.GenerateName] public static partial class StaticTarget { }",
"[Tin.GenerateName] file partial class FileTarget { }",
"[Tin.GenerateName] public partial record RecordTarget;",
"[Tin.GenerateName] public partial class Conflict { public string GeneratedName; }",
"class Base { public void GeneratedName() { } } [Tin.GenerateName] partial class Child : Base { }",
"[Tin.GenerateName, Tin.GenerateName] public partial class Repeated { }",
"[Tin.GenerateName] partial class Twice { } [Tin.GenerateName] partial class Twice { }"
};
foreach (string text in unsupported)
{
var rejected = NewDriver().RunGeneratorsAndUpdateCompilation(Input(text), out _, out diagnostics);
NoCrash(rejected);
Require(diagnostics.Any(item => item.Id == "TINSG001"), "Unsupported target not diagnosed");
Require(rejected.GetRunResult().Results.Single().GeneratedSources.Length == 1,
"Unsupported target generated members");
}
Console.WriteLine("PASS: unsupported shapes and repeated markers are rejected");
const string names = """
[ Tin.GenerateName ] partial class GlobalType { }
namespace A { [Tin.GenerateName] public partial class Same { } }
namespace B { [Tin.GenerateName] public partial class Same { } }
namespace @event { [Tin.GenerateName] public partial class @class { } }
""";
var nameDriver = NewDriver().RunGeneratorsAndUpdateCompilation(Input(names), out output, out diagnostics);
NoCrash(nameDriver);
Require(diagnostics.Length == 0 && nameDriver.GetRunResult().Results.Single()
.GeneratedSources.Length == 5, "Target names collided");
Require(!output.GetDiagnostics().Any(item => item.Severity == DiagnosticSeverity.Error),
"Escaped or global names do not compile");
Console.WriteLine("PASS: namespace, global and keyword names remain distinct");
var independent = NewDriver().RunGenerators(input);
Require(firstSources.SequenceEqual(Sources(independent)), "Fresh driver produced different source");
Console.WriteLine("PASS: fresh driver reproduces the same source");This executable fixture uses the current .NET 10 process's trusted platform assemblies as its metadata references. A production generator test suite should choose reference assemblies for every consumer target it claims to support. This harness does not establish support for every target framework, SDK, IDE or AOT configuration.
From the common parent folder:
dotnet run --project Tin.Tests/Tin.Tests.csproj
Expected program stdout after a successful build, with no failing assertion:
PASS: expected source and output compilation PASS: unchanged input reuses the target model PASS: unrelated file leaves generated source unchanged PASS: target rename replaces the generated source PASS: removing the marker removes generated members PASS: non-partial target reports TINSG001 at its name PASS: unsupported shapes and repeated markers are rejected PASS: namespace, global and keyword names remain distinct PASS: fresh driver reproduces the same source
Check your understanding
Does successful Emit execute GeneratedName? No. Here it writes the compiled assembly to a memory stream. The harness never loads that assembly or calls its method. Running Tin.App in the previous lesson is the separate behavior check that prints Demo.Parcel.
Do the naming cases compare every generated file character for character? No. They require five generated sources and no compiler errors. Only the earlier Demo.Parcel fixture compares its generated class with the complete expected text. Add per-target text assertions if that stronger contract matters.
Read failures before changing assertions
- Text matches, compiler errors remain: inspect the generated source together with the user input and reference set. A snapshot can faithfully preserve an invalid program.
- Unchanged-input reuse fails: first check that the returned driver was retained and reused, tracking was enabled before the first run, and the test supplied user trees rather than previous generated trees. Then inspect equality. Do not immediately blame allocations.
- A rename leaves old output: compare the complete set of hint names, not only the new member. Stale members are a correctness defect even when the new member exists.
- A negative case prints no success: read the exception and diagnostics. The process must finish successfully after every assertion; an earlier PASS line is not enough.
Video companion
Watch the 54:00 xUnit and Verify snapshot-testing chapter of C# Source Generators – Why and How, presented by Jim Wooley on Microsoft Visual Studio. It provides a complementary snapshot-testing walkthrough. The direct-driver tests above separately establish diagnostic, updated-compilation and change-propagation contracts; a matching snapshot alone does not establish those guarantees.
Analogy
A driver test is a print-shop proof check: compare the added page, inspect any error note, then verify that the entire assembled book passes its checks. Checking only that the clerk finished does not tell you whether the added page is correct. Repeating the same order with the same retained work record tests reuse; starting a new record tests reproducibility.
Quick reference
- Retain the returned GeneratorDriver.
- Assert GeneratedSources text, HintName, count and absence of generator exceptions.
- Assert diagnostic ID, severity and source span for invalid targets.
- Check compiler errors in outputCompilation, not only generator diagnostics.
- Enable tracking before the first run; distinguish Cached from Unchanged.
- Test unchanged, unrelated, changed and removed inputs without feeding generated output back in.