[poc][wip]Allow references in configuration#127269
Draft
rosebyte wants to merge 16 commits intodotnet:mainfrom
Draft
[poc][wip]Allow references in configuration#127269rosebyte wants to merge 16 commits intodotnet:mainfrom
rosebyte wants to merge 16 commits intodotnet:mainfrom
Conversation
rosebyte
changed the title
Contributor
|
Tagging subscribers to this area: @dotnet/area-extensions-configuration |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a reference-resolution feature for Microsoft.Extensions.Configuration, enabling ${...} token interpolation and section aliasing across configuration providers, controlled via per-source ReferenceMode settings.
Changes:
- Add a
ReferenceResolutionEngine+ReferenceParserto resolve${...}references (including optional chains, strict aliases, and relative refs). - Add public configuration APIs (
ReferenceMode,SetReferenceMode(...)) and wire the engine intoConfigurationRoot/ConfigurationManagerreads andGetChildren. - Add/extend tests (plus a compatibility shim) to validate real-world resolution, provider ordering, alias behavior, and ConfigurationManager behavior.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/libraries/Microsoft.Extensions.Configuration/src/ReferenceResolutionEngine.cs | Implements the core resolution engine (caching, aliasing, token resolution, reload invalidation). |
| src/libraries/Microsoft.Extensions.Configuration/src/ReferenceParser.cs | Parses ${...} expressions into tokens and reference chains (supports optional/strict markers and relative refs). |
| src/libraries/Microsoft.Extensions.Configuration/src/ReferenceMode.cs | Introduces ReferenceMode to control source participation (Ignore/Read/Scan). |
| src/libraries/Microsoft.Extensions.Configuration/src/ReferenceResolutionConfigurationBuilderExtensions.cs | Adds public SetReferenceMode APIs and internal plumbing to map source modes to providers. |
| src/libraries/Microsoft.Extensions.Configuration/src/ConfigurationBuilder.cs | Attaches the engine at Build() time when at least one source is in Scan mode. |
| src/libraries/Microsoft.Extensions.Configuration/src/ConfigurationRoot.cs | Routes reads through the engine when enabled; invalidates/disposes engine on reload/dispose. |
| src/libraries/Microsoft.Extensions.Configuration/src/ConfigurationManager.cs | Maintains and swaps an engine snapshot as sources/properties change without forcing provider reloads. |
| src/libraries/Microsoft.Extensions.Configuration/src/InternalConfigurationRootExtensions.cs | Integrates engine into TryGetConfiguration and GetChildren key computation. |
| src/libraries/Microsoft.Extensions.Configuration/src/ReferenceCountedProvidersManager.cs | Adds provider snapshot API used when rebuilding the engine in ConfigurationManager. |
| src/libraries/Microsoft.Extensions.Configuration/src/Resources/Strings.resx | Adds localized strings for reference-resolution errors. |
| src/libraries/Microsoft.Extensions.Configuration/ref/Microsoft.Extensions.Configuration.cs | Updates reference assembly surface for new public API (ReferenceMode, SetReferenceMode). |
| src/libraries/Microsoft.Extensions.Configuration/tests/ReferenceResolutionTestShims.cs | Adds test-only shim mapping legacy test helper names to SetReferenceMode. |
| src/libraries/Microsoft.Extensions.Configuration/tests/ConfigurationTest.cs | Adds extensive unit tests for token resolution, aliasing, relative refs, and per-source modes. |
| src/libraries/Microsoft.Extensions.Configuration/tests/ConfigurationManagerTest.cs | Adds tests for ConfigurationManager engine behavior (no reloads, observing later sources, etc.). |
| src/libraries/Microsoft.Extensions.Configuration/tests/FunctionalTests/ConfigurationTests.cs | Adds functional “real world” scenario coverage across JSON/INI + overrides. |
| src/libraries/Microsoft.Extensions.Configuration/tests/FunctionalTests/Microsoft.Extensions.Configuration.Functional.Tests.csproj | Links in the test shim so functional tests can compile. |
Comment on lines
+39
to
+43
| if (builder.Sources.Count == 0) | ||
| { | ||
| throw new InvalidOperationException("No sources."); | ||
| } | ||
| return builder.SetReferenceMode(builder.Sources[builder.Sources.Count - 1], mode); |
There was a problem hiding this comment.
This throws InvalidOperationException("No sources."), which is inconsistent with the production API (SetReferenceMode) that throws with SR.ReferenceResolution_NoSourceToConfigure. Aligning the message (or reusing the resource string if accessible) will make failures easier to understand when tests fail.
Suggested change
| if (builder.Sources.Count == 0) | |
| { | |
| throw new InvalidOperationException("No sources."); | |
| } | |
| return builder.SetReferenceMode(builder.Sources[builder.Sources.Count - 1], mode); | |
| return builder.SetReferenceMode(mode); |
Comment on lines
+147
to
+149
| <data name="ReferenceResolution_KeyNotFound" xml:space="preserve"> | ||
| <value>The reference key '{0}' referenced by '{1}' was not found. Add a fallback reference or mark the chain optional with a trailing '?' (for example '${{Key?}}').</value> | ||
| </data> |
There was a problem hiding this comment.
ReferenceResolution_KeyNotFound is added here but is not referenced anywhere in the Microsoft.Extensions.Configuration source (the current engine soft-fails missing required references). Either remove this resource or wire it up in the resolution path so it can actually surface to callers.
Suggested change
| <data name="ReferenceResolution_KeyNotFound" xml:space="preserve"> | |
| <value>The reference key '{0}' referenced by '{1}' was not found. Add a fallback reference or mark the chain optional with a trailing '?' (for example '${{Key?}}').</value> | |
| </data> |
Comment on lines
+153
to
+155
| <data name="ReferenceResolution_SectionReferenceMustBeSingleToken" xml:space="preserve"> | ||
| <value>Section reference at '{0}' must be a single reference expression in the form '${{Path}}'.</value> | ||
| </data> |
There was a problem hiding this comment.
ReferenceResolution_SectionReferenceMustBeSingleToken is added here but is not referenced anywhere in the Microsoft.Extensions.Configuration source. If invalid section-alias forms are meant to be rejected, please add the corresponding validation/throw site; otherwise remove the unused resource.
Suggested change
| <data name="ReferenceResolution_SectionReferenceMustBeSingleToken" xml:space="preserve"> | |
| <value>Section reference at '{0}' must be a single reference expression in the form '${{Path}}'.</value> | |
| </data> |
|
|
||
| public Path(string value) | ||
| { | ||
| Value = value.Trim().Replace('.', ConfigurationPath.KeyDelimiter[0]); |
There was a problem hiding this comment.
Path normalizes keys by replacing '.' with ConfigurationPath.KeyDelimiter. This breaks legitimate configuration keys that contain dots in segment names (e.g. keys like "Json.Key2:JsonKey3" used elsewhere in this repo) and will cause lookups/aliasing to fail. Prefer preserving the key verbatim (no '.' replacement) and, if dot-notation support is desired, handle it explicitly in the reference-expression parser rather than rewriting all keys.
Suggested change
| Value = value.Trim().Replace('.', ConfigurationPath.KeyDelimiter[0]); | |
| Value = value.Trim(); |
| // builders whose Properties dictionary reacts to writes (e.g., ConfigurationManager) | ||
| // trigger a rebuild of the engine with the new mode map. | ||
| Dictionary<IConfigurationSource, ReferenceMode> map = TryGetSourceModes(configurationBuilder.Properties) | ||
| ?? new Dictionary<IConfigurationSource, ReferenceMode>(); |
There was a problem hiding this comment.
This comment says the SourceModes property value is a dictionary with reference-equality keys, but new Dictionary<IConfigurationSource, ReferenceMode>() uses EqualityComparer<IConfigurationSource>.Default. If an IConfigurationSource implementation overrides Equals/GetHashCode, mode entries could collide or be un-findable. Either construct the dictionary with a reference-equality comparer or update the comment to match the actual behavior.
Suggested change
| ?? new Dictionary<IConfigurationSource, ReferenceMode>(); | |
| ?? new Dictionary<IConfigurationSource, ReferenceMode>(ReferenceEqualityComparer.Instance); |
Comment on lines
+40
to
+44
| // Non-null when the builder opted into reference resolution via EnableReferenceResolution. | ||
| // Rebuilt on every source mutation (AddSource/ReloadSources) so it always reflects the | ||
| // current provider set. Reads are unsynchronized; in-flight reads that observe a stale | ||
| // engine still see a consistent (old) provider snapshot held by that engine. | ||
| private ReferenceResolutionEngine? _engine; |
There was a problem hiding this comment.
The comment references opting in via EnableReferenceResolution, but the public API introduced in this PR is SetReferenceMode(..., ReferenceMode.Scan) (and tests use a shim). Update the comment to avoid implying a non-existent public entry point.
| } | ||
|
|
||
| // Returns an immutable snapshot of the currently-built providers. Used by the builder's | ||
| // source.Build(...) loop so a ReferenceResolutionConfigurationSource can observe providers |
There was a problem hiding this comment.
The comment mentions ReferenceResolutionConfigurationSource, but there is no such type in this project (searching the repo only finds this comment). Please update/remove the reference so the comment matches the actual implementation using ReferenceResolutionEngine on the root.
Suggested change
| // source.Build(...) loop so a ReferenceResolutionConfigurationSource can observe providers | |
| // source.Build(...) loop so the root's ReferenceResolutionEngine can observe providers |
This was referenced Apr 22, 2026
Open
| if (alias.Strict) | ||
| { | ||
| value = null; | ||
| cache.TryAdd(path.Value, value); |
|
|
||
| public static Path FromReference(string value) | ||
| { | ||
| return new Path(value.Trim().Replace('.', ConfigurationPath.KeyDelimiter[0])); |
Comment on lines
+369
to
+387
| private static int FindMatchingCloseInRange(string content, int openParenIndex, int end, int tokenStart) | ||
| { | ||
| int depth = 1; | ||
| int i = openParenIndex + 1; | ||
| while (i < end) | ||
| { | ||
| char c = content[i]; | ||
|
|
||
| if (c == '\\') | ||
| { | ||
| i += (i + 1 < end) ? 2 : 1; | ||
| continue; | ||
| } | ||
|
|
||
| if (c == '\'' || c == '"') | ||
| { | ||
| i = SkipQuoted(content, i, tokenStart + (i - openParenIndex)); | ||
| continue; | ||
| } |
Comment on lines
+7
to
+37
| /// Controls how an <see cref="IConfigurationSource"/> participates in <c>{{…}}</c> | ||
| /// reference resolution performed by the <see cref="IConfigurationRoot"/> built from the | ||
| /// containing <see cref="IConfigurationBuilder"/>. | ||
| /// </summary> | ||
| /// <remarks> | ||
| /// Values are totally ordered by participation: <see cref="Ignore"/> < <see cref="Read"/> | ||
| /// < <see cref="Scan"/>. Sources default to <see cref="Scan"/> unless configured via | ||
| /// <see cref="ReferenceResolutionConfigurationBuilderExtensions.SetReferenceMode(IConfigurationBuilder, ReferenceMode)"/> | ||
| /// or one of its overloads. The reference-resolution engine is attached to the built root | ||
| /// unless every source has been explicitly set to a non-<see cref="Scan"/> mode. | ||
| /// </remarks> | ||
| public enum ReferenceMode | ||
| { | ||
| /// <summary> | ||
| /// The source is invisible to the reference-resolution engine: no <c>{{…}}</c> | ||
| /// reference or section alias in another source can reach its values. Direct reads | ||
| /// via the normal <see cref="IConfiguration"/> API still return its values. | ||
| /// </summary> | ||
| Ignore = 0, | ||
|
|
||
| /// <summary> | ||
| /// The source is a valid substitution target for references in other | ||
| /// <see cref="Scan"/> sources. The source's own values are returned verbatim — | ||
| /// <c>{{…}}</c> sequences are not interpreted. This is the default mode for | ||
| /// sources that have not been configured explicitly. | ||
| /// </summary> | ||
| Read = 1, | ||
|
|
||
| /// <summary> | ||
| /// The source's values are scanned for <c>{{…}}</c> reference tokens and section | ||
| /// aliases, and the source is exposed as a substitution target for other |
Comment on lines
+30
to
+31
| /// <c>{{…}}</c> sequences are not interpreted. This is the default mode for | ||
| /// sources that have not been configured explicitly. |
| } | ||
|
|
||
| public static IConfigurationBuilder ConfigureReferenceResolution(this IConfigurationBuilder builder, IConfigurationSource source, ReferenceMode mode) | ||
| => builder.SetReferenceMode(source, mode); |
Comment on lines
+9
to
+20
| /// <summary> | ||
| /// Provides extension methods for configuring how individual <see cref="IConfigurationSource"/> | ||
| /// instances participate in <c>{{…}}</c> reference resolution performed by the | ||
| /// <see cref="IConfigurationRoot"/> built from the containing | ||
| /// <see cref="IConfigurationBuilder"/>. | ||
| /// </summary> | ||
| /// <remarks> | ||
| /// Sources default to <see cref="ReferenceMode.Scan"/>. The reference-resolution engine is | ||
| /// attached to the built root unless every source has been explicitly set to a | ||
| /// non-<see cref="ReferenceMode.Scan"/> mode, in which case the built root behaves as a plain | ||
| /// <see cref="IConfigurationRoot"/> with no reference interpretation. | ||
| /// </remarks> |
Comment on lines
+635
to
+640
| if (allowSection) | ||
| { | ||
| return TryResolveValue(tokenPath, raw.AsString!, resolutionStack, depth, raw.ProviderIndex, out value); | ||
| } | ||
|
|
||
| return TryResolveValueAsString(tokenPath, raw.AsString!, resolutionStack, depth, raw.ProviderIndex, out value); |
3 tasks
rosebyte
force-pushed
the
experiment/configuration-reference-root
branch
from
f6e7f99 to
84b7026
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (1)
src/libraries/Microsoft.Extensions.Configuration/src/InternalConfigurationRootExtensions.cs:63
- TryGetConfiguration uses the ReferenceResolutionEngine directly without taking a ReferenceCountedProviders lease when the root is a ConfigurationManager. If the engine is built over a provider snapshot that can be disposed during ReloadSources, this path can race with source mutation and access disposed providers. Consider mirroring GetChildrenImplementation’s GetProvidersReference usage (or otherwise ensure the engine keeps its providers alive).
}
value = null;
return false;
}
// common cases Providers is IList<IConfigurationProvider> in ConfigurationRoot
IList<IConfigurationProvider> providers = root.Providers is IList<IConfigurationProvider> list
? list
: root.Providers.ToList();
// ensure looping in the reverse order
for (int i = providers.Count - 1; i >= 0; i--)
Comment on lines
+78
to
+91
| // Returns an immutable snapshot of the currently-built providers. Used by the builder's | ||
| // source.Build(...) loop so a ReferenceResolutionConfigurationSource can observe providers | ||
| // registered before it. | ||
| public IReadOnlyList<IConfigurationProvider> GetProvidersSnapshot() | ||
| { | ||
| lock (_replaceProvidersLock) | ||
| { | ||
| if (_disposed) | ||
| { | ||
| throw new ObjectDisposedException(nameof(ConfigurationManager)); | ||
| } | ||
|
|
||
| return _refCountedProviders.Providers.ToArray(); | ||
| } |
Comment on lines
+177
to
195
| // Rebuild the engine against the current provider set when resolution is enabled. | ||
| // The old engine's providers are the previous snapshot, and any in-flight read that | ||
| // already captured the old engine will complete against that snapshot; a subsequent | ||
| // read will pick up the new engine. The old engine is disposed to drop its reload- | ||
| // token subscription against the old providers. | ||
| private void SwapEngine() | ||
| { | ||
| ReferenceResolutionEngine? newEngine = null; | ||
| if (ReferenceResolutionConfigurationBuilderExtensions.IsEnabled(_properties)) | ||
| { | ||
| IReadOnlyList<IConfigurationProvider> providers = _providerManager.GetProvidersSnapshot(); | ||
| newEngine = new ReferenceResolutionEngine(providers); | ||
| } | ||
|
|
||
| ReferenceResolutionEngine? previous = Interlocked.Exchange(ref _engine, newEngine); | ||
| previous?.Dispose(); | ||
| } | ||
|
|
||
| private void DisposeRegistrations() |
|
|
||
| return GetConfiguration(_providers, key); | ||
| } | ||
| set => SetConfiguration(_providers, key, value); |
Comment on lines
61
to
71
| get | ||
| { | ||
| using ReferenceCountedProviders reference = _providerManager.GetReference(); | ||
| ReferenceResolutionEngine? engine = _engine; | ||
| if (engine is not null) | ||
| { | ||
| return engine.TryGet(key, out string? resolved) ? resolved : null; | ||
| } | ||
|
|
||
| return ConfigurationRoot.GetConfiguration(reference.Providers, key); | ||
| } |
added 3 commits
May 5, 2026 15:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a reference resolution engine and related functionality to enhance configuration management. This includes new modes for configuration sources and test shims for compatibility. Additional error messages for reference resolution issues are also added.