Skip to content

feat: implement Multi-Provider with parity fixes#579

Draft
jonathannorris wants to merge 16 commits intomainfrom
feat/multi-provider-parity-568
Draft

feat: implement Multi-Provider with parity fixes#579
jonathannorris wants to merge 16 commits intomainfrom
feat/multi-provider-parity-568

Conversation

@jonathannorris
Copy link
Copy Markdown
Member

@jonathannorris jonathannorris commented Mar 16, 2026

Summary

  • Implements MultiProvider with FirstMatchStrategy, FirstSuccessfulStrategy, and ComparisonStrategy
  • Adds per-provider hook isolation via InternalHookProvider protocol
  • Supports both sync (ThreadPoolExecutor) and async (asyncio.gather) parallel evaluation
  • Includes event aggregation, status tracking, and provider lifecycle management
  • Delegates track() events to all READY sub-providers, matching the js-sdk reference

Parity Fixes

Addresses gaps identified in cross-SDK comparison against the js-sdk reference (#568):

  1. ContextVar propagation (High) — Each ThreadPoolExecutor worker gets its own contextvars.copy_context() so _hook_runtime is propagated correctly on Python < 3.12
  2. Event emission during partial init (High)_refresh_aggregate_status now accepts force=True during initialize() to ensure events are emitted even when aggregate status stays NOT_READY
  3. Provider status filtering (High) — Added _should_evaluate_provider to skip NOT_READY/FATAL providers during evaluation, matching shouldEvaluateThisProvider in the JS SDK
  4. ComparisonStrategy return value (Medium) — No-mismatch path now returns the first provider's result (not fallback), matching JS SDK determineFinalResult behavior
  5. InternalHookProvider protocol (Medium) — Replaced fragile getattr/callable duck-typing with a @runtime_checkable protocol in both client.py and _registry.py
  6. Registry status override scoping (Medium)get_provider_status now checks isinstance(provider, InternalHookProvider) instead of generic getattr(provider, "get_status"), preventing accidental status override by unrelated providers
  7. Tracking support (High)MultiProvider.track() delegates to all READY sub-providers; errors from individual providers are caught and logged so they don't break application flow, matching the js-sdk reference

Related Issues

Fixes #568

@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant enhancement by adding a MultiProvider capability, enabling the OpenFeature Python SDK to manage and evaluate feature flags from multiple providers concurrently or sequentially. It also includes critical fixes to ensure the SDK's behavior aligns closely with the OpenFeature JavaScript SDK, particularly in areas of context management, event handling, and provider lifecycle. These changes improve the flexibility and consistency of the SDK, allowing for more complex and robust feature flagging architectures.

Highlights

  • Multi-Provider Implementation: Implemented MultiProvider with three distinct evaluation strategies: FirstMatchStrategy, FirstSuccessfulStrategy, and ComparisonStrategy.
  • Per-Provider Hook Isolation: Introduced the InternalHookProvider protocol to enable per-provider hook isolation, allowing providers like MultiProvider to manage their own hook execution lifecycle.
  • Parallel Evaluation Support: Added support for both synchronous (ThreadPoolExecutor) and asynchronous (asyncio.gather) parallel evaluation of flags across multiple providers.
  • Parity Fix: ContextVar Propagation: Ensured correct ContextVar propagation for ThreadPoolExecutor workers on Python versions older than 3.12 by copying the context.
  • Parity Fix: Event Emission during Partial Init: Modified _refresh_aggregate_status to accept force=True during initialization, guaranteeing event emission even when the aggregate status remains NOT_READY.
  • Parity Fix: Provider Status Filtering: Implemented _should_evaluate_provider to skip NOT_READY or FATAL providers during flag evaluation, aligning with the JS SDK's behavior.
  • Parity Fix: ComparisonStrategy Return Value: Adjusted ComparisonStrategy to return the first provider's result (not fallback) in a no-mismatch scenario, matching the JS SDK's determineFinalResult.
  • Parity Fix: InternalHookProvider Protocol Usage: Replaced fragile getattr/callable duck-typing with a robust @runtime_checkable protocol for InternalHookProvider in client.py and _registry.py.
  • Parity Fix: Registry Status Override Scoping: Updated get_provider_status to specifically check for InternalHookProvider instances, preventing unintended status overrides by unrelated providers.
Changelog
  • openfeature/client.py
    • Imported InternalHookProvider.
    • Modified _establish_hooks_and_provider to conditionally use provider hooks based on InternalHookProvider status.
    • Added _provider_uses_internal_hooks method to check if a provider manages its own hooks.
    • Added _set_internal_provider_hook_runtime and _reset_internal_provider_hook_runtime methods to manage internal hook runtime context.
    • Integrated internal hook runtime management into evaluate_flag_details_async and evaluate_flag_details methods.
  • openfeature/provider/init.py
    • Updated __all__ to export new multi-provider related classes and protocols.
    • Added InternalHookProvider protocol definition for providers managing their own hooks.
    • Imported MultiProvider and related strategy classes from multi_provider module.
  • openfeature/provider/_registry.py
    • Imported InternalHookProvider.
    • Modified _initialize_provider to conditionally dispatch PROVIDER_READY and PROVIDER_ERROR events based on the provider's current status, preventing redundant events from InternalHookProvider.
    • Updated get_provider_status to delegate status retrieval to InternalHookProvider instances.
  • openfeature/provider/multi_provider.py
    • Added new file implementing the MultiProvider class.
    • Defined ProviderEntry dataclass for managing individual providers.
    • Defined _ProviderEvaluation and _ProviderHookRuntime dataclasses for internal use.
    • Implemented EvaluationStrategy protocol and concrete strategies: FirstMatchStrategy, FirstSuccessfulStrategy, and ComparisonStrategy.
    • Implemented MultiProvider as an AbstractProvider, handling provider registration, status aggregation, event handling, and flag resolution across multiple child providers.
    • Included logic for both synchronous and asynchronous parallel evaluation using ThreadPoolExecutor and asyncio.gather.
    • Provided detailed implementations for resolve_boolean_details, resolve_string_details, resolve_integer_details, resolve_float_details, and resolve_object_details (and their async counterparts) to delegate to child providers.
  • tests/test_multi_provider.py
    • Added new file containing unit tests for MultiProvider functionality.
    • Included tests for MultiProvider initialization, duplicate name handling, and ComparisonStrategy validation.
    • Tested FirstMatchStrategy behavior, including handling FLAG_NOT_FOUND and other errors.
    • Tested FirstSuccessfulStrategy behavior, including skipping errors and aggregating failures.
    • Verified ComparisonStrategy behavior, including fallback logic and on_mismatch callback.
    • Confirmed synchronous and asynchronous parallel evaluation using SyncBlocker and AsyncBlocker.
    • Validated provider hook isolation and lifecycle management within MultiProvider.
    • Ensured correct event aggregation and deduplication by MultiProvider.
    • Tested forwarding of PROVIDER_CONFIGURATION_CHANGED events.
    • Verified MultiProvider status after shutdown.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 16, 2026

Codecov Report

❌ Patch coverage is 87.18905% with 103 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.33%. Comparing base (5ff1cf0) to head (cefaaee).

Files with missing lines Patch % Lines
openfeature/provider/multi_provider/_provider.py 79.63% 67 Missing ⚠️
openfeature/provider/multi_provider/_strategies.py 81.73% 21 Missing ⚠️
tests/test_multi_provider.py 95.20% 15 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #579      +/-   ##
==========================================
- Coverage   98.35%   95.33%   -3.02%     
==========================================
  Files          45       50       +5     
  Lines        2183     2980     +797     
==========================================
+ Hits         2147     2841     +694     
- Misses         36      139     +103     
Flag Coverage Δ
unittests 95.33% <87.18%> (-3.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a MultiProvider feature, allowing OpenFeature clients to manage and evaluate multiple feature flag providers using various strategies (FirstMatch, FirstSuccessful, Comparison) with both sequential and parallel execution modes. It includes a new InternalHookProvider protocol to enable providers to manage their own hook execution and status internally, which the MultiProvider implements. The client-side flag evaluation logic has been updated to integrate this new internal hook management. Review comments suggest improving the readability of provider_hooks assignment, adding a comment to clarify ContextVars propagation in ThreadPoolExecutor for older Python versions, and enhancing the docstring for ComparisonStrategy.determine_final_result to better explain its behavior.

Comment thread openfeature/client.py
Comment thread openfeature/provider/multi_provider/_provider.py
Comment thread openfeature/provider/multi_provider.py Outdated
@jonathannorris jonathannorris force-pushed the feat/multi-provider-parity-568 branch from e1145f6 to d6fda15 Compare March 16, 2026 19:06
vikasrao23 and others added 9 commits May 8, 2026 14:12
Implements the Multi-Provider as specified in OpenFeature Appendix A.

The Multi-Provider wraps multiple underlying providers in a unified interface,
allowing a single client to interact with multiple flag sources simultaneously.

Key features implemented:
- MultiProvider class extending AbstractProvider
- FirstMatchStrategy (sequential evaluation, stops at first success)
- EvaluationStrategy protocol for custom strategies
- Provider name uniqueness (explicit, metadata-based, or auto-indexed)
- Parallel initialization of all providers with error aggregation
- Support for all flag types (boolean, string, integer, float, object)
- Hook aggregation from all providers

Use cases:
- Migration: Run old and new providers in parallel
- Multiple data sources: Combine env vars, files, and SaaS providers
- Fallback: Primary provider with backup sources

Example usage:
    provider_a = SomeProvider()
    provider_b = AnotherProvider()

    multi = MultiProvider([
        ProviderEntry(provider_a, name="primary"),
        ProviderEntry(provider_b, name="fallback")
    ])

    api.set_provider(multi)

Closes #511

Signed-off-by: vikasrao23 <vikasrao23@users.noreply.github.com>
Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
…hancements

Address Gemini code review feedback:
- Update initialize() docstring to reflect sequential (not parallel) initialization
- Add documentation notes to all async methods explaining they currently delegate to sync
- Clarify that parallel evaluation mode is planned but not yet implemented
- Update EvaluationStrategy protocol docs to set correct expectations

This brings documentation in line with actual implementation. True async and parallel
execution will be added in follow-up PRs.

Refs: #511
Signed-off-by: vikasrao23 <vikasrao23@users.noreply.github.com>
Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
CRITICAL FIXES:
- Fix FlagResolutionDetails initialization - remove invalid flag_key parameter
- Add error_code (ErrorCode.GENERAL) to all error results per spec

HIGH PRIORITY:
- Implement true async evaluation using _evaluate_with_providers_async
- All async methods now properly await provider async methods (no blocking)
- Implement parallel provider initialization using ThreadPoolExecutor

IMPROVEMENTS:
- Remove unused imports (asyncio, ProviderEvent, ProviderEventDetails, ProviderStatus)
- Add ErrorCode import for proper error handling
- Cache provider hooks to avoid re-aggregating on every evaluation
- Update docstrings to clarify current implementation status

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
HIGH PRIORITY FIXES:
- Fix name resolution logic to prevent collisions between explicit and auto-generated names
  - Check used_names set for metadata names before using them
  - Use while loop to find next available indexed name if collision detected
- Implement event propagation (spec requirement)
  - Override attach() and detach() methods to forward events to all providers
  - Import ProviderEvent and ProviderEventDetails
  - Enables cache invalidation and other event-driven features

MEDIUM PRIORITY IMPROVEMENTS:
- Parallel shutdown with proper error logging
  - Use ThreadPoolExecutor for concurrent shutdown
  - Add logging for shutdown failures
- Optimize ThreadPoolExecutor max_workers
  - Set to len(providers) for both initialize() and shutdown()
  - Ensures all providers can start immediately
- Improve type hints for better type safety
  - Add generic type parameters to FlagResolutionDetails in resolve_fn signatures
  - Specify Awaitable return type for async resolve_fn
  - Add generic types to results list declarations

All critical and high-priority feedback addressed. Ready for re-review.

Refs: #511
Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
This is more consistent with the other type imports in the file.

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
Co-authored-by: jonathan <jonathan@taplytics.com>
Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
- Fix ContextVar propagation to ThreadPoolExecutor workers (Python <3.12)
- Fix _refresh_aggregate_status dropping events during partial init failure
- Add shouldEvaluateThisProvider check to skip NOT_READY/FATAL providers
- Fix ComparisonStrategy to return first provider result on no-mismatch
- Add InternalHookProvider protocol replacing fragile duck-typing
- Scope get_status override in registry to InternalHookProvider only
- Rename camelCase instance variables to snake_case

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
- Add _is_internal_hook_provider class marker to avoid Mock false positives
  with runtime_checkable Protocol isinstance checks
- Fix mypy no-redef errors by hoisting evaluations declaration before branch
- Fix mypy no-any-return by assigning to typed local before returning
- Fix mypy attr-defined by using _as_internal_hook_provider narrowing helper
- Apply ruff formatting fixes

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
Delegates track events to all READY sub-providers, matching the
JS SDK reference implementation. Errors from individual providers
are caught and logged to avoid breaking application flow.

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
@jonathannorris jonathannorris force-pushed the feat/multi-provider-parity-568 branch from d6fda15 to 4d2cf01 Compare May 8, 2026 18:20
@jonathannorris jonathannorris marked this pull request as ready for review May 8, 2026 18:22
@jonathannorris jonathannorris requested review from a team as code owners May 8, 2026 18:22
Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
@jonathannorris jonathannorris marked this pull request as draft May 8, 2026 18:28
@jonathannorris jonathannorris requested a review from Copilot May 8, 2026 18:32
@jonathannorris
Copy link
Copy Markdown
Member Author

/gemini review

…ck false positives on Python <3.12

On Python <3.12, isinstance() against @runtime_checkable Protocols matches
Mock objects (structural subtyping checks were tightened in 3.12). The
existing getattr guard used a truthiness check, but Mock auto-creates
attributes as truthy MagicMock objects, so the guard was ineffective.
Switching to 'is True' identity check ensures only objects that explicitly
set _is_internal_hook_provider = True (a literal bool) pass the guard.

Also flattens nested conditional in ComparisonStrategy.determine_final_result.

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the MultiProvider implementation, which allows for the combination of multiple feature providers using strategies like FirstMatch, FirstSuccessful, and Comparison. It also adds the InternalHookProvider protocol, enabling providers to manage their own hook execution and status. The client and registry have been updated to support these new provider types. Feedback suggests a minor optimization in the MultiProvider evaluation logic to prevent unnecessary thread pool initialization when no providers are eligible.

Comment thread openfeature/provider/multi_provider/_provider.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements a new MultiProvider implementation for the Python SDK with multiple evaluation strategies (first-match, first-successful, comparison), per-provider hook isolation support via an internal hook protocol, aggregated provider status/event handling, and sync/async parallel evaluation to align behavior with the JS SDK reference.

Changes:

  • Added MultiProvider plus FirstMatchStrategy, FirstSuccessfulStrategy, and ComparisonStrategy, including sync/async parallel evaluation paths.
  • Introduced InternalHookProvider (@runtime_checkable Protocol + marker) and updated client/registry to support providers that manage hooks/status internally.
  • Added a comprehensive test suite covering strategies, hook lifecycle isolation, parallel evaluation, event/status aggregation, and track() delegation.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
uv.lock Updates locked version for the editable openfeature-sdk package.
tests/test_multi_provider.py Adds tests validating MultiProvider strategy semantics, hook isolation, parallelism, event aggregation, and tracking delegation.
openfeature/provider/multi_provider.py New MultiProvider implementation with strategies, hook isolation runtime, parallel evaluation, and aggregate status/event management.
openfeature/provider/_registry.py Uses InternalHookProvider to avoid unintended status overrides and avoids double-emitting init events when providers self-emit.
openfeature/provider/__init__.py Exposes MultiProvider and strategies from the provider package and defines the InternalHookProvider protocol.
openfeature/client.py Skips client-side provider hooks for internal-hook providers and sets/resets internal hook runtime around provider evaluation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread openfeature/provider/multi_provider.py Outdated
Comment thread openfeature/provider/multi_provider/_provider.py
Comment thread openfeature/provider/multi_provider/_provider.py
Comment thread openfeature/provider/multi_provider/_provider.py
Comment thread openfeature/provider/multi_provider/_provider.py
Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
…mine_final_result

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
… _provider modules

Signed-off-by: Jonathan Norris <jonathan.norris@dynatrace.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Multi-provider] Gaps identified relative to js-sdk reference implementation

5 participants