AI Features Without Vendor Lock-In: The Multi-Provider Architecture for SaaS

Most teams add AI features by calling the OpenAI SDK directly from a controller or service class. It works, and it ships fast. It also means every part of your application that touches an LLM now depends on one company's uptime, one pricing page, and one set of API decisions you don't control. When that vendor raises prices, deprecates a model, or has a bad day, your application inherits the problem.

A multi-provider LLM architecture avoids that by putting an abstraction layer between your application code and any single AI vendor. Instead of calling OpenAI's client directly, your code calls an interface. Underneath that interface sits OpenAI, Anthropic, Mistral, or a self-hosted Ollama endpoint, and swapping between them becomes a configuration change rather than a rewrite. This isn't a theoretical concern. Teams that skipped this step have had to rewrite entire AI features on short notice because a provider changed its API or its pricing model overnight.

Why single-vendor integration becomes a liability

The failure modes are predictable once you've watched a few of them play out. A provider has an outage during your peak traffic window and every AI-dependent feature in your product goes dark at once. A provider raises per-token pricing and your margins on AI features shrink without warning. A model gets deprecated with a few months' notice and the prompts you tuned against its quirks stop producing the same output. None of these are hypothetical: OpenAI, Anthropic, and Google have all deprecated models, changed rate limits, and adjusted pricing multiple times over the past two years.

The deeper problem is architectural. If OpenAI\Client is instantiated inside your controllers and services, every one of those call sites needs to change when you want to add a second provider, test a cheaper model for a specific task, or route sensitive data to a self-hosted model for compliance reasons. That refactor gets harder the longer you wait, because the integration spreads through the codebase one feature at a time.

The abstraction layer

The fix is a provider interface that every LLM call goes through, regardless of which vendor eventually handles the request. In a Symfony application, that looks like a small set of interfaces and a service per provider.

interface LlmProviderInterface
{
    public function complete(LlmRequest $request): LlmResponse;
    public function stream(LlmRequest $request): \Generator;
    public function supports(LlmCapability $capability): bool;
}

Each provider gets its own implementation: OpenAiProvider, AnthropicProvider, MistralProvider, OllamaProvider. Application code never references these classes directly. It asks a router for a provider that can handle the current request, and the router decides which implementation to hand back.

final class OpenAiProvider implements LlmProviderInterface
{
    public function __construct(
        private readonly OpenAiClient $client,
        private readonly PromptNormalizer $normalizer,
    ) {}

    public function complete(LlmRequest $request): LlmResponse
    {
        $payload = $this->normalizer->toOpenAiFormat($request);
        $response = $this->client->chat()->create($payload);

        return LlmResponse::fromOpenAi($response);
    }
}

The LlmRequest and LlmResponse objects are provider-agnostic. They carry the messages, the requested capability (classification, generation, embedding), and any constraints (max tokens, temperature, required context length). Each provider's implementation is responsible for translating that generic request into whatever shape its API expects, and translating the response back.

Routing by task, not by habit

Once the interface exists, the routing strategy stops being an afterthought and becomes a real decision. Different tasks have different requirements, and a single "best" model rarely fits all of them.

A support ticket classifier doesn't need your most capable model. A cheap, fast model handles that well and keeps the per-request cost low. Long-form generation, where quality differences are actually visible to users, is where the more expensive model earns its cost. Anything touching regulated or sensitive customer data might need to stay on a self-hosted Ollama instance rather than leaving your infrastructure at all, which matters for teams operating under GDPR or sector-specific compliance requirements.

final class TaskBasedRouter
{
    public function __construct(
        private readonly ProviderRegistry $registry,
    ) {}

    public function route(LlmRequest $request): LlmProviderInterface
    {
        return match ($request->getTaskType()) {
            TaskType::Classification => $this->registry->get('mistral-small'),
            TaskType::Generation => $this->registry->get('claude-sonnet'),
            TaskType::SensitiveData => $this->registry->get('ollama-local'),
            default => $this->registry->getDefault(),
        };
    }
}

This is also where capability detection earns its place. Not every provider supports every feature, and a request that needs a 200,000-token context window or native tool calling should never land on a provider that can't handle it.

final class CapabilityAwareRouter
{
    public function route(LlmRequest $request): LlmProviderInterface
    {
        $candidates = $this->registry->getForTaskType($request->getTaskType());

        foreach ($candidates as $provider) {
            if ($provider->supports($request->getRequiredCapability())) {
                return $provider;
            }
        }

        throw new NoCapableProviderException($request->getRequiredCapability());
    }
}

Failover: the part that actually justifies the effort

Routing by task is a cost and quality optimization. Failover is what protects you when a provider goes down. Without it, a 429 rate-limit response or a 500 from your one provider becomes a 500 in your own application, visible to your users at the worst possible time.

The pattern is a failover chain: try the primary provider, and on a retryable error, fall through to the next provider in the chain rather than surfacing the failure immediately.

final class FailoverProvider implements LlmProviderInterface
{
    /** @param LlmProviderInterface[] $providers */
    public function __construct(
        private readonly array $providers,
        private readonly LoggerInterface $logger,
    ) {}

    public function complete(LlmRequest $request): LlmResponse
    {
        $lastException = null;

        foreach ($this->providers as $provider) {
            try {
                return $provider->complete($request);
            } catch (RateLimitException|ServerErrorException $e) {
                $this->logger->warning('Provider failed, trying next', [
                    'provider' => $provider::class,
                    'error' => $e->getMessage(),
                ]);
                $lastException = $e;
                continue;
            }
        }

        throw new AllProvidersFailedException(previous: $lastException);
    }
}

A chain configured as OpenAI, then Anthropic, then a self-hosted Ollama fallback means a single vendor's outage degrades your service rather than stopping it. The fallback tier doesn't need to match the primary's quality exactly. Users tolerate a slightly different answer far better than they tolerate a broken feature.

The prompt normalization layer

This is the part that gets skipped and then causes the most debugging pain later. Providers don't agree on how system messages work, how tool calls are formatted, or how streaming responses are structured. OpenAI, Anthropic, and Mistral each expect subtly different request shapes, and a request built for one will either error or silently behave differently on another.

The normalization layer is what makes the rest of the architecture honest. It's a translation step, not a wrapper that just forwards data:

final class PromptNormalizer
{
    public function toOpenAiFormat(LlmRequest $request): array
    {
        return [
            'model' => $request->getModel(),
            'messages' => $this->buildOpenAiMessages($request),
            'max_tokens' => $request->getMaxTokens(),
        ];
    }

    public function toAnthropicFormat(LlmRequest $request): array
    {
        return [
            'model' => $request->getModel(),
            'system' => $request->getSystemPrompt(),
            'messages' => $this->buildAnthropicMessages($request),
            'max_tokens' => $request->getMaxTokens(),
        ];
    }
}

Anthropic takes the system prompt as a separate top-level field rather than a message in the array. Tool call formats differ enough between providers that a naive pass-through will break the moment you route the same request to a second vendor. Streaming protocols differ too: OpenAI and Anthropic both use server-sent events, but the chunk format and the way a stream signals completion aren't identical. Handling this in one place means every provider implementation stays simple, and the quirks live in exactly one file instead of scattered across the codebase.

Configuration, not code changes

The payoff of all this structure is that switching providers, or splitting traffic between them, becomes a configuration change:

llm_providers:
  classification:
    primary: mistral-small
    fallback: [openai-gpt-4o-mini]
  generation:
    primary: claude-sonnet
    fallback: [openai-gpt-4o, ollama-local]
  sensitive:
    primary: ollama-local
    fallback: []

When a new model launches, or an existing provider's pricing shifts, updating this file is the entire change. No controller touches an SDK directly, no deployment is needed beyond a config update, and the routing and failover logic keeps working exactly as before.

Where this fits in a broader AI strategy

None of this is about avoiding good providers. OpenAI and Anthropic both build strong models, and using them well is still the right call for most generation tasks. The point is that your application's availability and cost structure shouldn't be permanently welded to one vendor's decisions. Teams that build this abstraction early spend an extra day or two on the interface and provider implementations. Teams that skip it end up spending weeks retrofitting it later, usually right after an outage or a pricing change forces the question.

If your SaaS product is adding its first AI features, or if you're already locked into a single provider and want a path out without a rewrite, this is the kind of architecture decision worth getting right before it's load-bearing. Wolf-Tech works with SaaS teams on exactly this kind of custom software development and tech stack strategy, building the abstraction layers that keep AI features flexible as the provider landscape keeps shifting under them.

Questions about your own AI integration, or want a second opinion on an existing setup before it becomes harder to change? Reach out at hello@wolf-tech.io or visit wolf-tech.io.