Skip to main content
Build flexible AI applications that work with multiple providers by abstracting provider logic, implementing fallback strategies, and routing requests based on cost, speed, or feature requirements.
Multi-provider architecture provides resilience. By supporting multiple AI providers, your application can gracefully handle rate limits, outages, and pricing changes without code changes.

​
Provider Configuration Pattern

Create a configuration-driven approach instead of hardcoding providers: 
// config/providers.ts
export interface ProviderConfig {
  name: string;
  baseUrl: string;
  models: string[];
  features: string[];
  priority: number;
}

export const PROVIDERS: Record<string, ProviderConfig> = {
  openai: {
    name: 'OpenAI',
    baseUrl: 'https://api.openai.com/v1/chat/completions',
    models: ['gpt-4o', 'gpt-4o-mini', 'o3-mini'],
    features: ['streaming', 'function-calling', 'vision'],
    priority: 1
  },
  anthropic: {
    name: 'Anthropic',
    baseUrl: 'https://api.anthropic.com/v1/messages',
    models: ['claude-opus-4-5-20250514', 'claude-sonnet-4-5-20250514', 'claude-haiku-3-5-20241022'],
    features: ['streaming', 'function-calling', 'vision'],
    priority: 2
  },
  google: {
    name: 'Google',
    baseUrl: 'https://generativelanguage.googleapis.com/v1beta/models',
    models: ['gemini-2.5-flash', 'gemini-2.0-flash'],
    features: ['streaming', 'vision', 'multimodal'],
    priority: 3
  },
  groq: {
    name: 'Groq',
    baseUrl: 'https://api.groq.com/openai/v1/chat/completions',
    models: ['llama-3.3-70b-versatile', 'llama-3.1-8b-instant'],
    features: ['streaming', 'low-latency'],
    priority: 4
  }
};
For a complete provider abstraction class that wraps this configuration with unified request/response handling, see the full example in the Node.js SDK reference.

​
Provider Switching with Configuration

​
Environment-Based Selection

Switch providers via environment variables: 
// lib/get-provider.ts
import { PROVIDERS } from '@/config/providers';

export function getProviderConfig(providerName?: string) {
  const name = providerName || process.env.AI_PROVIDER || 'openai';
  const config = PROVIDERS[name];

  if (!config) {
    throw new Error(`Unknown provider: ${name}`);
  }

  return config;
}
Environment configuration: 
# .env.production
AI_PROVIDER=openai

# .env.development
AI_PROVIDER=groq  # Use faster Groq for development

​
Dynamic Provider Selection

Choose provider based on request characteristics: 
export function selectProvider(request: {
  requiresVision?: boolean;
  requiresFunctionCalling?: boolean;
  prioritizeSpeed?: boolean;
  prioritizeCost?: boolean;
}): string {
  // Vision required
  if (request.requiresVision) {
    return 'google';  // Gemini Pro Vision
  }

  // Speed priority (low latency)
  if (request.prioritizeSpeed) {
    return 'groq';  // Ultra-fast inference
  }

  // Cost priority
  if (request.prioritizeCost) {
    return 'groq';  // Most cost-effective
  }

  // Default: OpenAI for quality
  return 'openai';
}

// Usage
const providerName = selectProvider({ prioritizeSpeed: true });
const config = PROVIDERS[providerName];

​
Model-Based Routing

Route to providers based on desired model: 
export function getProviderForModel(modelName: string): string {
  for (const [providerKey, config] of Object.entries(PROVIDERS)) {
    if (config.models.includes(modelName)) {
      return providerKey;
    }
  }
  throw new Error(`No provider found for model: ${modelName}`);
}

// Usage
const providerName = getProviderForModel('claude-opus-4-5-20250514');
const config = PROVIDERS[providerName];

​
Fallback Strategies

​
Sequential Fallback

Try providers in priority order until one succeeds: 
export async function completionWithFallback(
  forwardToken: string,
  messages: Array<{ role: string; content: string }>,
  model?: string
): Promise<any> {
  // Sort providers by priority
  const sortedProviders = Object.entries(PROVIDERS)
    .sort(([, a], [, b]) => a.priority - b.priority)
    .map(([key]) => key);

  const errors: Array<{ provider: string; error: string }> = [];

  for (const providerName of sortedProviders) {
    try {
      console.log(`Attempting provider: ${providerName}`);
      const config = PROVIDERS[providerName];

      const response = await fetch(
        `https://api.lavapayments.com/v1/forward?u=${encodeURIComponent(config.baseUrl)}`,
        {
          method: 'POST',
          headers: {
            'Authorization': `Bearer ${forwardToken}`,
            'Content-Type': 'application/json'
          },
          body: JSON.stringify({
            model: model || config.models[0],
            messages,
            temperature: 0.7
          })
        }
      );

      if (!response.ok) {
        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
      }

      console.log(`Success with provider: ${providerName}`);
      return await response.json();
    } catch (error) {
      const errorMessage = error instanceof Error ? error.message : 'Unknown error';
      errors.push({ provider: providerName, error: errorMessage });
      console.warn(`Provider ${providerName} failed:`, errorMessage);
      continue;
    }
  }

  // All providers failed
  throw new Error(
    `All providers failed:\n${errors.map(e => `- ${e.provider}: ${e.error}`).join('\n')}`
  );
}

// Usage
try {
  const response = await completionWithFallback(
    forwardToken,
    [{ role: 'user', content: 'Hello' }]
  );
  console.log('Response:', response);
} catch (error) {
  console.error('All providers exhausted:', error);
}

​
Smart Retry with Exponential Backoff

Add exponential backoff for transient errors before falling back to the next provider: 
async function completionWithRetry(
  forwardToken: string,
  messages: Array<{ role: string; content: string }>,
  maxRetries = 3
): Promise<any> {
  const sortedProviders = Object.keys(PROVIDERS)
    .sort((a, b) => PROVIDERS[a].priority - PROVIDERS[b].priority);

  for (const providerName of sortedProviders) {
    let retries = 0;

    while (retries < maxRetries) {
      try {
        const config = PROVIDERS[providerName];
        const response = await fetch(
          `https://api.lavapayments.com/v1/forward?u=${encodeURIComponent(config.baseUrl)}`,
          {
            method: 'POST',
            headers: {
              'Authorization': `Bearer ${forwardToken}`,
              'Content-Type': 'application/json'
            },
            body: JSON.stringify({
              model: config.models[0],
              messages
            })
          }
        );

        if (!response.ok) {
          throw new Error(`HTTP ${response.status}`);
        }

        return await response.json();
      } catch (error) {
        const errorMessage = error instanceof Error ? error.message : '';

        // Check if error is retryable (rate limit, timeout)
        const isRetryable = errorMessage.includes('429') ||
                           errorMessage.includes('timeout') ||
                           errorMessage.includes('503');

        if (!isRetryable || retries >= maxRetries - 1) {
          // Non-retryable error or max retries reached, try next provider
          break;
        }

        // Exponential backoff: 1s, 2s, 4s
        const delay = Math.pow(2, retries) * 1000;
        console.log(`Retrying ${providerName} in ${delay}ms...`);
        await new Promise(resolve => setTimeout(resolve, delay));

        retries++;
      }
    }
  }

  throw new Error('All providers and retries exhausted');
}

​
Cost-Aware Routing

Fallback to cheaper providers when budget is a concern: 
interface ProviderWithCost extends ProviderConfig {
  estimatedCostPer1kTokens: number;
}

async function costAwareFallback(
  forwardToken: string,
  messages: Array<{ role: string; content: string }>,
  maxCostUsd: number
): Promise<any> {
  // Try cheapest providers first
  const providersByCost = Object.entries(PROVIDERS)
    .sort(([, a], [, b]) =>
      (a as ProviderWithCost).estimatedCostPer1kTokens -
      (b as ProviderWithCost).estimatedCostPer1kTokens
    );

  for (const [providerName, config] of providersByCost) {
    try {
      const response = await fetch(
        `https://api.lavapayments.com/v1/forward?u=${encodeURIComponent(config.baseUrl)}`,
        {
          method: 'POST',
          headers: {
            'Authorization': `Bearer ${forwardToken}`,
            'Content-Type': 'application/json'
          },
          body: JSON.stringify({
            model: config.models[0],
            messages
          })
        }
      );

      if (!response.ok) throw new Error(`HTTP ${response.status}`);
      return await response.json();
    } catch (error) {
      continue;
    }
  }

  throw new Error(`No provider within budget: $${maxCostUsd}`);
}

​
Feature Detection

Check provider capabilities before using advanced features: 
function hasFeature(providerName: string, feature: string): boolean {
  const config = PROVIDERS[providerName];
  return config?.features.includes(feature) || false;
}

// Usage
if (hasFeature('openai', 'function-calling')) {
  // Include function definitions in request
}

if (hasFeature('google', 'vision')) {
  // Include image URLs in request
}

​
Graceful Feature Degradation

Fallback when features aren’t supported: 
async function completionWithFeatureFallback(
  forwardToken: string,
  messages: Array<{ role: string; content: string }>,
  options: { functions?: any[] } = {}
): Promise<any> {
  const preferredProvider = 'openai';

  try {
    if (options.functions && !hasFeature(preferredProvider, 'function-calling')) {
      console.warn('Function calling not supported, removing from request');
      delete options.functions;
    }

    const config = PROVIDERS[preferredProvider];
    const response = await fetch(
      `https://api.lavapayments.com/v1/forward?u=${encodeURIComponent(config.baseUrl)}`,
      {
        method: 'POST',
        headers: {
          'Authorization': `Bearer ${forwardToken}`,
          'Content-Type': 'application/json'
        },
        body: JSON.stringify({
          model: config.models[0],
          messages,
          ...(options.functions ? { tools: options.functions } : {})
        })
      }
    );

    if (!response.ok) throw new Error(`HTTP ${response.status}`);
    return await response.json();
  } catch (error) {
    // Fallback to another provider without advanced features
    const fallbackConfig = PROVIDERS['groq'];
    const response = await fetch(
      `https://api.lavapayments.com/v1/forward?u=${encodeURIComponent(fallbackConfig.baseUrl)}`,
      {
        method: 'POST',
        headers: {
          'Authorization': `Bearer ${forwardToken}`,
          'Content-Type': 'application/json'
        },
        body: JSON.stringify({
          model: fallbackConfig.models[0],
          messages
        })
      }
    );

    if (!response.ok) throw new Error(`HTTP ${response.status}`);
    return await response.json();
  }
}

​
Troubleshooting

Check:
  • Exception is being caught properly in try/catch
  • Provider priority order is correct (lower number = higher priority)
  • Error is thrown (not just logged) when provider fails
Debug:
for (const providerName of sortedProviders) {
  try {
    console.log('Trying provider:', providerName);
    const response = await makeRequest(providerName);
    console.log('Success!');
    return response;
  } catch (error) {
    console.error('Provider failed:', providerName, error);
    // IMPORTANT: Must continue, not return/throw here
    continue;
  }
}
Issue: Provider claims to support feature but request failsReasons:
  • Provider configuration outdated (features changed)
  • Feature available but different API format required
  • Feature requires specific model (not all models support all features)
Solution: Add model-specific feature checks:
function hasFeature(providerName: string, feature: string, model?: string): boolean {
  const config = PROVIDERS[providerName];
  if (!config?.features.includes(feature)) {
    return false;
  }

  // Model-specific feature support
  if (feature === 'vision' && model) {
    return model.includes('vision') || model.includes('4');
  }

  return true;
}
Problem: Estimated costs don’t match actual Lava costsExplanation:
  • Lava costs include provider base cost + merchant fee + service charge
  • Provider pricing varies by model and usage
  • Costs are only accurate AFTER request completes
Solution: Track usage from response body and calculate cost based on your pricing:
const usage = data.usage?.total_tokens || 0;
const requestId = response.headers.get('x-lava-request-id');

// Calculate cost based on your configured pricing
console.log('Tokens used:', usage, 'Request ID:', requestId);
Cause: Provider uses different response format than expectedSolution: Handle multiple response formats:
function extractContent(data: any): string {
  // Try each known format
  if (data.choices?.[0]?.message?.content) return data.choices[0].message.content;
  if (data.content?.[0]?.text) return data.content[0].text;
  if (data.candidates?.[0]?.content?.parts?.[0]?.text) {
    return data.candidates[0].content.parts[0].text;
  }

  // Log unknown format for debugging
  console.error('Unknown response format:', JSON.stringify(data, null, 2));
  throw new Error('Unknown response format');
}

​
Next Steps

Streaming

Handle streaming responses across different providers

Forward Proxy & Authentication

Understand URL encoding and provider routing

Supported Providers

See all supported providers and their features

Webhooks

Receive real-time notifications for usage events