Enhanced API client implementation with proper request queueing during token refresh

API Client with Request Queueing

Overview

This document outlines our implementation of the request queueing pattern for API clients to handle token refresh scenarios efficiently. When multiple API requests encounter an expired access token simultaneously, our approach ensures that only one token refresh operation occurs, while other requests are queued and automatically retried upon successful token refresh, providing a seamless user experience without exposing authentication complexities to application logic.

This document focuses on the request queueing mechanism specifically. For details about the overall authentication architecture, token management, and security considerations, see the Authentication Architecture documentation.

Purpose & Scope

This reference targets developers implementing or maintaining API communication layers. It covers:

The problem of concurrent API requests during token expiration
Request queueing implementation strategies
Handling of public vs. authenticated endpoints
Error handling during token refresh
Testing strategies for request queueing

Prerequisites

To get the most out of this guide on request queueing, we recommend you first understand these core concepts and review the related architectural documents:

API Client Architecture: You should be thoroughly familiar with the mandatory patterns defined in the API Client Architecture, particularly the roles of BaseApiClient, PublicApiClient, and AuthenticatedApiClient.
Authentication Architecture: A strong grasp of the concepts in the Authentication Architecture document is crucial. This includes understanding how tokenService manages tokens, how authEvents are used, and the basic token refresh flow.
Axios Interceptors: The queueing logic is implemented using Axios request and response interceptors. Familiarity with how these work is essential.
JavaScript Asynchronous Operations: A good understanding of Promises, async/await is necessary to follow the code examples.

The Problem

When multiple API requests are made simultaneously and the access token has expired, each request will receive a 401 response. Without proper handling, this would trigger multiple token refresh requests, causing:

Race conditions
Invalid refresh token errors
Poor user experience
Potential security issues

Solution: Request Queue Pattern

We implement a queue mechanism that:

Detects when a refresh is in progress
Queues subsequent failed requests
Processes all queued requests after refresh completes
Handles refresh failures gracefully

Handling Public vs. Authenticated API Endpoints

In our project, the differentiation between public API endpoints (which do not require authentication tokens) and authenticated endpoints (which do) is managed by our established standard API client architecture. This architecture, detailed in the API Client Architecture, utilizes separate PublicApiClient and AuthenticatedApiClient instances.

PublicApiClient: Used for endpoints like login, registration, or public data fetching. It does not attach authentication tokens.
AuthenticatedApiClient: Used for endpoints requiring user authentication. It handles attaching access tokens and is responsible for the token refresh mechanism, including request queueing.

This document focuses on the request queueing mechanism implemented within the AuthenticatedApiClient when a token refresh is triggered. The following sections will detail this queueing logic and how it integrates into the token refresh process of the AuthenticatedApiClient.

For comprehensive details on how PublicApiClient and AuthenticatedApiClient are structured and used, please refer to:

Complete Implementation with Queue (within `AuthenticatedApiClient`)

This section details how request queueing is integrated into our standard AuthenticatedApiClient. The AuthenticatedApiClientWithQueue class shown below extends the BaseApiClient and incorporates the queueing logic.

About the `AuthenticatedApiClientWithQueue` Example

The AuthenticatedApiClientWithQueue class presented below demonstrates the complete integration of request queueing into an authenticated API client.

It illustrates the fully-featured version of the AuthenticatedApiClient (which is introduced in the API Client Architecture and further detailed in the Authentication Architecture guide).
In your actual project codebase, this queueing logic would typically be part of your main AuthenticatedApiClient. The name AuthenticatedApiClientWithQueue is used here for clear focus on the queueing enhancements.
It extends the canonical BaseApiClient, whose full definition is in the API Client Architecture.
It relies on tokenService and authEvents from the Authentication Architecture guide. For this example, assume these services provide methods like getAccessToken, getRefreshToken, setTokens, clearTokens, and an EventEmitter for auth-related events.

// Conceptual Path: core/shared/api/authenticatedClientWithQueue.ts
// import { BaseApiClient } from './base'; // This import would be from your project's actual shared location for BaseApiClient
import axios, { AxiosInstance, AxiosRequestConfig, AxiosError, AxiosResponse } from 'axios'; // Still needed for refreshClient
import { tokenService } from '@/core/domains/auth/tokenService'; // Actual path should be verified
import { authEvents } from '@/core/domains/auth/events'; // Actual path should be verified
 
// BaseApiClient is NOT redefined in this code block. 
// AuthenticatedApiClientWithQueue extends the canonical BaseApiClient, 
// whose full definition and explanation can be found in the "API Client Architecture Guide".
 
export class AuthenticatedApiClientWithQueue extends BaseApiClient { // BaseApiClient would be imported
  private isRefreshing = false;
  private refreshPromise: Promise<string | void> | null = null;
  private requestQueue: Array<{
    config: AxiosRequestConfig; // Store original config
    resolve: (value: any) => void;
    reject: (error: any) => void;
  }> = [];
 
  constructor(config: AxiosRequestConfig) { 
    super(config); 
    this.setupInterceptors();
  }
 
  private setupInterceptors() {
    // Request interceptor - add token to requests
    this.client.interceptors.request.use(
      async (config) => {
        const token = await tokenService.getAccessToken();
        if (token) {
          config.headers = config.headers || {};
          config.headers.Authorization = `Bearer ${token}`;
        }
        return config;
      },
      (error) => Promise.reject(error)
    );
 
    // Response interceptor - handle token refresh and queueing
    this.client.interceptors.response.use(
      (response) => response, 
      async (error: AxiosError) => {
        const originalRequest = error.config as AxiosRequestConfig & { _retry?: boolean };
 
        if (!originalRequest || error.response?.status !== 401 || originalRequest._retry) {
          return Promise.reject(error);
        }
 
        originalRequest._retry = true;
 
        if (this.isRefreshing) {
          return this.queueRequest(originalRequest);
        } else {
          this.isRefreshing = true;
          this.refreshPromise = this.handleTokenRefresh().finally(() => {
            this.isRefreshing = false;
          });
 
          try {
            const newToken = await this.refreshPromise;
            if (typeof newToken === 'string') { 
              originalRequest.headers!.Authorization = `Bearer ${newToken}`;
              return this.client(originalRequest);
            } else {
                return Promise.reject(error); 
            }
          } catch (refreshError) {
            return Promise.reject(refreshError);
          }
        }
      }
    );
  }
 
  private async handleTokenRefresh(): Promise<string | void> {
    try {
      const refreshTokenValue = await tokenService.getRefreshToken();
      if (!refreshTokenValue) {
        throw new Error('No refresh token available');
      }
 
      const refreshClient = axios.create({ 
        baseURL: this.client.defaults.baseURL, 
      });
      const response = await refreshClient.post('/auth/refresh', { 
        refreshToken: refreshTokenValue,
      });
 
      const { accessToken, refreshToken: newRefreshToken } = response.data;
      await tokenService.setTokens(accessToken, newRefreshToken);
      authEvents.emit('auth:tokenRefreshed', { accessToken });
 
      this.processRequestQueue(accessToken);
      this.refreshPromise = null; 
      return accessToken; 
 
    } catch (error) {
      authEvents.emit('auth:sessionExpired'); 
      this.rejectRequestQueue(error); 
      await tokenService.clearTokens(); 
      this.refreshPromise = null; 
      return Promise.reject(error); 
    }
  }
 
  private queueRequest(config: AxiosRequestConfig): Promise<any> {
    return new Promise((resolve, reject) => {
      this.requestQueue.push({ config, resolve, reject });
    });
  }
 
  private processRequestQueue(token: string) {
    this.requestQueue.forEach(({ config, resolve, reject }) => {
      config.headers!.Authorization = `Bearer ${token}`;
      this.client(config)
        .then(resolve)
        .catch(reject);
    });
    this.requestQueue = [];
  }
 
  private rejectRequestQueue(error: any) {
    this.requestQueue.forEach(({ reject }) => {
      reject(error);
    });
    this.requestQueue = [];
  }
  // The HTTP helper methods (get, post, put, delete) are inherited from BaseApiClient
  // and do not need to be redefined here unless overridden.
}
 
// Note on instantiation:
// The AuthenticatedApiClientWithQueue class defined above would typically be instantiated 
// as a singleton. This process should align with patterns detailed in the 
// "API Client Architecture Guide" and the "Multi-API Architecture" guide.
// This often involves creating a specific configuration (e.g., with the baseURL 
// from environment variables) and exporting the client instance from a central module.
// The following illustrates a conceptual example:
 
// Conceptual example: core/shared/api/client.ts 
 
// import { AuthenticatedApiClientWithQueue } from './path/to/authenticatedClientWithQueue'; // Actual import path
// import { AxiosRequestConfig } from 'axios';
 
// const apiConfig: AxiosRequestConfig = {
//   baseURL: process.env.REACT_APP_API_URL || '/api',
//   // other shared configs for this specific client instance
// };
 
// export const authenticatedApiWithQueue = 
//   new AuthenticatedApiClientWithQueue(apiConfig);

Request Queue Core Functionality

The core of the request queueing mechanism, as implemented within the AuthenticatedApiClientWithQueue (which aligns with the project's AuthenticatedApiClient pattern), revolves around these methods:

The `queueRequest` Method

This method is invoked when an API request fails due to an expired token (401 error) and a token refresh operation is already underway. It defers the execution of the failed request by adding it to a queue.

Key Function: queueRequest (within AuthenticatedApiClientWithQueue)

/**
 * Adds a request to the queue during token refresh.
 * @param config The original request configuration that failed.
 * @returns A new Promise that resolves/rejects when the queued request is processed.
 */
private queueRequest(config: AxiosRequestConfig): Promise<any> {
  return new Promise((resolve, reject) => {
    this.requestQueue.push({ config, resolve, reject });
  });
}

The `processRequestQueue` Method

Called after a successful token refresh, this method iterates over all requests in the queue, updates their authorization headers with the new token, and retries them.

/**
 * Processes all requests currently in the queue with the new token.
 * @param token The new access token.
 */
private processRequestQueue(token: string) {
  this.requestQueue.forEach(({ config, resolve, reject }) => {
    config.headers!.Authorization = `Bearer ${token}`;
    this.client(config) // Uses the client instance from BaseApiClient
      .then(resolve)
      .catch(reject);
  });
  this.requestQueue = []; // Clear the queue
}

The `rejectRequestQueue` Method

If the token refresh attempt fails, this method is called to reject all pending requests in the queue, ensuring that no requests are left hanging and that the application can handle the refresh failure appropriately.

/**
 * Rejects all requests currently in the queue with the given error.
 * @param error The error that occurred (e.g., during token refresh).
 */
private rejectRequestQueue(error: any) {
  this.requestQueue.forEach(({ reject }) => {
    reject(error);
  });
  this.requestQueue = []; // Clear the queue
}

These methods work in concert within the response interceptor of the AuthenticatedApiClientWithQueue to provide seamless handling of token expiration and refresh.

Integration in the Response Interceptor

The updated response interceptor in AuthenticatedApiClientWithQueue uses these methods:

// Inside AuthenticatedApiClientWithQueue's setupInterceptors method:
this.client.interceptors.response.use(
  (response) => response,
  async (error: AxiosError) => {
    const originalRequest = error.config as AxiosRequestConfig & { _retry?: boolean };
 
    if (!originalRequest || error.response?.status !== 401 || originalRequest._retry) {
      return Promise.reject(error);
    }
    originalRequest._retry = true;
 
    if (this.isRefreshing) {
      return this.queueRequest(originalRequest);
    } else {
      this.isRefreshing = true;
      this.refreshPromise = this.handleTokenRefresh().finally(() => {
        this.isRefreshing = false;
      });
 
      try {
        const newToken = await this.refreshPromise;
        if (typeof newToken === 'string') {
          originalRequest.headers!.Authorization = `Bearer ${newToken}`;
          return this.client(originalRequest);
        } else {
          return Promise.reject(error); 
        }
      } catch (refreshError) {
        return Promise.reject(refreshError);
      }
    }
  }
);

Visualizing the Queue Flow

Testing Request Queueing

Testing the request queueing functionality is critical to ensure it works correctly under various conditions. Below are comprehensive test examples covering different scenarios.

Setup For Testing

First, set up the test environment with the necessary mocks. Note that AuthenticatedApiClientWithQueue would typically be imported from its definition, and its constructor might require specific configuration (e.g., baseURL) based on your project setup.

// core/shared/api/__tests__/request-queue.test.ts
// import { apiClient } from '../client'; // Removed: apiClient is no longer used directly here
import { AuthenticatedApiClientWithQueue } from '../authenticatedClientWithQueue'; // Assuming this is the path to the class defined above
import { tokenService } from '@/core/domains/auth/tokenService';
import MockAdapter from 'axios-mock-adapter';
import { authEvents } from '@/core/domains/auth/events';
import { AxiosRequestConfig } from 'axios';
 
describe('API Client Request Queueing with AuthenticatedApiClientWithQueue', () => {
  let authenticatedClient: AuthenticatedApiClientWithQueue;
  let mockAxios: MockAdapter;
  const mockTokenRefreshEmit = jest.fn();
  const mockSessionExpiredEmit = jest.fn();
 
  const clientConfig: AxiosRequestConfig = { baseURL: '/' }; // Example base URL
 
  beforeEach(() => {
    authenticatedClient = new AuthenticatedApiClientWithQueue(clientConfig);
    // The client property is protected in BaseApiClient, so we cast to any to access it for mocking.
    // In a real test setup, you might have a way to get the underlying Axios instance or use a more integrated test approach.
    mockAxios = new MockAdapter((authenticatedClient as any).client);
    
    jest.spyOn(tokenService, 'getAccessToken').mockResolvedValue('expired-token');
    jest.spyOn(tokenService, 'getRefreshToken').mockResolvedValue('valid-refresh-token');
    jest.spyOn(tokenService, 'setTokens').mockResolvedValue(undefined);
    jest.spyOn(tokenService, 'clearTokens').mockResolvedValue(undefined);
    
    authEvents.on('auth:tokenRefreshed', mockTokenRefreshEmit);
    authEvents.on('auth:sessionExpired', mockSessionExpiredEmit);
  });
 
  afterEach(() => {
    mockAxios.reset();
    jest.clearAllMocks();
    authEvents.off('auth:tokenRefreshed', mockTokenRefreshEmit);
    authEvents.off('auth:sessionExpired', mockSessionExpiredEmit);
  });
});

Test Case 1: Successful Token Refresh and Request Retry

// Test Case 1: Successful Token Refresh and Request Retry
test('should handle token refresh and retry the original request when a 401 is returned', async () => {
  // Mock a 401 response for the first request, then a 200 for the retry
  mockAxios.onGet('/api/data').replyOnce(401).onGet('/api/data').replyOnce(200, { success: true });
  
  // Mock the token refresh response
  mockAxios.onPost('/api/auth/refresh').replyOnce(200, {
    access_token: 'new-access-token',
    refresh_token: 'new-refresh-token'
  });
  
  // Override to simulate a real token refresh
  jest.spyOn(tokenService, 'getRefreshToken').mockResolvedValue('valid-refresh-token');
  jest.spyOn(authenticatedClient as any, 'handleTokenRefresh').mockImplementation(async () => {
    const resp = await (authenticatedClient as any).client.post('/api/auth/refresh', { 
      refreshToken: 'valid-refresh-token' 
    });
    
    const tokens = resp.data;
    await tokenService.setTokens(tokens.access_token, tokens.refresh_token);
    authEvents.emit('auth:tokenRefreshed');
    return tokens.access_token;
  });
  
  // Make the request
  const response = await authenticatedClient.get('/api/data');
  
  // Assertions
  expect(response.data).toEqual({ success: true });
  expect(tokenService.setTokens).toHaveBeenCalledWith('new-access-token', 'new-refresh-token');
  expect(mockTokenRefreshEmit).toHaveBeenCalled();
});

Test Case 2: Multiple Concurrent Requests During Token Refresh

// Test Case 2: Multiple Concurrent Requests During Token Refresh
test('should queue multiple requests during token refresh and process them after refresh', async () => {
  // Set up mock responses
  // All first attempts return 401
  mockAxios.onGet('/api/data/1').replyOnce(401);
  mockAxios.onGet('/api/data/2').replyOnce(401);
  mockAxios.onGet('/api/data/3').replyOnce(401);
  
  // All retries return 200 with different data
  mockAxios.onGet('/api/data/1').replyOnce(200, { id: 1 });
  mockAxios.onGet('/api/data/2').replyOnce(200, { id: 2 });
  mockAxios.onGet('/api/data/3').replyOnce(200, { id: 3 });
  
  // Token refresh response
  mockAxios.onPost('/api/auth/refresh').replyOnce(200, {
    access_token: 'new-access-token',
    refresh_token: 'new-refresh-token'
  });
  
  // Setup token refresh mock
  let refreshComplete = false;
  let refreshPromise: Promise<void>;
  
  jest.spyOn(authenticatedClient as any, 'handleTokenRefresh').mockImplementation(async () => {
    refreshPromise = new Promise<void>(resolve => {
      setTimeout(() => {
        refreshComplete = true;
        resolve();
      }, 50); // Small delay to simulate network latency
    }).then(async () => {
      const resp = await (authenticatedClient as any).client.post('/api/auth/refresh');
      const tokens = resp.data;
      await tokenService.setTokens(tokens.access_token, tokens.refresh_token);
      authEvents.emit('auth:tokenRefreshed');
      return tokens.access_token;
    });
    
    return refreshPromise;
  });
  
  // Make concurrent requests
  const request1 = authenticatedClient.get('/api/data/1');
  const request2 = authenticatedClient.get('/api/data/2');
  const request3 = authenticatedClient.get('/api/data/3');
  
  // Wait for all requests to complete
  const [response1, response2, response3] = await Promise.all([request1, request2, request3]);
  
  // Assertions
  expect(refreshComplete).toBe(true);
  expect(response1.data).toEqual({ id: 1 });
  expect(response2.data).toEqual({ id: 2 });
  expect(response3.data).toEqual({ id: 3 });
  expect(tokenService.setTokens).toHaveBeenCalledTimes(1);
  expect(mockTokenRefreshEmit).toHaveBeenCalledTimes(1);
});

Test Case 3: Failed Token Refresh

// Test Case 3: Failed Token Refresh
test('should reject all queued requests if token refresh fails', async () => {
  // Set up mock responses
  mockAxios.onGet('/api/data/1').replyOnce(401);
  mockAxios.onGet('/api/data/2').replyOnce(401);
  
  // Token refresh fails
  mockAxios.onPost('/api/auth/refresh').replyOnce(403, { 
    error: 'Invalid refresh token' 
  });
  
  // Setup token refresh mock that will fail
  jest.spyOn(authenticatedClient as any, 'handleTokenRefresh').mockImplementation(async () => {
    try {
      const resp = await (authenticatedClient as any).client.post('/api/auth/refresh');
      return resp.data.access_token;
    } catch (error) {
      await tokenService.clearTokens();
      authEvents.emit('auth:sessionExpired');
      throw error;
    }
  });
  
  // Make requests
  const request1 = authenticatedClient.get('/api/data/1');
  const request2 = authenticatedClient.get('/api/data/2');
  
  // Assertions
  await expect(request1).rejects.toThrow();
  await expect(request2).rejects.toThrow();
  expect(tokenService.clearTokens).toHaveBeenCalled();
  expect(mockSessionExpiredEmit).toHaveBeenCalled();
});

Design Principles

Our request queueing implementation follows these key design principles:

Separation of Concerns: The queueing mechanism is encapsulated within the AuthenticatedApiClientWithQueue class, keeping authentication complexities hidden from application code.
Efficiency: Only one token refresh operation is performed when multiple requests fail simultaneously.
Transparency: Requests are automatically retried after token refresh without requiring application code intervention.
Robustness: Failed token refreshes are handled gracefully, with appropriate error propagation and event notifications.
Progressive Enhancement: The queueing mechanism enhances the base API client capabilities defined in the API Client Architecture.

Alternative Approaches Considered

We evaluated several approaches before settling on our current implementation:

1. Global Axios Instance with Queue

Approach: Using a single global Axios instance with interceptors managing the queue.

Pros:

Simpler implementation with less code
Centralized queue management

Cons:

Less flexibility for API clients with different base URLs or configurations
Harder to test in isolation
Potential for queue conflicts between different API domains

2. Redux Middleware for API Requests

Approach: Using Redux middleware to manage API requests and handle token refreshing.

Pros:

Integrated with state management
Potentially more visibility into request states

Cons:

Tighter coupling between API layer and state management
More complex implementation
Not suitable for non-Redux applications

3. Separate Token Manager Service

Approach: Creating a separate service to manage token refresh, with API clients checking token validity before requests.

Pros:

Clearer separation of responsibilities
Potentially more reusable across different API client implementations

Cons:

More complex coordination between services
Potential for race conditions if not carefully implemented
Additional overhead of token validity checks before each request

Conclusion

The request queueing pattern implemented in our AuthenticatedApiClientWithQueue provides a robust solution for handling token refreshes in a way that's transparent to application code. By following the principles and implementation described in this document, developers can ensure that API authentication remains seamless even when tokens expire during active application use.

This implementation builds upon the foundation established in the API Client Architecture and complements the authentication strategies outlined in the Authentication Architecture document.

API Client with Request Queueing

On this page