# Circuit Breaker Documentation

Circuit Breaker is a higher-order function utility for protecting polling requests. When consecutive errors reach a threshold, the circuit breaker opens and rejects all subsequent calls. Only the recovery mechanism (if enabled) can make requests to detect if the service has recovered, thus preventing frequent failed requests from burdening the system.

## Features

- ✅ **Automatic Circuit Breaking**: Automatically opens the circuit breaker when consecutive errors reach the threshold
- ✅ **Request Rejection**: During circuit open state, all requests are immediately rejected and return `undefined` without throwing errors
- ✅ **Recovery Mechanism**: Automatically detects service recovery by periodically calling the original function
- ✅ **Pause/Resume**: Temporarily pause the circuit breaker to allow requests to pass through, with automatic state restoration on resume
- ✅ **State Management**: Manage states of different instances through keys, with independent reset support
- ✅ **React Hook Support**: Provides `useCircuitBreaker` Hook for convenient use in React components
- ✅ **Type Safety**: Full TypeScript type support

## API Reference

### `withCircuitBreaker`

Higher-order function for wrapping any function that returns a Promise.

```typescript
function withCircuitBreaker<T extends (...args: any[]) => Promise<any>>(config: CircuitBreakerConfig): (fn: T) => T
```

### `useCircuitBreaker`

React Hook version that returns the wrapped function and a clear function.

```typescript
function useCircuitBreaker<T extends (...args: any[]) => Promise<any>>(
  fn: T,
  config: CircuitBreakerConfig
): [T, () => void]
```

### `CircuitBreakerConfig`

Configuration interface:

```typescript
interface CircuitBreakerConfig {
  /** Unique identifier for caching error state and resetting */
  key: string
  /** Number of consecutive errors before opening the circuit */
  errorThreshold: number
  /** Recovery detection interval in milliseconds */
  recoveryMs: number
  /** If false, circuit breaker is disabled and function executes normally (default: true) */
  enabled?: boolean
  /** If true, automatically poll the original function for recovery detection after recoveryMs (default: true) */
  recovery?: boolean
  /** Callback executed when circuit breaker opens */
  onThrottleOpen?: () => void
  /** Callback executed when recovery detection succeeds (function call succeeds). `isDuringPaused` indicates if recovery was triggered during a paused state */
  onRecoverySuccess?: (isDuringPaused: boolean) => void
}
```

### Utility Functions

- `resetCircuitBreaker(key: string)`: Reset the circuit breaker state for the specified key
- `getCircuitBreakerState(key: string)`: Get the current state for the specified key (returns `errorCount`, `isOpen`, `throttleCount`, and `isPaused`)
- `pauseCircuitBreaker(key: string)`: Pause the circuit breaker for the specified key. During pause, requests are allowed to pass through. If a request succeeds, the circuit breaker is reset. If a request fails, the original state is preserved.
- `resumeCircuitBreaker(key: string)`: Resume the circuit breaker for the specified key, restoring the state that was saved before pausing

## Usage Examples

### Example 1: React Hook Usage (ChartCard Example)

This is a real-world usage example from the `ChartCard` component, demonstrating how to use `useCircuitBreaker` in a React component:

```typescript
import { useCircuitBreaker, CircuitBreakerConfig } from 'dbaas/utils/circuit-breaker'
import { useEffect, useMemo } from 'react'

const ERROR_THRESHOLD = 50
const RECOVERY_MS = 1 * 60 * 1000 // 1 minute

const ChartCard: React.FC<ChartCardProps> = ({
  sourceURI,
  // ... other props
}) => {
  // Use useMemo to cache config, avoiding unnecessary recreation
  const circuitBreakerConfig = useMemo<CircuitBreakerConfig>(
    () => ({
      key: sourceURI, // Use sourceURI as unique identifier
      errorThreshold: ERROR_THRESHOLD, // Open circuit after 50 consecutive errors
      recoveryMs: RECOVERY_MS, // Recovery detection interval
      enabled: enableCircuitBreaker, // Enable/disable circuit breaker
      // recovery defaults to true, so automatic recovery detection is enabled by default
      onThrottleOpen: () => {
        // Called when circuit breaker opens
        // Log error when circuit opens
        log.api.error(API_THROTTLED_ERROR_MSG, new Error('Circuit breaker opened'))
        console.log('Circuit breaker opened, all requests will be rejected')
      },
      onRecoverySuccess: (isDuringPaused) => {
        // Called when recovery detection succeeds
        if (isDuringPaused) {
          console.log('Recovery successful during pause, circuit breaker closed')
        } else {
          console.log('Recovery successful via automatic detection, circuit breaker closed')
        }
      }
    }),
    [sourceURI] // Depend on sourceURI, recreate config when it changes
  )

  // Use useCircuitBreaker to wrap fetchPromeData function
  const [fetchPromeData, clearCircuitBreaker] = useCircuitBreaker(
    ((params) => {
      if (!sourceURI) {
        return Promise.resolve({ data: [] })
      }
      const { startTimeSec, endTimeSec, stepSec, query } = params
      return getMetricData({
        sourceURI,
        startTime: startTimeSec,
        endTime: endTimeSec,
        timeStep: stepSec,
        promql: query
      }).then((res) => res.data)
    }) as FetchPromeData,
    circuitBreakerConfig
  )

  // Clear circuit breaker state when component unmounts
  useEffect(() => {
    return () => {
      clearCircuitBreaker()
    }
  }, [])

  // Use the wrapped fetchPromeData
  return (
    <Chart
      fetchPromeData={fetchPromeData}
      // ... other props
    />
  )
}
```

### Example 2: Plain Function Wrapping (Non-React)

In non-React environments, you can directly use the `withCircuitBreaker` higher-order function:

```typescript
import { withCircuitBreaker } from 'dbaas/utils/circuit-breaker'

// Original API call function
async function fetchUserData(userId: string) {
  const response = await fetch(`/api/users/${userId}`)
  if (!response.ok) {
    throw new Error(`Failed to fetch user: ${response.statusText}`)
  }
  return response.json()
}

// Wrap with withCircuitBreaker
const protectedFetchUserData = withCircuitBreaker({
  key: 'user-api',
  errorThreshold: 5, // Open circuit after 5 consecutive errors
  recoveryMs: 60000, // Recovery detection interval (1 minute)
  // recovery defaults to true, so automatic recovery detection is enabled by default
  onThrottleOpen: () => {
    console.log('Circuit breaker opened, all requests will be rejected')
    // Log error when circuit opens
    log.api.error('Circuit breaker opened for user-api')
  },
  onRecoverySuccess: (isDuringPaused) => {
    if (isDuringPaused) {
      console.log('Recovery successful during pause, circuit breaker closed')
    } else {
      console.log('Recovery successful via automatic detection, circuit breaker closed')
    }
  }
})(fetchUserData)

// Use the wrapped function
async function loadUser(userId: string) {
  try {
    const userData = await protectedFetchUserData(userId)
    if (userData === undefined) {
      // Request was rejected because circuit breaker is open
      console.log('Request was rejected, circuit breaker is open')
      return
    }
    console.log('User data:', userData)
  } catch (error) {
    // Only network errors during normal operation are thrown
    console.log('Network error occurred:', error)
  }
}
```

### Example 3: Polling Scenario

In polling scenarios, circuit breakers can prevent frequent failed requests:

```typescript
import { withCircuitBreaker } from 'dbaas/utils/circuit-breaker'

// Polling function
async function pollServerStatus() {
  const response = await fetch('/api/status')
  return response.json()
}

// Create protected polling function
const protectedPoll = withCircuitBreaker({
  key: 'server-status-poll',
  errorThreshold: 10, // Open circuit after 10 consecutive errors
  recoveryMs: 30000, // Recovery detection interval (30 seconds)
  // recovery defaults to true, so automatic recovery detection is enabled by default
  onThrottleOpen: () => {
    console.log('Circuit breaker opened for polling, all requests will be rejected')
    // Log error when circuit opens
    log.api.error('Circuit breaker opened for server-status-poll')
  },
  onRecoverySuccess: (isDuringPaused) => {
    if (isDuringPaused) {
      console.log('Recovery successful during pause for polling, circuit breaker closed')
    } else {
      console.log('Recovery successful via automatic detection for polling, circuit breaker closed')
    }
  }
})(pollServerStatus)

// Set up polling
const pollInterval = setInterval(async () => {
  try {
    const status = await protectedPoll()
    if (status === undefined) {
      // Request was rejected because circuit breaker is open
      // Recovery mechanism will automatically detect when service recovers
      return
    }
    console.log('Server status:', status)
  } catch (error) {
    // Only network errors during normal operation are thrown
    console.error('Polling failed:', error)
  }
}, 5000) // Poll every 5 seconds
```

### Example 4: Pause/Resume Usage

You can temporarily pause the circuit breaker to allow requests to pass through, useful for manual testing or temporary bypass:

```typescript
import { withCircuitBreaker, pauseCircuitBreaker, resumeCircuitBreaker } from 'dbaas/utils/circuit-breaker'

// Create protected function
const protectedFetchData = withCircuitBreaker({
  key: 'my-api',
  errorThreshold: 5,
  recoveryMs: 60000
})(fetchData)

// Use the protected function
async function loadData() {
  try {
    const data = await protectedFetchData(params)
    console.log('Data:', data)
  } catch (error) {
    console.error('Error:', error)
  }
}

// Pause the circuit breaker (e.g., for manual testing)
pauseCircuitBreaker('my-api')

// During pause, requests will pass through normally
// - If request succeeds, circuit breaker is automatically reset
// - If request fails, original state is preserved

// Resume the circuit breaker, restoring the state before pause
resumeCircuitBreaker('my-api')

// Check pause status
import { getCircuitBreakerState } from 'dbaas/utils/circuit-breaker'
const state = getCircuitBreakerState('my-api')
if (state?.isPaused) {
  console.log('Circuit breaker is paused')
}
```

## How It Works

### State Flow

1. **Normal State (Closed)**

   - Function executes normally
   - Resets error count on success
   - Increments error count on failure

2. **Circuit Open State (Open)**

   - Enters when consecutive errors reach `errorThreshold`
   - `onThrottleOpen` callback is executed when the circuit opens
   - **All subsequent requests are immediately rejected** and return `undefined` without throwing errors
   - Only the recovery mechanism (if enabled) can make requests to detect service recovery
   - If `recovery` is `true` (default), automatic recovery detection is scheduled:
     - After `recoveryMs`, the original function is called with the last arguments (from when circuit opened)
     - If detection succeeds, `onRecoverySuccess(false)` is called (indicating automatic recovery, not during pause) and circuit closes
     - If detection fails, another attempt is scheduled after `recoveryMs` (continues until success)

3. **Paused State**

   - Can be entered by calling `pauseCircuitBreaker(key)`
   - Current state (errorCount, isOpen, openedAt) is saved before pausing
   - Recovery timer is cleared when paused
   - **During pause, all requests are allowed to pass through** (circuit breaker logic is bypassed)
   - If a request succeeds during pause, the circuit breaker is automatically reset (clears all state including paused state) and `onRecoverySuccess(true)` is called (indicating recovery during pause)
   - If a request fails during pause, the original state is preserved (circuit breaker state remains unchanged)
   - Recovery detection does not run during pause
   - Can be resumed by calling `resumeCircuitBreaker(key)`, which restores the saved state

4. **Recovery Mechanism**
   - If `recovery` is `true` (default), automatic recovery detection is enabled:
     - The original function is automatically called every `recoveryMs` using the last arguments from when circuit opened
     - If detection succeeds, `onRecoverySuccess(false)` is called (indicating automatic recovery, not during pause) and circuit breaker automatically closes and resets all state
     - If detection fails, another attempt is scheduled after `recoveryMs` (continues polling until success)
   - Recovery timer is automatically cleared when circuit closes, is reset, or is paused
   - Recovery detection does not run during pause
   - **Only the recovery mechanism can make requests during circuit open state** (when not paused)

### Error Handling

- **Request rejected (circuit open)**: Returns `undefined` immediately without throwing errors
- **Normal network errors**: Thrown directly during normal operation, handled by the caller
- **Recovery detection errors**: Silently handled by the recovery mechanism, does not affect the caller

## Best Practices

1. **Choose appropriate keys**: Use meaningful unique identifiers, such as API endpoints, data source URIs, etc.
2. **Set reasonable thresholds**: Adjust `errorThreshold` and `recoveryMs` according to actual business scenarios
3. **Clean up state**: Call `clearCircuitBreaker()` when React components unmount to clean up state
4. **Error logging**: Log errors in `onThrottleOpen` callback when circuit opens, as this is the only place where you can log circuit breaker events
5. **Recovery mechanism**: `recovery` is enabled by default (set to `true`). It automatically polls the original function for recovery detection. Configure `onRecoverySuccess` callback to get notified when recovery succeeds. Set `recovery: false` to disable automatic recovery detection
6. **Monitor state**: Use `getCircuitBreakerState()` to monitor circuit breaker state for debugging and monitoring
7. **Handle undefined returns**: Always check if the result is `undefined` when circuit breaker might be open
8. **Pause/Resume usage**: Use `pauseCircuitBreaker()` and `resumeCircuitBreaker()` for temporary bypass scenarios (e.g., manual testing, debugging, or allowing specific requests). Remember to resume after pausing, or the circuit breaker will remain in paused state

## Notes

- Circuit breaker state is globally shared, instances with the same key share state
- When circuit is open, all requests are immediately rejected and return `undefined` without throwing errors (unless paused)
- Only the recovery mechanism (if enabled) can make requests during circuit open state (unless paused)
- When paused, all requests are allowed to pass through. If a request succeeds during pause, the circuit breaker is reset. If it fails, the original state is preserved
- Normal errors (during normal operation) are thrown directly and need to be handled by the caller
- Errors during pause are also thrown directly and need to be handled by the caller
- When using in React components, it's recommended to use `useMemo` to cache the config object
- `recovery` is enabled by default (set to `true`). When circuit opens, automatic recovery detection is scheduled. The original function is called every `recoveryMs` using the last arguments from when circuit opened. `onRecoverySuccess(false)` is called when automatic detection succeeds (not during pause). `onRecoverySuccess(true)` is called when a request succeeds during pause. The recovery timer is automatically cleared when circuit closes, is reset, or is paused. Set `recovery: false` to disable automatic recovery detection
- Recovery detection does not run when circuit breaker is paused
- Log errors in `onThrottleOpen` callback, as this is the only opportunity to log when circuit opens
- When resuming a paused circuit breaker, the state (errorCount, isOpen, openedAt) is restored to what it was before pausing