# CQL Native Protocol Specification

This document describes the CQL (Cassandra Query Language) Native Protocol as implemented by the gocql driver. The protocol is used for communication between Cassandra clients and servers.

## Table of Contents

1. [Protocol Overview](#protocol-overview)
2. [Connection Establishment](#connection-establishment)
3. [Frame Structure](#frame-structure)
4. [Protocol Versions](#protocol-versions)
5. [Frame Opcodes](#frame-opcodes)
6. [Frame Flags](#frame-flags)
7. [Data Types](#data-types)
8. [Consistency Levels](#consistency-levels)
9. [Error Codes](#error-codes)
10. [Request/Response Flow](#requestresponse-flow)
11. [Authentication Types and Flows](#authentication-types-and-flows)
12. [Detailed Payload Specifications](#detailed-payload-specifications)
13. [Data Type Encoding Reference](#data-type-encoding-reference)
14. [Result Kinds](#result-kinds)
15. [Segment Format (Protocol v5+)](#segment-format-protocol-v5)
16. [CRC Algorithms](#crc-algorithms)
17. [Stream Management](#stream-management)
18. [Compression](#compression)
19. [Tracing](#tracing)
20. [Custom Payload](#custom-payload)
21. [Warnings](#warnings)
22. [Best Practices](#best-practices)
23. [References](#references)

---

## Protocol Overview

The CQL Native Protocol is a binary protocol that operates over TCP. It uses a request-response model where clients send frames to the server and receive responses. The protocol supports:

- Multiple protocol versions (v1-v5)
- Stream multiplexing (multiple concurrent requests on a single connection)
- Compression
- Tracing
- Custom payloads
- Authentication

---

## Connection Establishment

### Initial Handshake

1. **Client connects** to server on port 9042 (default)

2. **Options Request**: Client sends `OPTIONS` frame to discover server capabilities

3. **Supported Response**: Server responds with supported options (compression, CQL versions, etc.)

4. **Startup**: Client sends `STARTUP` frame with:
   - `CQL_VERSION`: The CQL version to use
   - `COMPRESSION`: (optional) Compression algorithm if supported
   - `DRIVER_NAME`: Driver identification
   - `DRIVER_VERSION`: Driver version

5. **Ready or Authenticate**:
   - `READY`: Authentication not required
   - `AUTHENTICATE`: Authentication required, server provides authenticator class name

6. **Authentication** (if required):
   - Client sends `AUTH_RESPONSE` with credentials
   - Server responds with `AUTH_CHALLENGE`, `AUTH_SUCCESS`, or `ERROR`

7. **Ready**: Connection is ready for queries

### Protocol v5+ Considerations

For protocol v5+, the framing format is not applied to message exchanges before the initial handshake is completed. After startup completes, the v5 segment format is used.

---

## Frame Structure

### Classic Frame Format (Protocol v1-v4)

```
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version  | Flags   | Stream ID (16 bits)                       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Opcode  | Length (32 bits, big-endian)                        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
| Payload (Length bytes)                                        |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
```

**Header Size**: 9 bytes

**Fields**:
- **Version** (1 byte): Protocol version with direction bit
  - Bit 7: Direction (0 = Request, 1 = Response)
  - Bits 6-0: Version number (1-5)
- **Flags** (1 byte): Frame flags
- **Stream ID** (2 bytes): Stream identifier (0-32767 for v3+, 0-127 for v1-v2)
- **Opcode** (1 byte): Frame operation type
- **Length** (4 bytes): Payload length in bytes (big-endian)
- **Payload**: Variable length frame body

### Segment Format (Protocol v5+)

Protocol v5 introduces a segment-based framing format that allows multiple frames to be sent in a single segment and supports larger payloads.

#### Uncompressed Segment

```
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload Length (17 bits)      | S | CRC24 (24 bits)           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
| Payload (variable length)                                     |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| CRC32 (32 bits)                                                |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
```

**Header Size**: 6 bytes

**Fields**:
- **Payload Length** (17 bits): Length of the payload in bytes
- **S** (1 bit): Self-contained flag (1 = frame is complete in this segment)
- **CRC24** (24 bits): CRC24 checksum of the first 3 header bytes
- **Payload**: Frame data (can be a complete frame or partial frame)
- **CRC32** (32 bits): CRC32 checksum of the payload

#### Compressed Segment

```
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Compressed Len (17 bits) | Uncompressed Len (17 bits) | S |CRC|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
| Compressed Payload (variable length)                          |
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| CRC32 (32 bits)                                                |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
```

**Header Size**: 8 bytes

**Fields**:
- **Compressed Len** (17 bits): Length of compressed payload
- **Uncompressed Len** (17 bits): Length before compression (0 = no compression)
- **S** (1 bit): Self-contained flag
- **CRC** (24 bits): CRC24 checksum
- **Compressed Payload**: Compressed frame data
- **CRC32** (32 bits): CRC32 checksum of payload

**Note**: If uncompressed length is 0, the compressed payload should be used as-is without decompression.

---

## Protocol Versions

| Version | Description |
|---------|-------------|
| v1 | Initial version, limited features |
| v2 | Adds tracing, pagination |
| v3 | Adds prepared statements, batch queries, compression flags |
| v4 | Adds result metadata ID, custom payload, warnings |
| v5 | **Modern framing format**, segments, larger payloads, batch flags |

**Supported versions in gocql**: v3-v5

**Version negotiation**: Client starts with highest supported version, server may downgrade.

---

## Frame Opcodes

| Opcode | Value | Name | Direction | Description |
|--------|-------|------|-----------|-------------|
| 0x00 | 0 | ERROR | Response | Error response |
| 0x01 | 1 | STARTUP | Request | Connection startup |
| 0x02 | 2 | READY | Response | Ready for queries |
| 0x03 | 3 | AUTHENTICATE | Response | Authentication required |
| 0x05 | 5 | OPTIONS | Request | Query server options |
| 0x06 | 6 | SUPPORTED | Response | Supported options |
| 0x07 | 7 | QUERY | Request | Execute CQL query |
| 0x08 | 8 | RESULT | Response | Query result |
| 0x09 | 9 | PREPARE | Request | Prepare statement |
| 0x0A | 10 | EXECUTE | Request | Execute prepared statement |
| 0x0B | 11 | REGISTER | Request | Register for events |
| 0x0C | 12 | EVENT | Response | Server event notification |
| 0x0D | 13 | BATCH | Request | Execute batch query |
| 0x0E | 14 | AUTH_CHALLENGE | Response | Authentication challenge |
| 0x0F | 15 | AUTH_RESPONSE | Request | Authentication response |
| 0x10 | 16 | AUTH_SUCCESS | Response | Authentication success |

---

## Frame Flags

| Flag | Value | Description |
|------|-------|-------------|
| `flagCompress` | 0x01 | Frame body is compressed (v1-v4 only) |
| `flagTracing` | 0x02 | Request tracing, response contains trace ID |
| `flagCustomPayload` | 0x04 | Frame contains custom payload |
| `flagWarning` | 0x08 | Frame contains warnings (v4+) |
| `flagBetaProtocol` | 0x10 | Beta protocol flag (v5+) |

**Note**: In protocol v5+, compression is handled at the segment level, not the frame level.

---

## Data Types

### Type Codes

| Type | Code | Description |
|------|------|-------------|
| `TypeCustom` | 0x0000 | Custom type |
| `TypeAscii` | 0x0001 | ASCII string |
| `TypeBigInt` | 0x0002 | 64-bit signed integer |
| `TypeBlob` | 0x0003 | Binary data |
| `TypeBoolean` | 0x0004 | Boolean value |
| `TypeCounter` | 0x0005 | Counter column |
| `TypeDecimal` | 0x0006 | Decimal number |
| `TypeDouble` | 0x0007 | 64-bit floating point |
| `TypeFloat` | 0x0008 | 32-bit floating point |
| `TypeInt` | 0x0009 | 32-bit signed integer |
| `TypeText` | 0x000A | UTF-8 string |
| `TypeTimestamp` | 0x000B | Timestamp |
| `TypeUUID` | 0x000C | UUID |
| `TypeVarchar` | 0x000D | UTF-8 string |
| `TypeVarint` | 0x000E | Arbitrary precision integer |
| `TypeTimeUUID` | 0x000F | Time-based UUID |
| `TypeInet` | 0x0010 | IP address |
| `TypeDate` | 0x0011 | Date without time |
| `TypeTime` | 0x0012 | Time of day |
| `TypeSmallInt` | 0x0013 | 16-bit signed integer |
| `TypeTinyInt` | 0x0014 | 8-bit signed integer |
| `TypeDuration` | 0x0015 | Duration |
| `TypeList` | 0x0020 | List collection |
| `TypeMap` | 0x0021 | Map collection |
| `TypeSet` | 0x0022 | Set collection |
| `TypeUDT` | 0x0030 | User-defined type |
| `TypeTuple` | 0x0031 | Tuple type |

### Type Encoding

#### Simple Types

- **Integer types** (TinyInt, SmallInt, Int, BigInt, Counter): Big-endian signed integer
- **Float/Double**: IEEE 754 format
- **Boolean**: 0x00 = false, 0x01 = true
- **String/Varchar/Text**: UTF-8 encoded bytes
- **Blob**: Raw bytes
- **UUID**: 16 bytes in standard UUID format
- **Timestamp**: 64-bit signed integer (milliseconds since Unix epoch)
- **Date**: 32-bit unsigned integer (days since Unix epoch + 2^31)
- **Time**: 64-bit signed integer (nanoseconds since midnight)
- **Inet**: 4 bytes (IPv4) or 16 bytes (IPv6)
- **Decimal**: Variable-length integer (varint) + 32-bit scale
- **Varint**: Variable-length signed integer (big-endian)
- **Duration**: Months (varint) + days (varint) + nanoseconds (varint)

#### Collection Types

**List**: `[value_count][value][value]...`
- `value_count`: 16-bit unsigned integer
- Each value: `[length][data]` where length is 32-bit integer

**Set**: Same as list, but values must be unique

**Map**: `[key_count][key][value]...`
- `key_count`: 16-bit unsigned integer
- Each key: `[length][data]`
- Each value: `[length][data]`

#### User-Defined Types (UDT)

```
[field_count][field_name_1][field_value_1][field_name_2][field_value_2]...
```

- `field_count`: 16-bit unsigned integer
- Field names are strings
- Field values are encoded based on their type

#### Tuple Types

```
[element_count][element_1][element_2]...
```

- `element_count`: 16-bit unsigned integer
- Each element: `[length][data]` or null (length = -1)

#### Vector Types (Protocol v5+)

```
[element_count][element_type][element_1][element_2]...
```

- `element_count`: 32-bit unsigned integer
- `element_type`: Type code of vector elements
- Elements are encoded based on type

---

## Consistency Levels

| Level | Value | Description |
|-------|-------|-------------|
| `ANY` | 0x00 | Write to at least one node (any) |
| `ONE` | 0x01 | Write to one, read from one |
| `TWO` | 0x02 | Write to two, read from two |
| `THREE` | 0x03 | Write to three, read from three |
| `QUORUM` | 0x04 | Write/read to majority of replicas |
| `ALL` | 0x05 | Write/read to all replicas |
| `LOCAL_QUORUM` | 0x06 | Quorum within local DC |
| `EACH_QUORUM` | 0x07 | Quorum for each DC |
| `SERIAL` | 0x08 | Linearizable consistency |
| `LOCAL_SERIAL` | 0x09 | Local DC serial |
| `LOCAL_ONE` | 0x0A | One node in local DC |

---

## Error Codes

| Code | Name | Description |
|------|------|-------------|
| 0x0000 | `ErrCodeServer` | Server error (internal failure) |
| 0x000A | `ErrCodeProtocol` | Protocol violation |
| 0x0100 | `ErrCodeCredentials` | Authentication required |
| 0x1000 | `ErrCodeUnavailable` | Not enough replicas available |
| 0x1001 | `ErrCodeOverloaded` | Node is overloaded |
| 0x1002 | `ErrCodeBootstrapping` | Node is bootstrapping |
| 0x1003 | `ErrCodeTruncate` | Truncation error |
| 0x1100 | `ErrCodeWriteTimeout` | Write operation timed out |
| 0x1200 | `ErrCodeReadTimeout` | Read operation timed out |
| 0x1300 | `ErrCodeReadFailure` | Read operation failed |
| 0x1400 | `ErrCodeFunctionFailure` | User-defined function failed |
| 0x1500 | `ErrCodeWriteFailure` | Write operation failed |
| 0x1600 | `ErrCodeCDCWriteFailure` | CDC write failure |
| 0x1700 | `ErrCodeCASWriteUnknown` | CAS operation partially completed |
| 0x2000 | `ErrCodeSyntax` | Syntax error |
| 0x2100 | `ErrCodeUnauthorized` | Unauthorized |
| 0x2200 | `ErrCodeInvalid` | Invalid query |
| 0x2300 | `ErrCodeConfig` | Configuration error |
| 0x2400 | `ErrCodeAlreadyExists` | Keyspace/table already exists |
| 0x2500 | `ErrCodeUnprepared` | Prepared statement not found |

---

## Segment Format (Protocol v5)

### Overview

Protocol v5 replaces the classic frame format with a segment-based approach that:

1. Allows multiple frames in a single segment
2. Supports larger payloads (up to 0x1FFFF bytes per segment)
3. Includes CRC checksums for data integrity
4. Handles compression at segment level

### Reading Segments

**For uncompressed segments:**
1. Read 6 bytes header
2. Verify CRC24 of first 3 bytes
3. Extract payload length (17 bits) and self-contained flag
4. Read payload
5. Read and verify CRC32

**For compressed segments:**
1. Read 8 bytes header
2. Verify CRC24 of first 5 bytes
3. Extract compressed length (17 bits), uncompressed length (17 bits), and flag
4. Read compressed payload
5. If uncompressed length > 0, decompress payload
6. Read and verify CRC32

### Writing Segments

**For uncompressed:**
1. Write payload length (17 bits) + self-contained flag (1 bit)
2. Calculate and write CRC24 of first 3 bytes
3. Write payload
4. Calculate and write CRC32 of payload

**For compressed:**
1. Compress payload
2. If compressed is larger, use uncompressed (set uncompressed_len=0)
3. Write compressed length, uncompressed length, and flag
4. Calculate and write CRC24
5. Write payload
6. Calculate and write CRC32

### Self-Contained Flag

- **1**: The segment contains one or more complete frames
- **0**: The segment contains a partial frame, more segments follow

### Frame Processing in Segments

When a segment is self-contained:
1. Parse the frame header from the segment payload
2. Read the frame body based on length
3. Process the frame
4. If more data remains in the segment, repeat

When a segment is not self-contained:
1. Accumulate segments until a complete frame is assembled
2. Process the complete frame

---

## Request/Response Flow

### Query Execution

**Request: QUERY**
```
[Header: QUERY]
[Body:
  - Query string (long string)
  - Consistency (short)
  - Flags (byte/uint)
  - [Optional: Values]
  - [Optional: Page size]
  - [Optional: Paging state]
  - [Optional: Serial consistency]
  - [Optional: Timestamp]
  - [Optional: Keyspace (v5+)]
  - [Optional: Now in seconds (v5+)]
]
```

**Response: RESULT**
```
[Header: RESULT]
[Body:
  - Kind (int)
  - [Kind-specific data]
]
```

### Prepared Statement Flow

**1. Prepare**
```
Client -> Server: PREPARE "SELECT * FROM table WHERE id = ?"
Server -> Client: RESULT (prepared statement ID + metadata)
```

**2. Execute**
```
Client -> Server: EXECUTE [ID] [values]
Server -> Client: RESULT (rows or void)
```

### Batch Query Flow

**Request: BATCH**
```
[Header: BATCH]
[Body:
  - Type (byte)
  - Statement count (short)
  - For each statement:
    - [Prepared ID or Query string]
    - [Values]
  - Consistency (short)
  - Flags (byte/uint)
  - [Optional: Serial consistency]
  - [Optional: Timestamp]
  - [Optional: Keyspace (v5+)]
  - [Optional: Now in seconds (v5+)]
]
```

**Response: RESULT** (void or rows)

### Authentication Flow

```
Client -> Server: STARTUP {CQL_VERSION: "3.0.0"}
Server -> Client: AUTHENTICATE "org.apache.cassandra.auth.PasswordAuthenticator"
Client -> Server: AUTH_RESPONSE [credentials]
Server -> Client: AUTH_SUCCESS (or AUTH_CHALLENGE for multi-round)
Client -> Server: READY
```

**Note:** See [Authentication Types and Flows](#authentication-types-and-flows) for detailed payload formats, supported authenticator types, and multi-round authentication examples.

### Event Registration

```
Client -> Server: REGISTER ["TOPOLOGY_CHANGE", "STATUS_CHANGE", "SCHEMA_CHANGE"]
Server -> Client: READY
Server -> Client: EVENT (asynchronously when events occur)
```

---

## Authentication Types and Flows

The CQL native protocol supports various authentication mechanisms through a SASL-based challenge-response system. The client implements the `Authenticator` interface to handle authentication exchanges.

### Authenticator Interface

All authenticators must implement this interface:

```go
type Authenticator interface {
    Challenge(req []byte) (resp []byte, auth Authenticator, err error)
    Success(data []byte) error
}
```

**Methods:**
- `Challenge(req []byte)`: Called when server sends `AUTH_CHALLENGE`. Returns response bytes, potentially a new authenticator for multi-round, and any error.
- `Success(data []byte)`: Called when server sends `AUTH_SUCCESS` with final data.

### Built-in Authenticators

#### 1. PasswordAuthenticator

**Purpose:** Username/password authentication using null-terminated byte format.

**Implementation:**
```go
type PasswordAuthenticator struct {
    Username              string
    Password              string
    AllowedAuthenticators []string  // Optional restrict list
}
```

**Challenge Payload Format:**
```
[0x00][username_bytes][0x00][password_bytes]
```

**Example with username="cassandra", password="cassandra":**
```
Payload: 00 63 61 73 73 61 6E 64 72 61 00 63 61 73 73 61 6E 64 72 61
         |  | "cassandra"  |  | "cassandra"
         |  +--------------+  +-------------+
         null separator      null separator
```

**Success Payload:** Empty (length = 0)

**Configuration:**
```go
// Allow any authenticator (default)
cluster.Authenticator = gocql.PasswordAuthenticator{
    Username: "user",
    Password: "pass",
}

// Restrict to specific authenticators
cluster.Authenticator = gocql.PasswordAuthenticator{
    Username: "user",
    Password: "pass",
    AllowedAuthenticators: []string{
        "org.apache.cassandra.auth.PasswordAuthenticator",
    },
}
```

### Supported Authenticator Types

The following authenticator class names are recognized by various Cassandra distributions:

| Authenticator Class | Provider | Description |
|---------------------|----------|-------------|
| `org.apache.cassandra.auth.PasswordAuthenticator` | Apache Cassandra | Standard username/password |
| `org.apache.cassandra.auth.MutualTlsAuthenticator` | Apache Cassandra | Certificate-based mutual TLS |
| `org.apache.cassandra.auth.MutualTlsWithPasswordFallbackAuthenticator` | Apache Cassandra | mTLS with password fallback |
| `com.datastax.bdp.cassandra.auth.DseAuthenticator` | DataStax Enterprise | DSE authentication |
| `com.scylladb.auth.SaslauthdAuthenticator` | ScyllaDB | SASL auth via saslauthd |
| `com.scylladb.auth.TransitionalAuthenticator` | ScyllaDB | No-op authenticator for testing |
| `com.instaclustr.cassandra.auth.SharedSecretAuthenticator` | Instaclustr | Shared secret authentication |
| `com.instaclustr.cassandra.auth.InstaclustrPasswordAuthenticator` | Instaclustr | Instaclustr password auth |
| `io.aiven.cassandra.auth.AivenAuthenticator` | Aiven | Aiven-specific authentication |
| `com.amazon.helenus.auth.HelenusAuthenticator` | Helenus | Helenus authentication |
| `com.ericsson.bss.cassandra.ecaudit.auth.AuditAuthenticator` | Ericsson | Audit-focused authentication |

### Authentication Interaction Flows

#### Single-Round Authentication (PasswordAuthenticator)

```
Client                                    Server
  |                                         |
  |-------- STARTUP {options} ------------>|
  |                                         |
  |<------ AUTHENTICATE "class-name" -------|
  |                                         |
  |-------- AUTH_RESPONSE [credentials] -->|
  |                                         |
  |<------ AUTH_SUCCESS [empty] ------------|
  |                                         |
  |-------- READY ------------------------>|
  |                                         |
```

**Payload Details:**
1. **STARTUP**: `[string_map]` with CQL_VERSION, DRIVER_NAME, etc.
2. **AUTHENTICATE**: `[string]` authenticator class name
3. **AUTH_RESPONSE**: `[int]` length + `[bytes]` auth data
4. **AUTH_SUCCESS**: `[int]` length + `[bytes]` success data (empty for PasswordAuthenticator)
5. **READY**: No payload

#### Multi-Round Authentication (SASL)

```
Client                                    Server
  |                                         |
  |-------- STARTUP {options} ------------>|
  |                                         |
  |<------ AUTHENTICATE "class-name" -------|
  |                                         |
  |-------- AUTH_RESPONSE [initial] ------>|
  |                                         |
  |<------ AUTH_CHALLENGE [challenge] -----|
  |                                         |
  |-------- AUTH_RESPONSE [response] ----->|
  |                                         |
  |<------ AUTH_CHALLENGE [challenge] -----|
  |                                         |
  |-------- AUTH_RESPONSE [final] -------->|
  |                                         |
  |<------ AUTH_SUCCESS [data] ------------|
  |                                         |
  |-------- READY ------------------------>|
  |                                         |
```

**Flow Details:**
1. Client sends initial `AUTH_RESPONSE` based on server's authenticator class
2. Server responds with `AUTH_CHALLENGE` containing challenge data
3. Client's `Challenge()` method processes challenge and returns response
4. This repeats until server sends `AUTH_SUCCESS`
5. Client's `Success()` method is called with final data

**Custom Authenticator Example:**
```go
type CustomAuthenticator struct {
    Token string
    Nonce string
}

func (c *CustomAuthenticator) Challenge(req []byte) ([]byte, Authenticator, error) {
    // Parse server challenge
    if bytes.Equal(req, []byte("step1")) {
        // First round: send token
        return []byte(c.Token), c, nil
    }
    if bytes.Equal(req, []byte("step2")) {
        // Second round: send nonce response
        return []byte(c.Nonce), nil, nil
    }
    return nil, nil, fmt.Errorf("unexpected challenge")
}

func (c *CustomAuthenticator) Success(data []byte) error {
    // Verify server success data
    if !bytes.Equal(data, []byte("OK")) {
        return fmt.Errorf("authentication failed")
    }
    return nil
}
```

### Error Handling

**Authentication Errors:**
- Server may respond with `ERROR` frame instead of `AUTH_CHALLENGE`/`AUTH_SUCCESS`
- Common error codes:
  - `0x0100`: Authentication error
  - `0x1300`: Read failure during auth
  - `0x1500`: Write failure during auth

**Client-Side Errors:**
- `Challenge()` returning error aborts authentication
- `Success()` returning error aborts connection
- `AllowedAuthenticators` mismatch returns error before sending response

---

## Detailed Payload Specifications

This section provides the exact byte-level payload format for each request and response type. All multi-byte values are in big-endian format unless otherwise specified.

### Request Payload Formats

#### STARTUP (Opcode 0x01)
**Client → Server**

**Payload Format:**
```
[string_map]
```

**Details:**
- `string_map`: A map of string key-value pairs
  - `[short]`: Number of key-value pairs (N)
  - For each pair:
    - `[short]`: Length of key
    - `[string]`: Key bytes
    - `[short]`: Length of value
    - `[string]`: Value bytes

**Example with CQL_VERSION and DRIVER_NAME:**
```
00 02                      // 2 key-value pairs
00 0B 43 51 4C 5F 56 45 52 53 49 4F 4E  // "CQL_VERSION" (11 bytes)
00 05 33 2E 30 2E 30       // "3.0.0" (5 bytes)
00 0B 44 52 49 56 45 52 5F 4E 41 4D 45  // "DRIVER_NAME" (11 bytes)
00 06 67 6F 63 6C 2F 76    // "gocql/v" (6 bytes)
```

#### OPTIONS (Opcode 0x05)
**Client → Server**

**Payload Format:**
```
(empty)
```

**Details:**
- No payload body

#### AUTH_RESPONSE (Opcode 0x0F)
**Client → Server**

**Payload Format:**
```
[bytes]
```

**Details:**
- `[int]`: Length of authentication data (N)
- `[N bytes]`: Authentication data

**For PasswordAuthenticator specifically:**
The payload is a byte array with the following structure:
```
[0x00][username][0x00][password]
```

**Important details:**
- The payload starts with a null byte (0x00)
- Followed by the username as raw bytes (no length prefix)
- Followed by another null byte (0x00)
- Followed by the password as raw bytes (no length prefix)
- The total length is: 2 + len(username) + len(password)

**Example with username="user" and password="pass":**
```
Frame Body:
00 00 00 0C                // [int] Length: 12 bytes total
00                         // First null separator
75 73 65 72                // "user" (4 bytes)
00                         // Second null separator
70 61 73 73                // "pass" (4 bytes)

Total: 4 (int) + 1 + 4 + 1 + 4 = 14 bytes in frame body
```

**Example with username="cassandra" and password="cassandra":**
```
Frame Body:
00 00 00 14                // [int] Length: 20 bytes total
00                         // First null separator
63 61 73 73 61 6E 64 72 61 // "cassandra" (9 bytes)
00                         // Second null separator
63 61 73 73 61 6E 64 72 61 // "cassandra" (9 bytes)
```

#### QUERY (Opcode 0x07)
**Client → Server**

**Payload Format:**
```
[long_string][consistency][flags][optional fields...]
```

**Details:**
- `[int]`: Length of query string
- `[string]`: Query string
- `[short]`: Consistency level (0x00-0x0A)
- `[uint]` (v5+) or `[byte]` (v3-v4): Flags

**Query Flags (bitmask):**
- `0x01`: Values present
- `0x02`: Skip metadata
- `0x04`: Page size present
- `0x08`: Paging state present
- `0x10`: Serial consistency present
- `0x20`: Default timestamp present
- `0x40`: Named values present
- `0x80`: Keyspace present (v5+)
- `0x100`: Now in seconds present (v5+)

**Optional fields (in order when flags set):**
- `[short]`: Number of values (if 0x01 flag)
- For each value:
  - `[short]`: Length of value (or `[string]` if named)
  - `[bytes]`: Value data
- `[int]`: Page size (if 0x04 flag)
- `[bytes]`: Paging state (if 0x08 flag)
- `[short]`: Serial consistency (if 0x10 flag)
- `[long]`: Timestamp in microseconds (if 0x20 flag)
- `[string]`: Keyspace name (if 0x80 flag, v5+)
- `[int]`: Now in seconds (if 0x100 flag, v5+)

#### PREPARE (Opcode 0x09)
**Client → Server**

**Payload Format:**
```
[long_string][optional flags][optional keyspace]
```

**Details:**
- `[int]`: Length of statement
- `[string]`: Statement to prepare
- `[uint]` (v5+): Flags (only 0x01 for keyspace present)
- `[string]`: Keyspace name (if flags & 0x01)

#### EXECUTE (Opcode 0x0A)
**Client → Server**

**Payload Format:**
```
[short_bytes][optional result_metadata_id][query_params]
```

**Details:**
- `[short]`: Length of prepared ID
- `[bytes]`: Prepared statement ID
- `[short]` (v5+): Length of result metadata ID
- `[bytes]` (v5+): Result metadata ID
- Query parameters (same format as QUERY)

#### BATCH (Opcode 0x0D)
**Client → Server**

**Payload Format:**
```
[type][statement_count][statements...][consistency][flags][optional fields...]
```

**Details:**
- `[byte]`: Batch type (0=logged, 1=unlogged, 2=counter)
- `[short]`: Number of statements (N)

**For each statement:**
- `[byte]`: Kind (0=query string, 1=prepared ID)
- If kind=0: `[long_string]`: Query string
- If kind=1: `[short_bytes]`: Prepared ID
- `[short]`: Number of values
- For each value:
  - `[short]`: Length (or `[string]` if named)
  - `[bytes]`: Value data

**After statements:**
- `[short]`: Consistency
- `[uint]` (v5+) or `[byte]` (v3-v4): Flags

**Batch Flags:**
- `0x10`: Serial consistency present
- `0x20`: Default timestamp present
- `0x80`: Keyspace present (v5+)
- `0x100`: Now in seconds present (v5+)

**Optional fields:**
- `[short]`: Serial consistency (if 0x10)
- `[long]`: Timestamp (if 0x20)
- `[string]`: Keyspace (if 0x80, v5+)
- `[int]`: Now in seconds (if 0x100, v5+)

#### REGISTER (Opcode 0x0B)
**Client → Server**

**Payload Format:**
```
[string_list]
```

**Details:**
- `[short]`: Number of event types (N)
- For each event type:
  - `[short]`: Length
  - `[string]`: Event type name ("TOPOLOGY_CHANGE", "STATUS_CHANGE", "SCHEMA_CHANGE")

### Response Payload Formats

#### READY (Opcode 0x02)
**Server → Client**

**Payload Format:**
```
(empty)
```

**Details:**
- No payload body

#### SUPPORTED (Opcode 0x06)
**Server → Client**

**Payload Format:**
```
[string_multimap]
```

**Details:**
- `[short]`: Number of keys (N)
- For each key:
  - `[short]`: Length of key
  - `[string]`: Key
  - `[short]`: Number of values (M)
  - For each value:
    - `[short]`: Length of value
    - `[string]`: Value

#### AUTHENTICATE (Opcode 0x03)
**Server → Client**

**Payload Format:**
```
[string]
```

**Details:**
- `[short]`: Length of authenticator class name
- `[string]`: Authenticator class name (e.g., "org.apache.cassandra.auth.PasswordAuthenticator")

#### AUTH_CHALLENGE (Opcode 0x0E)
**Server → Client**

**Payload Format:**
```
[bytes]
```

**Details:**
- `[int]`: Length of challenge data
- `[bytes]`: Challenge data

**Usage in multi-round authentication:**
- The server sends challenge data specific to the authenticator
- Client's `Challenge()` method processes this and returns a response
- The response is sent back via `AUTH_RESPONSE`
- This cycle repeats until `AUTH_SUCCESS` is received

**Example scenarios:**
- **SASL mechanisms**: Challenge contains server nonce, SCRAM data, etc.
- **Custom authenticators**: Challenge format is implementation-specific
- **mTLS with fallback**: May challenge for certificate or password

**Client response flow:**
```
AUTH_CHALLENGE [challenge_data] → Challenge(challenge_data) → AUTH_RESPONSE [response_data]
```

#### AUTH_SUCCESS (Opcode 0x10)
**Server → Client**

**Payload Format:**
```
[bytes]
```

**Details:**
- `[int]`: Length of success data
- `[bytes]`: Success data (may be empty)

**For PasswordAuthenticator specifically:**
The payload is typically empty (length = 0). The server sends:
```
00 00 00 00                // [int] Length: 0 bytes
```

This indicates successful authentication with no additional data.

#### ERROR (Opcode 0x00)
**Server → Client**

**Payload Format:**
```
[code][message][additional data based on code]
```

**Details:**
- `[int]`: Error code
- `[short]`: Length of message
- `[string]`: Error message

**Additional data per error code:**

| Code | Name | Additional Data |
|------|------|-----------------|
| 0x1000 | UNAVAILABLE | `[short]` consistency, `[int]` required, `[int]` alive |
| 0x1100 | WRITE_TIMEOUT | `[short]` consistency, `[int]` received, `[int]` blockfor, `[short]` write_type |
| 0x1200 | READ_TIMEOUT | `[short]` consistency, `[int]` received, `[int]` blockfor, `[byte]` data_present |
| 0x1300 | READ_FAILURE | `[short]` consistency, `[int]` received, `[int]` blockfor, `[int]` num_failures, `[byte]` data_present, `[map<inet,short>]` error_map (v5+) |
| 0x1500 | WRITE_FAILURE | `[short]` consistency, `[int]` received, `[int]` blockfor, `[int]` num_failures, `[string]` write_type, `[map<inet,short>]` error_map (v5+) |
| 0x2400 | ALREADY_EXISTS | `[short]` keyspace_length, `[string]` keyspace, `[short]` table_length, `[string]` table |
| 0x2500 | UNPREPARED | `[short]` id_length, `[bytes]` prepared_id |

#### RESULT (Opcode 0x08)
**Server → Client**

**Payload Format:**
```
[kind][kind_specific_data]
```

**Details:**
- `[int]`: Result kind

**Result Kinds:**

**1. Void (0x0001)**
- No additional data

**2. Rows (0x0002)**
```
[metadata][row_count][rows]
```

**Metadata:**
- `[int]`: Flags
- `[int]`: Column count
- If global spec (flag 0x01): `[string]` keyspace, `[string]` table
- For each column:
  - If not global: `[string]` keyspace, `[string]` table
  - `[string]` column name
  - `[short]` type code, followed by type-specific data

**Note:** The `pkey_count` and `pkey_indices` fields are NOT included in Rows metadata. They only appear in Prepared metadata (see Prepared section below).

**Row data:**
- `[int]`: Row count
- For each row:
  - For each column:
    - `[int]`: Length (or -1 for null)
    - `[bytes]`: Value

**3. SetKeyspace (0x0003)**
- `[string]`: Keyspace name

**4. Prepared (0x0004)**
```
[short_bytes][optional result_metadata_id][prepared_metadata][result_metadata]
```

- `[short]`: Length of prepared ID
- `[bytes]`: Prepared ID
- If v5+: `[short]` length of result metadata ID, `[bytes]` result metadata ID
- `[prepared_metadata]`: Metadata for bind parameters (see format below)
- `[result_metadata]`: Metadata for result rows (same format as Rows metadata)

**Prepared Metadata (for binding parameters):**
```
- [int]: Flags
- [int]: Column count (number of bind markers)
- If v4+: [int] pk_count, then [short] pk_indices...
- If global spec (flag 0x01): [string] keyspace, [string] table
- For each column:
  - If not global: [string] keyspace, [string] table
  - [string] column name
  - [short] type code, followed by type-specific data
```

**Note:** The `pk_count` field (v4+) indicates which bind markers correspond to partition key columns. This is used for token-aware routing. If `pk_count` is 0, no bind markers correspond to partition keys.

**5. SchemaChange (0x0005)**
```
[string][string][string or more...]
```

- `[string]`: Change type ("CREATED", "DROPPED", "UPDATED")
- `[string]`: Target type ("KEYSPACE", "TABLE", "TYPE", "FUNCTION", "AGGREGATE")
- If target="KEYSPACE": `[string]` keyspace
- If target="TABLE" or "TYPE": `[string]` keyspace, `[string]` table/type
- If target="FUNCTION" or "AGGREGATE": `[string]` keyspace, `[string]` name, `[string_list]` arg_types

#### EVENT (Opcode 0x0C)
**Server → Client (asynchronous)**

**Payload Format:**
```
[string][event_specific_data]
```

**Details:**
- `[string]`: Event type ("TOPOLOGY_CHANGE", "STATUS_CHANGE", "SCHEMA_CHANGE")

**TOPOLOGY_CHANGE / STATUS_CHANGE:**
- `[string]`: Change ("NEW_NODE" or "REMOVED_NODE" or "UP" or "DOWN")
- `[inet]`: IP address and port

**SCHEMA_CHANGE:**
- Same format as RESULT SchemaChange

---

## Data Type Encoding Reference

This section describes how primitive data types are encoded in the protocol.

### Primitive Type Encodings

| Type | Code | Encoding Format |
|------|------|-----------------|
| `[byte]` | - | Single byte |
| `[short]` | - | 2 bytes, big-endian unsigned integer |
| `[int]` | - | 4 bytes, big-endian signed integer |
| `[long]` | - | 8 bytes, big-endian signed integer |
| `[uint]` | - | 4 bytes, big-endian unsigned integer |
| `[string]` | - | `[short]` length + UTF-8 bytes |
| `[long_string]` | - | `[int]` length + UTF-8 bytes |
| `[bytes]` | - | `[int]` length + bytes (length = -1 for null) |
| `[short_bytes]` | - | `[short]` length + bytes |
| `[string_list]` | - | `[short]` count + `[string]`... |
| `[string_map]` | - | `[short]` count + (`[string]` key + `[string]` value)... |
| `[string_multimap]` | - | `[short]` count + (`[string]` key + `[string_list]` values)... |
| `[bytes_map]` | - | `[short]` count + (`[string]` key + `[bytes]` value)... |
| `[inet]` | - | `[byte]` address_length (4 or 16) + `[bytes]` address + `[short]` port |
| `[consistency]` | - | `[short]` (same as consistency level values) |

### CQL Type Encodings

These are the wire formats for CQL data types when used in query parameters or result rows:

| CQL Type | Protocol Code | Wire Format |
|----------|---------------|-------------|
| `ascii` | 0x0001 | `[bytes]` (UTF-8 string) |
| `bigint` | 0x0002 | `[long]` (8-byte big-endian) |
| `blob` | 0x0003 | `[bytes]` (raw binary) |
| `boolean` | 0x0004 | `[byte]` (0x00=false, 0x01=true) |
| `counter` | 0x0005 | `[long]` (same as bigint) |
| `decimal` | 0x0006 | `[int]` scale + `[varint]` unscaled value |
| `double` | 0x0007 | `[long]` (IEEE 754 64-bit) |
| `float` | 0x0008 | `[int]` (IEEE 754 32-bit) |
| `int` | 0x0009 | `[int]` (4-byte big-endian) |
| `text` | 0x000A | `[bytes]` (UTF-8 string) |
| `timestamp` | 0x000B | `[long]` (milliseconds since Unix epoch) |
| `uuid` | 0x000C | `[bytes]` (16 bytes) |
| `varchar` | 0x000D | `[bytes]` (UTF-8 string) |
| `varint` | 0x000E | `[bytes]` (variable-length signed integer) |
| `timeuuid` | 0x000F | `[bytes]` (16 bytes, Type 1 UUID) |
| `inet` | 0x0010 | `[bytes]` (4 bytes for IPv4, 16 for IPv6) |
| `date` | 0x0011 | `[int]` (days since Unix epoch + 2^31) |
| `time` | 0x0012 | `[long]` (nanoseconds since midnight) |
| `smallint` | 0x0013 | `[short]` (2-byte big-endian) |
| `tinyint` | 0x0014 | `[byte]` (1-byte signed) |
| `duration` | 0x0015 | `[varint]` months + `[varint]` days + `[varint]` nanoseconds |
| `list` | 0x0020 | `[short]` count + for each: `[int]` length + `[bytes]` value |
| `map` | 0x0021 | `[short]` count + for each: `[int]` key_length + `[bytes]` key + `[int]` value_length + `[bytes]` value |
| `set` | 0x0022 | `[short]` count + for each: `[int]` length + `[bytes]` value |
| `udt` | 0x0030 | `[short]` field_count + for each: `[string]` name + `[bytes]` value |
| `tuple` | 0x0031 | `[short]` element_count + for each: `[int]` length + `[bytes]` value |

### Special Values

- **Null**: Represented as `[int]` with value `-1` (0xFFFFFFFF)
- **Unset** (v4+): Represented as `[int]` with value `-2` (0xFFFFFFFE)
- **Empty string**: `[int]` length = 0, no bytes follow

---

## Result Kinds

| Kind | Value | Description |
|------|-------|-------------|
| `Void` | 0x0001 | No result (INSERT/UPDATE/DELETE) |
| `Rows` | 0x0002 | Result set with rows |
| `SetKeyspace` | 0x0003 | Keyspace was set |
| `Prepared` | 0x0004 | Prepared statement result |
| `SchemaChange` | 0x0005 | Schema change notification |

### Rows Result Format

```
[Metadata]
[Rows count (int)]
[Rows data]
```

**Metadata:**
```
- Flags (int)
- Column count (int)
- [If global spec: Keyspace + Table]
- For each column:
  - [Keyspace (if not global)]
  - [Table (if not global)]
  - Name
  - Type
```

**Note:** Rows metadata does NOT include `pkey_count` or `pkey_indices`. These fields only appear in Prepared metadata (for bind parameters).

**Flags in Metadata:**
- `0x0001`: Global table specification
- `0x0002`: Has more pages
- `0x0004`: No metadata
- `0x0008`: Metadata changed (v4+)

---

## CRC Algorithms

### CRC24

Used for segment header integrity.

**Polynomial**: 0x1974F0B (Koopman polynomial)
**Initial value**: 0x875060

```go
func Crc24(buf []byte) uint32 {
    crc := 0x875060
    for _, b := range buf {
        crc ^= int(b) << 16
        for i := 0; i < 8; i++ {
            crc <<= 1
            if crc & 0x1000000 != 0 {
                crc ^= 0x1974F0B
            }
        }
    }
    return uint32(crc)
}
```

### CRC32

Used for segment payload integrity.

**Polynomial**: IEEE 802.3 (standard CRC32)
**Initial value**: 0xFA2D55CA (with initial bytes prepended)

```go
func Crc32(b []byte) uint32 {
    crc := crc32.NewIEEE()
    crc.Write([]byte{0xfa, 0x2d, 0x55, 0xca}) // Initial bytes
    crc.Write(b)
    return crc.Sum32()
}
```

---

## Stream Management

### Stream ID Allocation

- **Protocol v1-v2**: 0-127 (7 bits)
- **Protocol v3+**: 0-32767 (15 bits)

### Stream Lifecycle

1. **Acquire**: Client gets available stream ID
2. **Send**: Frame sent with stream ID
3. **Wait**: Wait for response on same stream ID
4. **Release**: Stream ID returned to pool after response or timeout

### Concurrency

Multiple streams can be active simultaneously on a single connection. The gocql driver uses a stream pool to manage available IDs.

---

## Compression

### Supported Algorithms

- **LZ4**: Fast compression
- **Snappy**: Google's compression algorithm

### Compression Behavior

**Protocol v1-v4:**
- Compression applied to frame body (after header)
- `flagCompress` indicates compressed body
- Header remains uncompressed

**Protocol v5+:**
- Compression applied at segment level
- Compressed segments include uncompressed length
- If compression doesn't reduce size, payload is sent uncompressed with `uncompressed_len = 0`

---

## Tracing

### Request Tracing

Client sets `flagTracing` in request frame.

### Trace Response

Response includes:
1. Trace ID in frame header (first 16 bytes of payload)
2. Can query `system_traces.sessions` and `system_traces.events` for details

---

## Custom Payload

### Request Payload

Set `flagCustomPayload` and include key-value map before frame body.

### Response Payload

Server may include custom payload in responses.

---

## Warnings

### Query Warnings

Server can send warnings (e.g., for unlogged batches, large IN clauses).

**Protocol v4+**: Warnings included in frame header via `flagWarning`.

---

## Best Practices

### Connection Management

1. Use connection pooling
2. Reuse prepared statements
3. Implement proper error handling
4. Handle topology changes via events

### Frame Handling

1. Always verify protocol version
2. Check CRCs in v5 segments
3. Handle stream ID correctly
4. Implement proper timeouts

### Error Handling

1. Distinguish retryable vs non-retryable errors
2. Handle unavailable errors with retry policies
3. Respect server-provided retry hints

---

## References

- [Apache Cassandra Native Protocol Specification](https://github.com/apache/cassandra/blob/trunk/doc/native_protocol_v5.spec)
- [gocql Driver Source Code](https://github.com/apache/cassandra-gocql-driver)