cafe 1.0.138

Readme

• BouncyCastle TLS 1.3 Handshake Flow
  • State Machine
  • Full Handshake Sequence (No HelloRetryRequest)
  • Key Exchange (ECDHE)
  • Secret Derivation
  • TLS 1.3 vs TLS 1.2 Differences in BouncyCastle
  • Pitfalls Encountered
    • 1. KeyExchangeAlgorithm.NULL (0)
    • 2. Certificate.Encode() certificateRequestContext
    • 3. RSA-PSS SignatureAndHashAlgorithm
• HTTP/2 Frame Handling
  • Overview
  • Key HTTP/2 Concepts
    • Connection Preface
    • Frame Format
    • Frame Types
    • Common Flags
    • Stream Lifecycle
    • HPACK Header Compression
    • Pseudo-Headers
  • Http2FrameHandler Usage
    • Class Overview
    • Key Methods
    • Virtual Handler Methods
    • Helper Methods for Constructing Frames
    • Usage Pattern
    • Integration with HttpParser.HandleHttp2
    • Limitations
• HTTP Server Core Components
  • TcpHost
    • Class Overview
    • Constructors
    • Key Methods
    • Usage Pattern
  • HttpHandler
    • Class Overview
    • Key Methods
    • Request Processing Flow
    • Keep-Alive Logic
  • HttpParser
    • Class Overview
    • Key Methods
    • HTTP/1.1 Parsing Flow
    • HTTP/2 Parsing Flow
  • HttpSessionContext
    • Class Overview
    • Key Properties
    • Key Methods
    • Usage Pattern
    • Integration with Other Components

BouncyCastle TLS 1.3 Handshake Flow

Based on BouncyCastle C# source (bc-csharp-master/crypto/src/tls/).

State Machine

Key states defined in TlsProtocol.cs:

Full Handshake Sequence (No HelloRetryRequest)

Client                          Server
  |                               |
  |  1. ClientHello               |
  |------------------------------>|  CS_START -> CS_CLIENT_HELLO
  |                               |  ReceiveClientHelloMessage()
  |                               |  Generate13ServerHello()
  |                               |    - Negotiate version/cipher suite
  |                               |    - ECDHE key exchange (embedded)
  |                               |    - Derive earlySecret, handshakeSecret, masterSecret
  |                               |
  |  2. ServerHello               |
  |<------------------------------|  CS_SERVER_HELLO
  |                               |  SendServerHelloMessage()
  |                               |
  |  [ChangeCipherSpec]           |
  |<------------------------------|  Middlebox compatibility (RFC 8446 D.4)
  |                               |
  |  === Encrypted from here ===  |
  |                               |  Send13ServerHelloCoda()
  |                               |    Establish13PhaseHandshake()
  |                               |      -> derive "c hs traffic", "s hs traffic"
  |                               |    EnablePendingCipherWrite()
  |                               |
  |  3. EncryptedExtensions       |
  |<------------------------------|  CS_SERVER_ENCRYPTED_EXTENSIONS
  |                               |  Send13EncryptedExtensionsMessage()
  |                               |
  |  4. [CertificateRequest]      |
  |<------------------------------|  (optional, client auth)
  |                               |
  |  5. Certificate               |
  |<------------------------------|  CS_SERVER_CERTIFICATE
  |                               |  Establish13ServerCredentials() -> GetCredentials()
  |                               |  Send13CertificateMessage()
  |                               |    Certificate.Encode() requires:
  |                               |      TLS 1.3: certificateRequestContext = EmptyBytes (not null)
  |                               |      TLS 1.2: certificateRequestContext = null
  |                               |
  |  6. CertificateVerify         |
  |<------------------------------|  Generate13CertificateVerify()
  |                               |    TLS 1.3 only allows RSA-PSS (not PKCS#1 v1.5)
  |                               |    SignatureAndHashAlgorithm must use Intrinsic hash
  |                               |
  |  7. Finished                  |
  |<------------------------------|  CS_SERVER_FINISHED
  |                               |  Send13FinishedMessage()
  |                               |  Establish13PhaseApplication()
  |                               |    -> derive "c ap traffic", "s ap traffic"
  |                               |  EnablePendingCipherWrite()
  |                               |
  |  8. [Certificate]             |
  |------------------------------>|  (only if CertificateRequest was sent)
  |  9. [CertificateVerify]       |
  |------------------------------>|
  |                               |
  |  10. Finished                 |
  |------------------------------>|  CS_CLIENT_FINISHED
  |                               |  Receive13ClientFinished()
  |                               |    Verify client's verify_data
  |                               |  EnablePendingCipherRead()
  |                               |  CompleteHandshake()
  |                               |
  | <==== Application Data ====> |  CS_END

Key Exchange (ECDHE)

TLS 1.3 key exchange happens inside Generate13ServerHello(), NOT via the m_keyExchange object:

1. SelectKeyShareGroup() - negotiate ECDH group from client/server supported groups
2. FindEarlyKeyShare() - check if client sent a share for the negotiated group
3. If no match → trigger HelloRetryRequest
4. If match → create TlsAgreement:
   • agreement.ReceivePeerValue(clientShare.KeyExchange) - feed client's public key
   • agreement.GenerateEphemeral() - generate server ephemeral key pair
   • agreement.CalculateSecret() - compute shared secret

Secret Derivation

Three phases of key derivation via HKDF:

0-RTT:  earlySecret = HKDF-Extract(0, PSK_or_0)
                              |
Handshake: handshakeSecret = HKDF-Extract(derivedSecret, sharedSecret)
                              |
App:       masterSecret = HKDF-Extract(derivedSecret, 0)

• Establish13PhaseSecrets() - derives all three phase secrets
• Establish13PhaseHandshake() - derives handshake traffic keys from handshakeSecret
• Establish13PhaseApplication() - derives application traffic keys from masterSecret

TLS 1.3 vs TLS 1.2 Differences in BouncyCastle

Pitfalls Encountered

1. KeyExchangeAlgorithm.NULL (0)

TLS 1.3 cipher suites map to KeyExchangeAlgorithm.NULL = 0. DefaultTlsServer.GetCredentials() doesn't handle this case, throws internal_error. Fix: override GetCredentials() to handle NULL.

2. Certificate.Encode() certificateRequestContext

Certificate constructed via new Certificate(TlsCertificate[]) sets certificateRequestContext = null. TLS 1.3's Encode() asserts it must be non-null. Fix: for TLS 1.3, reconstruct with new Certificate(TlsUtilities.EmptyBytes, entries).

3. RSA-PSS SignatureAndHashAlgorithm

GetInstance(HashAlgorithm.sha256, SignatureAlgorithm.rsa_pss_rsae_sha256) creates a wrong instance (hash=4). The correct instance has hash=8 (Intrinsic). Fix: use SignatureAndHashAlgorithm.rsa_pss_rsae_sha256 static field. TLS 1.3 clients reject PKCS#1 v1.5 in CertificateVerify with illegal_parameter(47).

HTTP/2 Frame Handling

Overview

HTTP/2 (RFC 9113) is a binary framing protocol that multiplexes multiple requests/responses over a single TCP connection. Unlike HTTP/1.1's text-based request/response model, HTTP/2 uses binary frames (HEADERS, DATA, SETTINGS, etc.) carried on numbered streams.

Key HTTP/2 Concepts

Connection Preface

The client begins by sending a 24-byte magic string (connection preface):

PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n

This is immediately followed by a SETTINGS frame (which may be empty). The server must send its own SETTINGS frame in response.

Frame Format

Every HTTP/2 frame has a fixed 9-byte header followed by a variable-length payload:

+-----------------------------------------------+
|                Length (24 bits)                |
+---------------+---------------+---------------+
|   Type (8)    |   Flags (8)   |
+-+-------------+---------------+-------------------------------+
|R|                Stream Identifier (31)                       |
+=+=============================================================+
|                   Frame Payload (0...)                       ...
+---------------------------------------------------------------+

• Length: 24-bit unsigned integer, max 2^24-1 (16,777,215) bytes
• Type: 8-bit frame type identifier
• Flags: 8-bit field, semantics depend on frame type
• R: 1-bit reserved, must be 0
• Stream Identifier: 31-bit unsigned integer, 0 for connection-level frames

Frame Types

Common Flags

Stream Lifecycle

Streams in HTTP/2 have well-defined states:

                  +--------+
          send PP |        | recv PP
      ,-----------|  idle  |-----------.
     /            |        |            \
    v             +--------+             v
+----------+                         +----------+
|          |       send H / recv H    |          |
| reserved |                         | reserved |
| (local)  |                         | (remote) |
|          |                         |          |
+----------+                         +----------+
      |         send H / recv H           |
      |            ,----------------------'
      v           v
   +--------+
   |        |
   |  open  |
   |        |
   +--------+
      |           ^
 send ES / recv ES |
      `-----------'

A stream transitions from idle → open when the first HEADERS frame is sent/received. The stream moves to half-closed (local) or half-closed (remote) when END_STREAM is sent or received, respectively. When both sides have sent/received END_STREAM, the stream is closed.

HPACK Header Compression

HTTP/2 mandates HPACK (RFC 7541) for header compression to reduce overhead from repeated header fields. HPACK uses:

• Static Table: 61 predefined header field name-value pairs (e.g., :method: GET at index 2, :path: / at index 4)
• Dynamic Table: Built incrementally during the connection, populated by literal header fields with indexing
• Huffman Coding: Optional Huffman-encoded string representation for compact transmission

HPACK Representations

Pseudo-Headers

HTTP/2 requests use pseudo-headers (prefixed with :) instead of a request line:

Http2FrameHandler Usage

Class Overview

Http2FrameHandler (cafe.network/http/Http2FrameHandler.cs) is an abstract base class that incrementally parses HTTP/2 frames from a byte stream and dispatches them to type-specific virtual handler methods.

Key Methods

Virtual Handler Methods

Override these in a subclass to implement frame processing logic:

protected virtual void OnData(Http2Frame frame) { }
protected virtual void OnHeaders(Http2Frame frame) { }
protected virtual void OnPriority(Http2Frame frame) { }
protected virtual void OnRstStream(Http2Frame frame) { }
protected virtual void OnSettings(Http2Frame frame) { }
protected virtual void OnPushPromise(Http2Frame frame) { }
protected virtual void OnPing(Http2Frame frame) { }
protected virtual void OnGoAway(Http2Frame frame) { }
protected virtual void OnWindowUpdate(Http2Frame frame) { }
protected virtual void OnContinuation(Http2Frame frame) { }
protected virtual void OnUnknownFrame(Http2Frame frame) { }

Helper Methods for Constructing Frames

Usage Pattern

// 1. Subclass Http2FrameHandler
class MyHandler : Http2FrameHandler
{
    public bool IsComplete { get; private set; }
    
    protected override void OnSettings(Http2Frame frame)
    {
        // Acknowledge settings
        var ack = Http2FrameHandler.CreateSettingsAckFrame();
        // ... send ack frame bytes back to client
    }

    protected override void OnHeaders(Http2Frame frame)
    {
        // Parse pseudo-headers (:method, :path, etc.) from payload
        // Check END_STREAM flag to determine if body follows
    }

    protected override void OnData(Http2Frame frame)
    {
        // Accumulate request body from payload
        if (frame.HasFlag(Http2FrameFlags.EndStream))
            IsComplete = true;
    }
}

// 2. Feed bytes into the handler
var handler = new MyHandler();
while (moreDataAvailable)
{
    int bytesRead = await stream.ReadAsync(buffer);
    handler.ProcessBytes(new ReadOnlySpan<byte>(buffer, 0, bytesRead));
}

Integration with HttpParser.HandleHttp2

The HttpParser class (cafe.network/http/HttpParser.cs) uses an internal Http2RequestHandler subclass to handle HTTP/2 request parsing:

1. After detecting the HTTP/2 connection preface, the remaining bytes are fed to the handler
2. Data is read from the stream in a loop, processed incrementally via ProcessBytes()
3. The handler collects HEADERS frames (parsing :method, :path, and regular headers) and DATA frames (accumulating the body)
4. When END_STREAM is received, parsing completes and BuildRequest() assembles an HttpRequest object

Request Parsing Flow

Client                          Server
  |                               |
  |  Connection Preface           |
  |  "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n" |
  |------------------------------>|  ValidateHttp2()
  |                               |
  |  SETTINGS frame (0, Ack=0)   |
  |------------------------------>|  OnSettings() - marks received
  |                               |
  |  HEADERS frame (Stream 1)    |
  |    :method: GET              |
  |    :path: /api/test          |
  |    END_STREAM=1               |  <- No body
  |------------------------------>|  OnHeaders() → ParseHeadersPayload()
  |                               |    → _streamMethod[1] = "GET"
  |                               |    → _streamPath[1] = "/api/test"
  |                               |  MarkStreamEnded(1)
  |                               |  CheckStreamComplete(1) → IsComplete
  |                               |
  | <======== BuildRequest() ======== |
  |     HttpRequest { Method="GET", Path="/api/test", Version="HTTP/2" }

With Request Body

  |                               |
  |  HEADERS frame (Stream 1)    |
  |    :method: POST             |
  |    :path: /api/submit        |
  |    END_HEADERS=1              |
  |------------------------------>|  OnHeaders()
  |                               |  (END_STREAM not set, body will follow)
  |                               |
  |  DATA frame (Stream 1)       |
  |    Payload: {"key":"val"}    |
  |    END_STREAM=1               |
  |------------------------------>|  OnData() → accumulate body
  |                               |  MarkStreamEnded(1)
  |                               |  CheckStreamComplete(1) → IsComplete
  |                               |
  | <======== BuildRequest() ======== |
  |     HttpRequest { Body = {"key":"val"}, ... }

Limitations

• HPACK Decoding is simplified: indexed header fields (those referencing the static/dynamic tables) are skipped. A full HPACK decoder with dynamic table support is needed for production use.
• Huffman Decoding is not implemented; Huffman-encoded string literals will produce garbled output.
• Server Push (PUSH_PROMISE frames) are not handled.
• Flow Control is not enforced; WINDOW_UPDATE processing is not implemented in the request handler.
• Continuation Fragmentation: Headers split across HEADERS + CONTINUATION frames are supported, but the frame ordering constraints are not validated.

HTTP Server Core Components

TcpHost

TcpHost Class Overview

TcpHost (cafe.network/TcpHost.cs) is the TCP server entry point. It listens on a specified port, accepts incoming connections, and dispatches each connection to HttpHandler for processing. It supports both plain TCP and TLS (via BouncyCastle) connections.

TcpHost Constructors

TcpHost Key Methods

TcpHost Usage Pattern

// 1. Plain HTTP server
var host = new TcpHost(keepAliveTimeoutSeconds: 30);
host.Listen(8080);

// 2. HTTPS server (TLS)
var cert = LoadCertificate();       // Org.BouncyCastle.Tls.Certificate
var key = LoadPrivateKey();         // Org.BouncyCastle.Crypto.AsymmetricKeyParameter
var tlsHost = new TcpHost(cert, key, keepAliveTimeoutSeconds: 30);
tlsHost.Listen(443);

// 3. Stop the server
host.Stop();

Internal Flow

Client                          Server
  |                               |
  |  TCP Connect                  |
  |------------------------------>|  AcceptTcpClientAsync()
  |                               |  HttpSessionContext.Instance.Clear()
  |                               |  Taskpool.Start("stream", HandleRequestAsync)
  |                               |
  |  [TLS Handshake if cert/key]  |
  |<----------------------------->|  TlsServerStream wrap
  |                               |
  |  HTTP Request                 |
  |------------------------------>|  HttpHandler.HandleRequestAsync(stream, ct)
  |                               |
  |  HTTP Response                |
  |<------------------------------|  Response.WriteAsync(stream)
  |                               |
  |  [Keep-Alive loop or close]   |

HttpHandler

HttpHandler Class Overview

HttpHandler (cafe.network/http/HttpHandler.cs) handles an HTTP connection within a keep-alive loop. For each request iteration, it clears the session context, parses the request via HttpParser, routes it through Router.Process(), and writes the response back. It is designed to be subclassed for custom handling logic.

HttpHandler Key Methods

HttpHandler Request Processing Flow

┌─────────────────────────────────────────────────┐
│  Keep-Alive Loop                                │
│                                                 │
│  1. HttpSessionContext.Instance.Clear()          │
│  2. HttpSessionContext.Instance.Setup()          │
│     → Creates new HttpRequest & HttpResponse     │
│  3. request = await HttpParser.ParseRequest()    │
│  4. response = Router.Process(request, itid)     │
│  5. Determine keep-alive from request headers    │
│  6. If status >= 400, force close connection     │
│  7. await response.WriteAsync(stream)            │
│  8. If not keep-alive → break loop               │
│                                                 │
└─────────────────────────────────────────────────┘

HttpHandler Keep-Alive Logic

HttpParser

HttpParser Class Overview

HttpParser (cafe.network/http/HttpParser.cs) is a static utility class that parses raw byte streams into HttpRequest objects. It automatically detects HTTP/1.1 vs HTTP/2 by checking for the HTTP/2 connection preface, and delegates to the appropriate internal handler.

HttpParser Key Methods

HttpParser HTTP/1.1 Parsing Flow

1. Read bytes until \r\n\r\n delimiter is found (end of headers)
2. Parse the request line: METHOD PATH VERSION
3. Parse headers line by line (Name: Value)
4. Parse query string from URL if ? is present
5. Read body using one of two strategies:
   • ChunkedBodyParser: when Transfer-Encoding: chunked is present
   • DirectBodyParser: when Content-Length is present (or no body)
6. Returns the populated HttpRequest

Stream bytes → Find \r\n\r\n → Parse request line + headers → Parse body → HttpRequest

HttpParser HTTP/2 Parsing Flow

1. Validate the 24-byte HTTP/2 connection preface (PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n)
2. Create an internal Http2RequestHandler (subclass of Http2FrameHandler)
3. Feed remaining bytes (after preface) into the handler
4. Continue reading from stream and feeding into handler.ProcessBytes() until handler.IsComplete
5. Call handler.BuildRequest() to assemble the final HttpRequest

Stream bytes → Validate preface → Http2RequestHandler.ProcessBytes() → BuildRequest() → HttpRequest

The Http2RequestHandler internal class:

• Collects HEADERS and CONTINUATION frames, parsing pseudo-headers (:method, :path) and regular headers
• Accumulates DATA frame payloads as the request body
• Tracks stream completion via END_STREAM flag
• Handles RST_STREAM (cleanup) and GOAWAY (throw IOException)

HttpSessionContext

HttpSessionContext Class Overview

HttpSessionContext (cafe.network/http/HttpSessionContext.cs) provides per-request context storage using AsyncLocal<T>. It holds the current HttpRequest and HttpResponse objects, making them accessible throughout the async call chain without explicit parameter passing.

HttpSessionContext Key Properties

HttpSessionContext Key Methods

HttpSessionContext Usage Pattern

// The context is automatically managed by HttpHandler in the keep-alive loop.
// Access it anywhere during request processing:

var request = HttpSessionContext.Instance.Request;
var response = HttpSessionContext.Instance.Response;

// Read request info
string method = request.Method;
string path = request.Path;

// Write response
response.StatusCode = 200;
response.Body = Encoding.UTF8.GetBytes("Hello World");

HttpSessionContext Integration with Other Components

Context Lifecycle per Connection

Connection Accepted
  │
  ├─ TcpHost: Clear()
  │
  └─ Keep-Alive Loop (HttpHandler):
       │
       ├─ Clear()  ──→  resets to empty context
       ├─ Setup()  ──→  creates new Request + Response
       ├─ HttpParser.ParseRequest()  ──→  populates Request
       ├─ Router.Process()  ──→  populates Response
       ├─ Response.WriteAsync()  ──→  sends to client
       │
       └─ finally: Clear()