DA FlatBuffers
High-performance binary serialization with field-level encryption for secure data transmission
Key Features
Field-Level Encryption
Encrypt individual fields within FlatBuffer records using AES-256-CTR. Binary structure stays intact for zero-copy reads.
Streaming Dispatcher
Route mixed FlatBuffer messages to type-specific ring buffers for zero-allocation, constant-memory streaming.
High Performance
Generate and encrypt 1 million records at 50+ MB/s using WebAssembly with Crypto++ or OpenSSL.
Multi-Language Support
Run flatc-wasm in any WASM runtime. Embedded FlatBuffers libraries for Go, Python, Rust, Java, C#, Swift, TypeScript, C++, and more.
Federal Compliance Standards
Cryptographic algorithms approved for protecting federal data at rest:
CNSA 2.0
Commercial National Security Algorithm Suite 2.0 mandates AES-256 for all National Security Systems.
View PDF →SP 800-111
Guide to Storage Encryption Technologies for End User Devices. Endorses AES-256 for data at rest.
View Publication →140-3
Security Requirements for Cryptographic Modules. AES-256 approved for all security levels (1-4).
View Standard →STIGs
Security Technical Implementation Guides require FIPS 140-validated AES encryption for CUI.
View STIG Library →Use Cases
Defense & Intelligence
Field-level encryption for mission data with per-user key derivation and need-to-know access control.
Healthcare (HIPAA)
Encrypt PHI fields while leaving non-sensitive metadata readable for indexing and routing.
Financial Services
Protect PII and transaction data at rest with audit-ready key management from HD wallets.
Zero-Trust Architectures
Data remains encrypted until the authorized recipient decrypts with their derived key.
Public Key Infrastructure
Alice encrypts data with Bob's public key. Only Bob can decrypt with his private key.
Login to try the interactive encryption demo
LoginHow ECIES Encryption Works
Ephemeral Key Generation
Alice generates a one-time ephemeral key pair for each encryption. This provides forward secrecy - if Alice's main key is compromised, past messages remain secure.
ECDH Key Exchange
Alice uses her ephemeral private key and Bob's public key to compute a shared secret via Elliptic Curve Diffie-Hellman. Only Alice and Bob can derive this shared secret.
HKDF Key Derivation
The shared secret is passed through HKDF-SHA256 to derive a symmetric AES-256 key and nonce. This stretches the shared secret into cryptographically strong key material.
AES-256-CTR Encryption
Data is encrypted using AES-256 in CTR mode. CTR mode is a stream cipher that preserves plaintext length - critical for FlatBuffers where field sizes must remain constant.
Header Transmission
The ephemeral public key and nonce are sent as a header alongside the ciphertext. Bob uses this header plus his private key to derive the same symmetric key and decrypt.
Supported Elliptic Curves
| Curve | Public Key Size | Use Cases | Notes |
|---|---|---|---|
X25519 |
32 bytes | General purpose, modern apps | Fastest, constant-time, recommended for new projects |
secp256k1 |
33 bytes (compressed) | Bitcoin, Ethereum, blockchain | Same keys can derive blockchain addresses |
P-256 / secp256r1 |
33 bytes (compressed) | TLS, NIST compliance, enterprise | Required for government/regulated industries |
P-384 / secp384r1 |
49 bytes (compressed) | High-security, government Top Secret | 192-bit security level, NSA Suite B approved |
Field-Level Encryption
AES-256 encryption with HD wallet key derivation, meeting federal data-at-rest requirements
Capabilities
Per-Field Encryption
Encrypt individual fields within FlatBuffer records. The binary structure stays intact so upstream code can read encrypted values without schema changes.
HD Wallet Key Derivation
Keys derived from BIP-39 seed phrases using HKDF-SHA256. Each field can have its own derived key for granular access control.
High Performance
Generate and encrypt 1 million records at 50+ MB/s using WebAssembly. Streaming export avoids memory accumulation.
Zero Server Dependency
All cryptographic operations run client-side in the browser. No data leaves your device unless you explicitly export it.
Cryptographic Implementation
Interactive Demo
Select which fields to encrypt, generate a record, and visualize the encryption:
Select Fields to Encrypt
Check the fields you want to encrypt. Unselected fields will remain in plaintext.
Schema Viewer
View FlatBuffers schema (.fbs) and equivalent JSON Schema
.fbs Schema
JSON Schema
Convert JSON ↔ FlatBuffer
JSON Input
Hex Input (FlatBuffer)
FlatBuffer Studio
Create schemas, generate code, and build FlatBuffers in the browser
Files
Schema Definition
Parsed Structure
Item Editor
Preview
// Schema preview will appear here...
Generated Files
Generated Code
// Generated code will appear here...
Data Entry
Buffer Output
Decoded JSON
Build or upload a buffer
Encryption Module (WASM)
AES-256, X25519, secp256k1, P-256, Ed25519
npm install flatc-wasm
View on npm
Streaming Dispatcher
Route mixed FlatBuffer messages to queues by type
How Streaming Works
Message Flow
FlatBuffers are self-describing binary messages with a 4-byte type identifier prefix. The streaming dispatcher reads this identifier and routes each message to the appropriate queue based on its type.
[TYPE_ID:4bytes][SIZE:4bytes][FLATBUFFER:N bytes]
Message Queues
Each message type has a dedicated queue with a fixed capacity. When full, oldest messages are overwritten (FIFO). This enables constant-memory streaming for high-throughput scenarios without allocations.
- O(1) insertion and removal
- Zero allocation after initialization
- Lock-free single-producer design
Use Cases
- Telemetry - Route sensor data by device type
- Gaming - Separate entity updates from events
- Finance - Dispatch orders, trades, and quotes
- IoT - Process heterogeneous device streams
API Reference
Wire Format
Each message is size-prefixed (little-endian) followed by a 4-byte file identifier for routing, then the FlatBuffer payload.
Memory Layout
Messages are stored in contiguous ring buffers in WASM linear memory. Each registered type gets its own buffer:
| Component | Location | Access Method |
|---|---|---|
| Ring Buffer Base | bufferPtr from registration |
getTypeInfo(fileId).bufferPtr |
| Message N | bufferPtr + (N * messageSize) |
getMessage(fileId, N) |
| Latest Message | bufferPtr + (head * messageSize) |
getLatestMessage(fileId) |
| Raw Memory View | HEAPU8.subarray(ptr, ptr + size) |
new Uint8Array(wasm.HEAPU8.buffer, ptr, size) |
getMessage() returns a Uint8Array view directly into WASM memory - no data is copied.
This view is valid until the next pushBytes() call that overwrites the ring buffer slot.
Linear Memory Hexdump
Initialization
Streaming Data
Access Patterns
Interactive Demo
Encrypted Streaming API
Complete producer/consumer streaming with per-field encryption. The producer sends an encryption header first, then encrypted size-prefixed messages. The consumer uses the header to decrypt, with brute force fallback for unknown record indices.
import { EncryptionContext } from 'flatc-wasm';
import { Builder } from 'flatbuffers';
import { Monster } from './monster_generated';
// 1. Create encryption context with receiver's public key
const ctx = EncryptionContext.forEncryption(bobPublicKey, {
algorithm: 'x25519',
context: 'monster-stream-v1'
});
// 2. Get the encryption header (sent FIRST to consumer)
const header = ctx.getHeaderJSON();
// { ephemeral_key: "a1b2c3...", nonce_start: "d4e5f6...",
// algorithm: 0, context: "monster-stream-v1" }
sendToConsumer(JSON.stringify(header));
// 3. Produce encrypted messages
for (let i = 0; i < recordCount; i++) {
ctx.setRecordIndex(i);
const builder = new Builder(256);
// ... build Monster FlatBuffer ...
const buf = builder.finishSizePrefixed(monster);
const encrypted = ctx.encryptBuffer(builder.asUint8Array());
sendToConsumer(encrypted);
}
import { EncryptionContext } from 'flatc-wasm';
import { Monster } from './monster_generated';
// 1. Receive the encryption header
const header = JSON.parse(receiveFromProducer());
const ctx = EncryptionContext.forDecryption(myPrivateKey, header);
// 2. Decrypt messages with known record index
function decryptMessage(encrypted, recordIndex) {
ctx.setRecordIndex(recordIndex);
return ctx.decryptBuffer(encrypted);
}
// 3. Brute force fallback (unknown record index)
function decryptWithRecovery(encrypted, maxAttempts = 1000) {
for (let i = 0; i < maxAttempts; i++) {
ctx.setRecordIndex(i);
try {
const result = ctx.decryptBuffer(encrypted);
// Validate FlatBuffer structure (magic bytes / vtable)
const view = new DataView(result.buffer);
const rootOffset = view.getUint32(0, true);
if (rootOffset < result.length) {
return { data: result, recordIndex: i };
}
} catch (e) { continue; }
}
throw new Error('Decryption failed: record index not found');
}
import (
"encoding/binary"
"encoding/json"
flatbuffers "github.com/google/flatbuffers/go"
)
// 1. Initialize WASM module and create encryption context
module := loadWasmModule("flatc-wasm.wasm")
ctx := module.EncryptionContextForEncryption(bobPubKey, "x25519", "stream-v1")
// 2. Send encryption header first
headerJSON := ctx.GetHeaderJSON()
sendToConsumer(headerJSON)
// 3. Produce encrypted size-prefixed messages
for i := 0; i < recordCount; i++ {
ctx.SetRecordIndex(i)
builder := flatbuffers.NewBuilder(256)
// ... build Monster FlatBuffer ...
builder.FinishSizePrefixed(monster)
buf := builder.FinishedBytes()
encrypted := ctx.EncryptBuffer(buf)
sendToConsumer(encrypted)
}
import (
"encoding/binary"
"encoding/json"
)
// 1. Receive encryption header
var header EncryptionHeader
json.Unmarshal(receiveFromProducer(), &header)
ctx := module.EncryptionContextForDecryption(myPrivKey, header)
// 2. Decrypt with known index
func decryptMessage(ctx *EncryptionContext, data []byte, idx int) []byte {
ctx.SetRecordIndex(idx)
return ctx.DecryptBuffer(data)
}
// 3. Brute force fallback
func decryptWithRecovery(ctx *EncryptionContext, data []byte) ([]byte, int) {
for i := 0; i < 1000; i++ {
ctx.SetRecordIndex(i)
result := ctx.DecryptBuffer(data)
if result != nil && isValidFlatBuffer(result) {
return result, i
}
}
return nil, -1
}
import struct, json
from flatc_wasm import EncryptionContext
# 1. Create encryption context
ctx = EncryptionContext.for_encryption(
bob_public_key, algorithm="x25519", context="stream-v1"
)
# 2. Send encryption header first
header = ctx.get_header_json()
send_to_consumer(json.dumps(header))
# 3. Produce encrypted messages
for i in range(record_count):
ctx.set_record_index(i)
buf = build_monster_flatbuffer(data[i]) # size-prefixed
encrypted = ctx.encrypt_buffer(buf)
send_to_consumer(encrypted)
import struct, json
from flatc_wasm import EncryptionContext
# 1. Receive encryption header
header = json.loads(receive_from_producer())
ctx = EncryptionContext.for_decryption(my_private_key, header)
# 2. Decrypt with known index
def decrypt_message(ctx, data, index):
ctx.set_record_index(index)
return ctx.decrypt_buffer(data)
# 3. Brute force fallback
def decrypt_with_recovery(ctx, data, max_attempts=1000):
for i in range(max_attempts):
ctx.set_record_index(i)
try:
result = ctx.decrypt_buffer(data)
if is_valid_flatbuffer(result):
return result, i
except Exception:
continue
raise RuntimeError("Record index not found")
use flatc_wasm::EncryptionContext;
// 1. Create encryption context
let ctx = EncryptionContext::for_encryption(
&bob_public_key, "x25519", "stream-v1"
)?;
// 2. Send header first
let header = ctx.get_header_json();
send_to_consumer(header.as_bytes());
// 3. Produce encrypted messages
for i in 0..record_count {
ctx.set_record_index(i);
let buf = build_monster(i); // size-prefixed
let encrypted = ctx.encrypt_buffer(&buf)?;
send_to_consumer(&encrypted);
}
use flatc_wasm::EncryptionContext;
// 1. Receive header
let header: EncryptionHeader = serde_json::from_slice(
&receive_from_producer()
)?;
let ctx = EncryptionContext::for_decryption(
&my_private_key, &header
)?;
// 2. Decrypt with known index
fn decrypt(ctx: &mut EncryptionContext, data: &[u8], idx: u32)
-> Result<Vec<u8>> {
ctx.set_record_index(idx);
ctx.decrypt_buffer(data)
}
// 3. Brute force fallback
fn decrypt_recover(ctx: &mut EncryptionContext, data: &[u8])
-> Result<(Vec<u8>, u32)> {
for i in 0..1000 {
ctx.set_record_index(i);
if let Ok(result) = ctx.decrypt_buffer(data) {
if is_valid_flatbuffer(&result) {
return Ok((result, i));
}
}
}
Err(anyhow!("Record index not found"))
}
import com.google.flatbuffers.FlatBufferBuilder;
// 1. Create encryption context
EncryptionContext ctx = EncryptionContext
.forEncryption(bobPublicKey, "x25519", "stream-v1");
// 2. Send header first
String headerJson = ctx.getHeaderJSON();
sendToConsumer(headerJson.getBytes());
// 3. Produce encrypted messages
for (int i = 0; i < recordCount; i++) {
ctx.setRecordIndex(i);
FlatBufferBuilder builder = new FlatBufferBuilder(256);
// ... build Monster ...
builder.finishSizePrefixed(monster);
byte[] encrypted = ctx.encryptBuffer(builder.sizedByteArray());
sendToConsumer(encrypted);
}
// 1. Receive header
EncryptionHeader header = EncryptionHeader
.fromJSON(new String(receiveFromProducer()));
EncryptionContext ctx = EncryptionContext
.forDecryption(myPrivateKey, header);
// 2. Decrypt with known index
byte[] decrypt(EncryptionContext ctx, byte[] data, int idx) {
ctx.setRecordIndex(idx);
return ctx.decryptBuffer(data);
}
// 3. Brute force fallback
DecryptResult decryptRecover(EncryptionContext ctx, byte[] data) {
for (int i = 0; i < 1000; i++) {
ctx.setRecordIndex(i);
try {
byte[] result = ctx.decryptBuffer(data);
if (isValidFlatBuffer(result))
return new DecryptResult(result, i);
} catch (Exception e) { continue; }
}
throw new RuntimeException("Record index not found");
}
using FlatBuffers;
// 1. Create encryption context
var ctx = EncryptionContext.ForEncryption(
bobPublicKey, "x25519", "stream-v1");
// 2. Send header first
string headerJson = ctx.GetHeaderJSON();
SendToConsumer(Encoding.UTF8.GetBytes(headerJson));
// 3. Produce encrypted messages
for (int i = 0; i < recordCount; i++) {
ctx.SetRecordIndex(i);
var builder = new FlatBufferBuilder(256);
// ... build Monster ...
builder.FinishSizePrefixed(monster);
var encrypted = ctx.EncryptBuffer(builder.SizedByteArray());
SendToConsumer(encrypted);
}
// 1. Receive header
var header = EncryptionHeader.FromJSON(
Encoding.UTF8.GetString(ReceiveFromProducer()));
var ctx = EncryptionContext.ForDecryption(
myPrivateKey, header);
// 2. Decrypt with known index
byte[] Decrypt(EncryptionContext ctx, byte[] data, int idx) {
ctx.SetRecordIndex(idx);
return ctx.DecryptBuffer(data);
}
// 3. Brute force fallback
(byte[] data, int idx) DecryptRecover(
EncryptionContext ctx, byte[] data) {
for (int i = 0; i < 1000; i++) {
ctx.SetRecordIndex(i);
try {
var result = ctx.DecryptBuffer(data);
if (IsValidFlatBuffer(result))
return (result, i);
} catch { continue; }
}
throw new Exception("Record index not found");
}
import FlatBuffers
// 1. Create encryption context
let ctx = try EncryptionContext.forEncryption(
publicKey: bobPublicKey,
algorithm: .x25519,
context: "stream-v1"
)
// 2. Send header first
let header = ctx.getHeaderJSON()
sendToConsumer(header.data(using: .utf8)!)
// 3. Produce encrypted messages
for i in 0..<recordCount {
ctx.setRecordIndex(i)
var builder = FlatBufferBuilder(initialSize: 256)
// ... build Monster ...
builder.finishSizePrefixed(offset: monster)
let encrypted = try ctx.encryptBuffer(builder.sizedByteArray)
sendToConsumer(encrypted)
}
import FlatBuffers
// 1. Receive header
let headerData = receiveFromProducer()
let header = try EncryptionHeader.fromJSON(headerData)
let ctx = try EncryptionContext.forDecryption(
privateKey: myPrivateKey, header: header)
// 2. Decrypt with known index
func decrypt(_ ctx: EncryptionContext,
_ data: Data, index: Int) throws -> Data {
ctx.setRecordIndex(index)
return try ctx.decryptBuffer(data)
}
// 3. Brute force fallback
func decryptRecover(_ ctx: EncryptionContext,
_ data: Data) throws -> (Data, Int) {
for i in 0..<1000 {
ctx.setRecordIndex(i)
if let result = try? ctx.decryptBuffer(data),
isValidFlatBuffer(result) {
return (result, i)
}
}
throw StreamError.recordIndexNotFound
}
Encryption Header Fields
The encryption header is sent once at the start of a stream. Without it, the consumer cannot decrypt any messages.
| Field | Type | Description |
|---|---|---|
ephemeral_public_key |
bytes | Sender's ephemeral public key for ECDH key exchange. Combined with receiver's private key to derive the shared encryption key. |
nonce_start |
12 bytes | Random starting nonce. Each field's nonce is derived as: nonce_start + (recordIndex * 65536 + fieldId). Sent once, not per-message. |
algorithm |
enum | Key exchange algorithm: 0 = X25519, 1 = secp256k1, 2 = P-256 (FIPS), 3 = P-384 |
context |
string | Application context mixed into HKDF key derivation. Must match between producer and consumer. |
timestamp |
uint64 | Session creation time (Unix ms). Used for key rotation and session expiry. |
FinishSizePrefixed when building to create streamable buffers.
Aligned Binary Format
Zero-overhead, fixed-size structs from FlatBuffers schemas for WASM/native interop
Why Aligned Format?
Zero-Copy Access
TypedArray views directly into WASM linear memory. No deserialization overhead - just cast and access.
Fixed-Size Layout
Predictable memory layout with proper alignment. Perfect for arrays of structs in shared memory.
FlatBuffers Schema
Use familiar FlatBuffers .fbs syntax. Structs and tables with scalars and fixed-size arrays are supported.
Multi-Language Output
Generate C++ headers, TypeScript classes, and JavaScript modules from a single schema definition.
Code Generator
Paste a FlatBuffers schema below to generate aligned binary format code:
Schema (FBS)
Generated Code
// Generated code will appear here...
Supported Features
Supported
- Scalars: bool, byte, ubyte, short, ushort, int, uint, long, ulong, float, double
- Fixed-size arrays:
[float:3],[int:16] - Hex array sizes:
[ubyte:0x100](256 bytes) - Nested structs (inlined by value)
- Enums with explicit base type
- Fixed-length strings (set String Length > 0)
Not Supported
- Variable-length strings (without String Length)
- Variable-length vectors
- Unions
- Tables with optional fields (use structs)
Usage Example
// C++ - Direct struct access in WASM
#include "aligned_types.h"
void processEntities(Entity* entities, size_t count) {
for (size_t i = 0; i < count; i++) {
Entity& e = entities[i];
e.health -= 10;
e.position.x += e.velocity.x * dt;
// Zero overhead - direct memory access
}
}
// Export for JS binding
extern "C" void update_entities(Entity* ptr, int count) {
processEntities(ptr, count);
}
WASM Interop Patterns
Since aligned structs have no embedded length metadata, array bounds must be communicated out-of-band. Here are common patterns for sharing data between WASM modules:
// C++ WASM - Export pointer and count separately
static Cartesian3 positions[10000];
static uint32_t count = 0;
extern "C" {
Cartesian3* get_positions() { return positions; }
uint32_t get_count() { return count; }
}
// TypeScript - Read using exported functions
const ptr = wasm.exports.get_positions();
const count = wasm.exports.get_count();
const positions = Cartesian3ArrayView.fromMemory(wasm.memory, ptr, count);
for (const pos of positions) {
console.log(`(${pos.x}, ${pos.y}, ${pos.z})`);
}
// Schema - Store indices instead of pointers
struct Cartesian3 { x: double; y: double; z: double; }
table Satellite {
norad_id: uint32;
position_index: uint32; // Index into positions array
velocity_index: uint32; // Index into velocities array
}
// TypeScript - Compute offset from index
const CARTESIAN3_SIZE = 24; // 3 doubles × 8 bytes
class SpaceData {
constructor(memory, positionsBase, satellitesBase) {
this.memory = memory;
this.positionsBase = positionsBase;
this.satellitesBase = satellitesBase;
}
// O(1) lookup by index
getPositionByIndex(index) {
const offset = this.positionsBase + index * CARTESIAN3_SIZE;
return Cartesian3View.fromMemory(this.memory, offset);
}
// Follow cross-reference from satellite to position
getSatellitePosition(satIndex) {
const sat = this.getSatelliteByIndex(satIndex);
return this.getPositionByIndex(sat.position_index);
}
}
// Schema - Manifest with indices into data array
table EphemerisManifest {
satellite_ids: [uint32:100];
start_indices: [uint32:100]; // Where each satellite's data starts
point_counts: [uint32:100]; // How many points per satellite
total_satellites: uint32;
}
struct EphemerisPoint {
julian_date: double;
x: double; y: double; z: double;
vx: double; vy: double; vz: double;
}
// TypeScript - Navigate using manifest
const POINT_SIZE = 56; // 7 doubles × 8 bytes
class EphemerisReader {
constructor(memory, manifestPtr, pointsPtr) {
this.manifest = EphemerisManifestView.fromMemory(memory, manifestPtr);
this.pointsBase = pointsPtr;
this.memory = memory;
}
// Get all points for a satellite
getSatellitePoints(satIndex) {
const startIdx = this.manifest.start_indices[satIndex];
const count = this.manifest.point_counts[satIndex];
const offset = this.pointsBase + startIdx * POINT_SIZE;
return new EphemerisPointArrayView(this.memory.buffer, offset, count);
}
// Get specific point: base + (startIndex + timeIndex) * size
getPoint(satIndex, timeIndex) {
const startIdx = this.manifest.start_indices[satIndex];
const offset = this.pointsBase + (startIdx + timeIndex) * POINT_SIZE;
return EphemerisPointView.fromMemory(this.memory, offset);
}
}
byte_offset = base_ptr + index × STRUCT_SIZE — Since struct sizes are fixed and known at compile time, any index can be converted to a byte offset with a single multiplication.
WASM Runtime Integration
Run the encryption module in any language with WebAssembly support
Why WASM Runtimes?
Single Auditable Implementation
One C++/Crypto++ codebase compiled to WASM. Audit once, deploy everywhere.
Battle-Tested Crypto
Crypto++ has 30+ years of production use in security-critical applications.
Cross-Language Interop
Data encrypted in Go can be decrypted in Python, Rust, Java, and vice versa.
Installation
Install via npm to get the FlatBuffers compiler with encryption support:
npm install flatc-wasm@25.12.19-wasm.43
Works with any WASM runtime: Node.js, browsers, Go (wazero), Python (wasmer), Rust (wasmer), Java (Chicory), C# (Wasmtime), Swift (WasmKit), and more.
WASM Runtime Integration
Run flatc-wasm in any language with a WebAssembly runtime. Load the .wasm binary and call exported functions directly:
Pure Go WebAssembly runtime - no CGo required
Load WASM modules with Cranelift JIT compilation
Native WASM execution with wasmer crate
Pure Java WASM interpreter - no JNI required
Wasmtime .NET bindings for WASM execution
Pure Swift WASM runtime for iOS/macOS
Native V8 WASM support - no extra runtime needed
All modern browsers support WASM natively
FlatBuffers Language Bindings
Download the FlatBuffers runtime library for your target language. These are the libraries your generated code imports to read/write FlatBuffer data:
FlatBuffers Go runtime package
FlatBuffers Python runtime with type hints
FlatBuffers Rust crate with no_std support
FlatBuffers Java runtime classes
FlatBuffers .NET runtime library
FlatBuffers Swift package for iOS/macOS
FlatBuffers TypeScript/JavaScript runtime
FlatBuffers C++ header-only library