Cosdata Transaction System Design

Overview

Conceptual Distinction: Transactions vs Streaming Data

Transactions (Explicit Only)

Streaming Data

Explicit Transactions

Conceptual Foundation

Transaction Lifecycle and Resource Management

Asynchronous Indexing Process

Progress Monitoring and Observability

Concurrency Model and Resource Protection

Streaming Data Ingestion

Design Philosophy and Use Cases

API Design for Streaming Operations

Streaming Upsert Endpoint

Streaming Update Endpoint

Streaming Delete Endpoint

Immediate Indexing and Availability

Epoch-Based WAL Management and Version Creation

Periodic Serialization and Version Creation

Interaction Between Transactions and Streaming Data

Epoch-Based Version Allocation

Version Assignment During Mixed Ingestion Scenarios

Transaction Lifecycle and Timeout Handling

Version Number Continuity and Gap Prevention

Version Management and Historical Consistency

Unified Version Control Architecture

Ordering Guarantees and Consistency

Integration with Context-Based Querying

Conclusion