# GenosDB: Introducing Oplog-Driven Intelligent Delta Sync and Full-State Fallback

Full-state replication is the simplest approach to P2P sync — and the most wasteful. Every reconnection sends the entire database, regardless of what changed. **GenosDB's oplog-driven delta sync** replaces this with a hybrid protocol: an operation log tracks every mutation, peers exchange only what they missed, and a full-state fallback handles extended disconnections automatically.

This release is the culmination of intensive engineering efforts to build a professional-grade synchronization layer suitable for demanding, real-time distributed applications.

## Core Architecture

The centerpiece of this release is a new dual-mode synchronization engine designed to balance performance with absolute reliability.

### 1. Delta Synchronization via Sliding-Window Oplog

The primary synchronization path is now driven by a lightweight, in-memory Operation Log.

- **Operation Log (Oplog):** A fixed-size, sliding-window Oplog is now maintained by each peer. This log stores a history of the most recent mutation operations (upsert, remove, link), represented as minimal, lightweight entries (`{type, id, timestamp}`).
- **Delta Exchange Protocol:** Upon connection, peers now initiate a `syncRequest` handshake, exchanging their latest known HybridClock timestamp. The responding peer queries its Oplog to compute the minimal set of operations (the "delta") required to bring the requesting peer up to date.
- **Payload Compression:** To further optimize network throughput, the computed delta payload (an array of "hydrated" operation objects) is serialized using MessagePack and subsequently compressed via pako (deflate). This results in a significant reduction in payload size, often exceeding 80%, minimizing latency on all connections.

### 2. Robust Full-State Sync Fallback Mechanism

To handle cases of severe desynchronization, such as a peer rejoining after an extended offline period, we have implemented a robust fallback mechanism.

- **Consistency Boundary Detection:** The `syncRequest` handler intelligently detects when a requesting peer's last known timestamp predates the oldest entry in the responder's sliding-window Oplog.
- **Automatic State Transition:** When this boundary condition is met, the system transparently aborts the delta-sync process and transitions to a full-state synchronization. It sends the entire current graph object, ensuring guaranteed eventual consistency.
- **State Reconciliation:** The receiving peer's `syncReceive` handler is designed to atomically apply the full state, which includes replacing its local graph and clearing its now-irrelevant Oplog to prevent historical conflicts.

## Delta-Aware Security

The new synchronization protocol required a significant enhancement to our SoftwareSecurityManager (SSM) to maintain our zero-trust security model.

- **Deep Packet Inspection for Deltas:** The `verifyIncomingOperations` method in the SSM has been refactored to be "container-aware". It now identifies `deltaSync` payloads, decompresses them, and performs granular permission verification (using the `rbac.can()` function) on **each individual operation** within the delta.
- **Atomic Validation:** The SSM reconstructs and re-compresses a new delta payload containing only the subset of operations for which the original sender had explicit permission. This prevents privilege escalation or unauthorized data mutation via sync packets.
- **Dependency Injection:** To maintain modularity and keep the RBAC module lightweight, the SSM now receives compression/serialization tools (pako, msgpack) via dependency injection from the main GDB instance, cleanly separating concerns between the security and transport layers.

## Bug Fixes

This architectural overhaul also allowed us to resolve several critical bugs:

- **Resolved Repetitive Sync Loop:** Corrected a race condition where a peer could re-request deltas it had just processed by implementing more rigorous management of the `globalTimestamp` after applying a change batch.
- **Fixed Cross-Session State Corruption:** The root cause of a bug where a rejoining peer could overwrite a more recent state has been addressed. The system is now fully resilient to this scenario through consistent conflict resolution on all write paths.

## Impact

GenosDB v0.4.0 is a high-performance distributed database for browser environments. This new engine lays the foundation for future optimizations and scaling capabilities. Our focus now shifts to intensive end-to-end testing, API stabilization, and comprehensive documentation as we move towards a v1.0.0 release.

We invite you to explore the new architecture and provide feedback.

---

> **This article is part of the official documentation of GenosDB (GDB).**
> GenosDB is a distributed, modular, peer-to-peer graph database built with a Zero-Trust Security Model, created by **Esteban Fuster Pozzi ([estebanrfp](https://github.com/estebanrfp))**.

📄 [Whitepaper](https://github.com/estebanrfp/gdb/blob/main/WHITEPAPER.md) | overview of GenosDB design and architecture

🛠 [Roadmap](https://github.com/estebanrfp/gdb/blob/main/ROADMAP.md) | planned features and future updates

💡 [Examples](https://github.com/estebanrfp/gdb/blob/main/docs/genosdb-examples.md) | code snippets and usage demos

📖 [Documentation](https://github.com/estebanrfp/gdb/blob/main/docs/index.md) | full reference guide

🔍 [API Reference](https://github.com/estebanrfp/gdb/blob/main/docs/genosdb-api-reference.md) | detailed API methods

📚 [Wiki](https://github.com/estebanrfp/gdb/wiki) | additional notes and guides

💬 [GitHub Discussions](https://github.com/estebanrfp/gdb/discussions) | community questions and feedback

🗂 [Repository](https://github.com/estebanrfp/gdb) | Minified production-ready files

📦 [Install via npm](https://www.npmjs.com/package/genosdb) | quick setup instructions

🌐 [Website](https://estebanrfp.com/) | [GitHub](https://github.com/estebanrfp) | [LinkedIn](https://www.linkedin.com/in/estebanrfp/)