Changelog

All notable changes to OxiDB, organized by version.

v0.42.0

2026-07-31 latest

Added — PostgreSQL wire protocol

  • OxiDB now speaks the PostgreSQL v3 protocol (OXIDB_PG_PORT, requires OXIDB_SQL=1). Verified with real drivers, not against the spec: psql 18, psycopg 3.3, Npgsql 8 in its default mode, pgjdbc 42.7 (including DatabaseMetaData introspection), and DBeaver 25.3 connects and browses with its native PostgreSQL driver. TLS works (sslmode=require); authentication is the same SCRAM-SHA-256 as the native port, so the same accounts work on both.
  • EF Core runs over the unmodified Npgsql provider: EnsureCreated, generated keys via RETURNING, joins, transactions, LATERAL — end to end. Getting there added binary timestamp parameters and results, AT TIME ZONE 'UTC', PostgreSQL 14's 3-argument date_trunc, and calendar INTERVAL arithmetic desugared onto the calendar-correct add_months.
  • System-catalog queries are answered with PostgreSQL's real column sets, filled from OxiDB's own schema; what cannot be answered truthfully is refused by name rather than answered empty — an empty result would be believed. Query results are buffered per message rather than written per row, which took a 1,000-row read from 526 to 1,796 ops/s.

Added — embedded EF Core (.NET)

  • UseOxiDb("Path=./mydata") runs the whole EF Core stack in-process — no server, SQLite-style: the database is a directory next to your application. The same DbContext points at a server by changing one connection string. One engine per directory is shared process-wide while each connection keeps its own interactive transaction, so concurrent transactions from multiple contexts work exactly as they do over TCP. Minimal example at examples/dotnet/EmbeddedEfCore/.
  • Closing an embedded database now checkpoints it: a cleanly exited application leaves a snapshot-only data directory (the WAL folded down to its bare header) — what a backup or sync tool wants to see. A crash still loses nothing; the log tail replays at the next open.

Changed — SQL engine: disk-first by default, measured against PostgreSQL

  • The SQL engine's disk-first mode is now the default (OXIDB_SQL_DISK_FIRST=0 restores all-resident rows): a warm 1.2M-row, index-heavy database costs ~39 MB of process memory instead of hundreds, and rows, primary keys, and every index live in mapped files bounded by the checkpoint interval — not by row count. The sparse row index costs 0.69 bytes per row, so 100M rows need ~69 MB resident where the previous layout needed 3.1 GB.
  • Measured against stock PostgreSQL 18 on identical data, 5 of 8 query workloads are at parity (0.93–1.10x: point lookups, composite-key lookups, secondary-index equality, range + ORDER BY + LIMIT, and low-selectivity index scans via a bitmap-heap-scan-equivalent cursor walk); full scans and joins run at 0.72–0.78x. The benchmark, method, and the honest remainder are in docs/query-benchmark.md.
  • Group commit: concurrent SQL writers now share fsyncs — a flat ~266 writes/s at every concurrency became ~1.2k/s at 16 connections.
  • Opening a database with a large unfolded log peaks far lower (a checkpoint now hard-links unchanged tables instead of rewriting them), and OXIDB_DOC=0 runs the server without the document engine entirely — 9.8 MB idle RSS for a SQL/TSDB-only deployment.

Added — SQL surface

  • Composite PRIMARY KEY (CONSTRAINT pk PRIMARY KEY (a, b)), enforced on every write path including transactions.
  • Integer width enforcement: SMALLINT/INT/TINYINT are range-checked constraints (PostgreSQL error code 22003) rather than silently widened; storage stays i64, and existing catalogs keep their old semantics.
  • CREATE UNIQUE INDEX is enforced — it validates existing rows, then rides the same uniqueness machinery as declared UNIQUE columns, on the live path, WAL replay, and after checkpoints alike. Shapes that cannot be enforced (multi-column, inside a transaction) are refused by name; before this, EF's IsUnique() quietly produced a plain index and duplicates sailed through a constraint the application believed in.

Fixed

  • A concurrent writer could silently overwrite another's row. A transaction reserved row ids by peeking a counter, so a simultaneous writer could be handed the same id and the later commit replaced the earlier row with no error anywhere. Ids and AUTO_INCREMENT values are now reserved from the engine as the transaction buffers each write, and the commit re-validates every key against committed state under the commit lock.
  • An indexed TIMESTAMP column probed with an integer parameter answered 0 rows — exactly what EF Core sends for every DateTime parameter. The index found the entries and the candidate verification then rejected all of them, because index-key equality disagreed with index-key ordering about cross-type numerics. Equality now agrees with ordering.
  • HAVING on a group key mis-read the projection on the streamed aggregation path (comparing the count where the key should be), and HAVING count(*) without an ORDER BY was rejected outright. Both answer correctly now.

v0.40.0

2026-07-28

Changed — OxiDB is source-available

  • Read the source, modify it, and run it in production for your own applications and business — free, at any scale, with no registration. A commercial licence covers two things and only two: offering OxiDB to third parties as a service, and distributing it, alone or embedded in a product. See the licence page.
  • This opens up the v0.33–v0.39 line rather than closing it further: those versions required a licence for any use at all, including running it yourself. Every prior release keeps the licence it was published with, and the MIT client libraries are unchanged — redistribution included.

Fixed — a transaction spanning two collections could recover half-applied

  • The reason this release matters more than its licence change. In disk-first mode — the default since 0.38 — a record reached the data file when a transaction applied it, before its commit mark was durable, and the index is rebuilt on open by scanning for active records. A crash between apply and commit therefore resurrected one collection's half of a transaction while the other half, correctly, was discarded by WAL replay. Deletes had the mirror of it: an uncommitted delete could remove a document permanently.
  • Found by running the crash half of the suite (cargo test -- --ignored), which the normal run skips: three tests failed on one bug — the Jepsen-style bank that kills the process mid-commit, the multi-collection atomicity drill, and exactly-once retry.
  • Records written by an uncommitted transaction are now written pending and are invisible to a rebuild; the record each one displaces stays live. Both are settled at the commit point, so a crash between the mark and the settle costs nothing — replay restores the write, gated on the same commit log. Compaction and checkpointing stand off while any transaction has work outstanding. The in-RAM mode had the same hole by a different route and is covered by the same guard.

Fixed

  • The .NET TCP client could not survive a server restart. It connected once and stayed connected, so a deploy left every request failing until the client process restarted. It now redials and re-authenticates — retrying only where it is safe, and never inside a transaction, where the connection is the transaction.
  • Reading with the wrong encryption key panicked instead of returning an error, reachable from a REST request: non-document bytes went straight to a parser that trusts a length in their header.

v0.39.15

2026-07-24

Fixed — WebSocket handshake

  • The server computed Sec-WebSocket-Accept with the wrong RFC 6455 GUID, so every client that validates the accept hash — browsers, ws, undici — refused the connection, and only clients that skipped the check could connect. The GUID is now the RFC value, so native WebSocket works everywhere; the JavaScript client's hand-rolled Node WebSocket workaround is gone in favour of the platform one (oxidb npm 0.26.0, Node 22+).

Added — realtime subscriptions

  • Live change streams over WebSocket, scoped to a tenant project. {"cmd":"auth","token":…,"db":"<ref>"} verifies against that project's ES256 key and pins the connection to that database.
  • Security rules are enforced on the WebSocket surface too, at parity with REST: find/count filter per row, writes check per document, and a subscribe delivers an RLS-filtered event stream — an event whose document the caller may not see is dropped rather than leaking its id. Engine fix: insert_many emitted change events with no document body (the path every REST insert takes), so subscribers now receive the inserted document.

Added — per-project file storage and backup

  • /api/storage — list buckets and objects, upload, download (original content type + ETag), delete, HEAD metadata. Isolation is per tenant database, a per-project storage quota is enforced at upload time, anonymous keys are read-only, and a non-empty bucket refuses to delete.
  • POST /api/backup?db=<ref> (admin) streams a tar.gz of that database as an attachment. Stateless — nothing is retained server-side to expire or leak.

Added — SQL: ALTER TABLE … ALTER COLUMN … TYPE

  • PostgreSQL ALTER COLUMN c [SET DATA] TYPE t and MySQL MODIFY COLUMN. Every row is dry-run cast first — an uncastable value or an over-length VARCHAR(n) aborts before anything reaches the WAL — then the column is rewritten in place, indexes rebuilt, and a checkpoint taken. Columns bound by PRIMARY KEY / AUTO_INCREMENT / UNIQUE / FOREIGN KEY are refused, since a cast can collide previously distinct keys.

Added — OxiBase

  • Email-based end-user auth — address verification, password reset and administrative user management, delivered over real SMTP. Without SMTP configured, the previous behaviour is unchanged.
  • Per-project request logs (the data plane tags each request with its target database; the control plane exposes a paged, filterable log endpoint) and TypeScript type generation — exact for SQL tables, inferred by sampling for document collections.
  • Dashboard: Files, Logs and Users tabs; editable SQL tables (rows and columns, including retype and drop); document row editing; a CodeMirror SQL editor with OxiDB dialect highlighting; one-click backup.

v0.39.10

2026-07-23

Added — OxiBase, a multi-tenant backend on top of OxiDB

  • A multi-tenant control plane. Provision isolated tenant projects, each with its own database and ES256/JWKS API keys (anon + service_role). Per-project end-user auth (sign-up / sign-in with rotating refresh tokens), path-based tenant addressing (<host>/<project>/rest/v1/…) so no wildcard cert is needed, and a static dashboard. Developer sign-in is Google-only.
  • Row-level security. A read rule that references doc.<field> is enforced per returned row — an unfiltered select returns only the caller's own rows. Security-rule expressions are validated before they are saved, so a typo can no longer become a silent fail-closed “deny all”.
  • Per-project resource quotas. Collection, SQL-table and total-document caps, owned by the control plane and enforced in the data plane at creation/insert time; shown and editable in the dashboard.

Added — observability

  • Request logging to OxiDB itself. The server can ship a structured message per request to OxiDB's own GELF ingest port, or to a lighter MessagePack log port (compact binary, no per-field auto-indexing) — so a load test's every operation lands in a queryable collection.
  • WASM OPFS persistence. The in-browser build can snapshot its database to the Origin Private File System and restore it on reload, so data survives page refreshes.

Changed — memory: one shared document cache

  • Resident memory no longer scales with the number of collections. The deserialized-value and encoded-bytes caches were per-collection, each with its own budget, so a many-collection / multi-tenant workload multiplied RAM. They are now a single process-global cache under one budget. On a 20-tenant, 100-collection, 500k-document load test this cut resident memory from ~1.3 GiB to ~307 MiB at the same throughput. Tune with OXIDB_DOC_CACHE_SIZE / OXIDB_DOC_BYTES_CACHE_SIZE.

v0.38.0

2026-07-19

Changed — disk-first document storage is the default

  • Resident memory no longer scales with your data. Document bodies now live in an mmap'd, zstd-compressed file with only a ~24 B/doc offset index in RAM; the OS page cache holds the hot part and gives it back under pressure. Measured on a live 48-hour IoT workload: 489 MB resident → ~35 MB, with identical indexed-read throughput and a ~17% write cost. OXIDB_DISK_FIRST=0 restores the always-resident mode; existing collections keep the format they were created with, so upgrading never reinterprets data. The entire test fleet now runs against disk-first, including the SIGKILL crash suites, and encryption at rest was re-verified against every file the engine writes.

Added — SQL

  • SELECT ... FOR UPDATE takes real row locks. It used to parse and silently not lock. Matched rows are now pessimistically locked until commit/rollback; concurrent UPDATE/DELETE/FOR UPDATE on them block (up to OXIDB_SQL_LOCK_TIMEOUT_MS, default 5000 — also how a deadlock resolves). Plain UPDATE/DELETE lock their rows too, closing the engine's lost-update window: two concurrent read-modify-write transactions on one row now serialize. Shapes that cannot lock base rows (joins, aggregates, DISTINCT, set ops, views, derived tables, FOR SHARE) are refused with a clear error, never accepted without the lock — and FOR UPDATE classifies as a write, so oxipool never routes it to a replica. Details.
  • {"cmd": "disk_usage"} — per-engine on-disk footprint of the data directory in one call (documents incl. the mmap'd share, SQL, time-series, blobs, OxiMem, MQTT/AMQP substrates, full-text, PITR archive, system).

Fixed — a WAL durability bug

  • Sealed WAL segments after the first were invisible. The segment scanner's fast path assumed .0 always exists, but the first online checkpoint deleted it — after which later segments were never retired (unbounded disk growth) and, far worse, never replayed at recovery: a crash between a seal and its persist lost the acknowledged writes in that segment. .0 is now a permanent empty sentinel, every checkpoint retires all covered segments (data dirs the old bug left behind self-heal), and three regression tests pin it — the crash-replay test was red before the fix.

v0.37.0

2026-07-17

Added — AMQP: the RabbitMQ protocol

  • AMQP 0-9-1 listener (OXIDB_AMQP_PORT, off by default) — RabbitMQ client code works unmodified, verified end-to-end with pika (Python), RabbitMQ.Client (.NET) and amqp091-go (Go). Work queues with competing consumers (the semantic MQTT cannot express), default + direct/fanout/topic exchanges, Basic.Qos prefetch, publisher confirms, mandatory Basic.Return, nack/reject. Anything outside the subset is refused with a clear channel error, never silently accepted.
  • Durability follows the protocol: a durable queue holding delivery_mode=2 messages is written through the document engine's WAL — the confirm is only sent after the fsync, and messages survive a SIGKILL (crash-tested; acknowledged messages stay consumed).
  • MQTT ↔ AMQP bridge via the pre-declared amq.topic exchange, the same mapping RabbitMQ's MQTT plugin uses (/ ↔ ., QoS ≥1 ↔ persistent): a sensor publishes MQTT, a worker pool consumes AMQP, one binary.
  • Faster than RabbitMQ on 5 of 6 benchmark scenarios (same Go client both sides): pipelined confirms 1.50×, confirm latency 1.74×, end-to-end throughput 1.29×, end-to-end latency 1.52×, 8-connection durable 1.11×. Behind it: per-burst fsync batching (one insert_many per pipeline burst; 264 → 53k msg/s), a cross-connection group committer (concurrent bursts share fsync rounds), and a cross-thread wake pipe in each connection's poll(2) set (delivery latency 51 ms → 0.02 ms). The one loss — single-connection durable — is the price of a real F_FULLFSYNC behind every confirm, which RabbitMQ's lazy interval flush does not pay.

Added — engine & clients

  • Online WAL checkpointing for the document engine — the write-ahead log is sealed and folded into the snapshot while writers run, so it no longer grows unboundedly between restarts. Size-triggered via OXIDB_WAL_CHECKPOINT_BYTES (default 64 MiB, 0 restores the old behaviour); crash-tested against SIGKILL mid-checkpoint.
  • Typed time-series surface in OxiDb.Client.Tcp (.NET) — TsdbWriteAsync, TsdbWriteLineProtocolAsync and TsdbQueryAsync with typed points, aggregations (TsdbAgg.Mean … Percentile(p)) and epoch-ms helpers, replacing hand-rolled raw commands.

Fixed

  • S3 ETags are now real MD5s — the previous ETag (truncated SHA-256) broke the AWS SDK for .NET, which re-computes the MD5 of what it uploaded and refuses a disagreeing ETag. aws-cli, boto3, MinIO SDKs and the AWS .NET SDK now all verify uploads cleanly, with no workaround flags.

v0.36.0

2026-07-17

Fixed — WebAssembly is back

  • The wasm32 build is repaired — v0.35.0 shipped binary-only because it had been broken for a while. 2.2 MB raw, 0.76 MB gzipped, verified in a browser. TransactionId (a u64) lived in the native-only tx_log module, so five portable modules each duplicated it behind a cfg and the sixth broke the build; it now has one portable home. Also fixed: explain's Instant (wasm32 has no monotonic clock) and a shutdown() that assumed background threads.

Fixed — clustering

  • The bootstrap node published an empty Raft address. raft_init registered the initial member with no address while every learner got a real one. Invisible for as long as that node leads — nobody dials the leader — but once it loses leadership no new leader can ever reach it, and it silently freezes at its old log while the cluster commits without it. Bootstrapping with no address is now refused outright.
  • oxipool sent every SQL read to the master. The read/write classifier looked for the statement in a field the SQL wire shape does not have, so everything read as a write and replicas never served a SQL query.

Performance — compound predicates & string affixes

  • AND / OR now short-circuit. Both sides were always evaluated — x > 995 AND TRUE cost 52% more than x > 995 alone. Every compound WHERE in the engine gains.
  • STARTS_WITH / ENDS_WITH — exact, literal, case-sensitive affix tests that compare borrowed bytes in place. Ordinal StartsWith/EndsWith previously rendered as per-row SUBSTRING+LENGTH, because LIKE is case-insensitive and a needle containing % would become a wildcard. Faster than LIKE without giving up the semantics that ruled it out.
  • Against PostgreSQL over EF Core both shapes flipped from losses to wins: any_compound 0.79x→1.21x, string_multi 0.67x→2.07x.

Added

  • oxidb-server --version / --help — probing the binary with --version used to start a server on the default port. --help is also the only in-binary documentation of the env-var configuration.
  • Engine-aware backup & restore for the SQL and time-series engines, both low-lock: the engine lock is held only to pin a generation, and the archive compresses with it released, so queries and writes continue throughout.

Tested

  • p99 soak harness (plus a stdlib-only driver for hosts with no Rust toolchain). On Linux alongside live instances: 1.6M ops at 5,387 ops/s, read p99 3.9 ms, zero errors, no latency drift, RSS flat.
  • Partition tests for the sharded router — a missing shard must fail loudly, never return a plausible undercount — and for asymmetric network failures, where one direction dies and the two ends disagree about whether the peer is alive.

v0.35.0

2026-07-16

Added — instant, online schema changes

  • ALTER TABLE ADD COLUMN / DROP COLUMN are O(1) — metadata-only, no row rewrite, no checkpoint. Add or drop a column on a 500M-row live table with zero downtime. ADD pads old rows with the default on read; DROP tombstones the column in place and projects it out.
  • Checkpoint compaction reclaims a dropped column's space, folded into a checkpoint that rewrites every row anyway.

Changed — crash-atomic durability

  • MANIFEST + generation checkpoints — each checkpoint writes a whole new gen.<N>/ and promotes it with a single atomic MANIFEST rename. A crash before it leaves the previous generation whole; catalog and snapshot arities can never disagree after a crash. Recovery replays only WAL records past a watermark.

Added — EF Core provider & SQL performance

  • Official EF Core specification tests: 3832/3832 green across all 12 Northwind suites, with full migrations and design-time scaffolding.
  • OxiDB beats PostgreSQL on the EF Core benchmark — contiguous scan cache, correlated-subquery decorrelation, streamed scans with push-down, single-pass GROUP BY, index-nested-loop joins, a 48→24-byte Value, and an OxiWire binary wire format.
  • Analytics surface — WITH / WITH RECURSIVE CTEs, set operations (UNION/EXCEPT/INTERSECT), LATERAL joins, DISTINCT ON, mode() WITHIN GROUP, CREATE SEQUENCE / NEXT VALUE FOR, multi-level correlation, case-insensitive LIKE + COLLATE.

Added — TSDB time-series engine

  • A standalone oxidb-tsdb engine (mounted like SQL, engine: "tsdb") — Gorilla-compressed columnar streams (~0.3 bytes/point), typed fields, InfluxDB line-protocol ingest, rate()/percentile, continuous-aggregate rollups, and MANIFEST-atomic persistence. Go client included.
  • OxiDB Studio desktop app — visual Query Designer, schema tree, editable result grids (macOS build on the downloads page).

Fixed

  • Linux (musl) build: qualified std::fs in the /proc stats readers.
  • Sequences persist in a separate sequences.json so a NEXT VALUE FOR can never desync a generation's catalog from its snapshots.

v0.34.7

2026-07-09

Added — full MQTT 3.1.1 broker

  • Topic wildcards — + (one level) and # (subtree) filters, backed by the OxiMem pattern-subscriber layer.
  • Retained messages — last-known-value delivery to new subscribers; empty retained payload clears.
  • QoS — QoS 1 delivery with packet ids; inbound QoS 2 completes the PUBREC/PUBREL/PUBCOMP handshake.
  • Last Will & Testament — published on abnormal disconnect or keepalive expiry (1.5× enforced).
  • Auth — OXIDB_MQTT_USER/OXIDB_MQTT_PASSWORD require matching CONNECT credentials.
  • Wire-test suite speaking raw MQTT bytes (6 tests).

v0.34.6

2026-07-09

Added — S3 API

  • ListObjectsV2 continuation tokens — aws s3 ls pages correctly over large buckets.
  • Lifecycle expiration — ?lifecycle Days rules per bucket with a background sweeper.
  • SigV4 wire-test suite under cargo test: signed roundtrip, corrupted-signature 403, multipart assembly, batch delete.

v0.34.2 – v0.34.5

2026-07-09

Added — OxiMem becomes a full Redis-class store

  • Transactions — MULTI/EXEC/DISCARD/WATCH/UNWATCH with O(1) version-counter WATCH and Redis EXECABORT semantics.
  • Server-side scripting — EVAL/EVALSHA/SCRIPT (Lua 5.4) with KEYS/ARGV, redis.call, cjson, redis.sha1hex; atomic, busy-script time limit, SCRIPT KILL.
  • Blocking ops — BLPOP/BRPOP/BZPOPMIN/BLMOVE/BRPOPLPUSH, condvar-woken.
  • Persistence — rebuild-on-boot from the SQL mirror (all five types, TTL-correct) and fast-mode snapshots.
  • Pub/sub — PSUBSCRIBE glob patterns, keyspace notifications, expired events.
  • 30+ new commands (set ops, GETDEL/COPY/GETEX, ZREMRANGEBY*, ZUNION/ZINTERSTORE, LMPOP/ZMPOP, bit ops, sub-scans with real cursors), Prometheus command counters + latency histogram.

Fixed

  • ZREVRANGE rank order — ZREVRANGE key 0 0 returned the lowest member; ranks now index the descending view.

v0.34.0

2026-07-08

Added

  • Group commit — concurrent transactions share fsyncs; hot-account workloads went from ~130 to 300+ tx/s on a laptop, 1.5–2.2k tx/s on a 4-core Linux VPS at full durability.
  • SELECT FOR UPDATE — find_for_update pessimistic document locks: contenders queue instead of conflict-storming.
  • Time-series aggregation — $ohlcv (tick→candle), range/time window frames, $densify, $fill.
  • Prometheus — GET /metrics on the REST listener; zero dependencies.
  • explain & slow-query profiler — real planner output plus OXIDB_SLOW_QUERY_MS capture.
  • Isolation characterization — the OCC model documented and pinned by tests; Jepsen-style crash suite, fsync-failure injection, Elle-style serializability checker, Raft partition tests.

Fixed

  • Torn transaction-log writes — commit log now replaced atomically (found by the Jepsen-style suite).
  • Same-document write composition — two updates to one document in a transaction no longer clobber each other.
  • fsync-failure durability hole — a rejected commit can no longer leak into a checkpoint.

v0.29 – v0.33

2026-06 – 2026-07

Highlights

  • SQL engine (ADR-0010) — standalone relational engine beside the document engine: DDL, DML, joins, GROUP BY/HAVING, secondary indexes, parameterized queries, per-engine transactions; beats PostgreSQL 15 on the reference workload.
  • Stored procedures — CREATE/DROP PROCEDURE, CALL, named params, atomic execution.
  • EF Core & ADO.NET (ADR-0013) — OxiDb.Data (Dapper-ready) and an EF Core 9 provider; interactive transactions with savepoints.
  • Multi-database (ADR-0012) — isolated databases with per-database SQL engines, RBAC, TTL/alert threads.
  • Licensing — v0.33.0+ is proprietary (commercial licensing); TCP client libraries remain MIT.

v0.28.18

2026-05-25

Added

  • OxiWire HELLO handshake — new cmd: "hello" returns server version, supported wire versions, stable-surface feature set, experimental feature set, and auth methods. Pre-auth, idempotent, backward-compatible (clients without HELLO default to wire v1). See oxidb-server/src/hello.rs and ADR-0003 Phase 2.
  • REST /v1/ URL prefix — GET /v1/hello returns server info; /v1/api/... is the 1.0 stable surface entry point. Legacy bare /api/... still routes during the deprecation window.
  • WebSocket subprotocol versioning — server advertises oxidb.v1 via Sec-WebSocket-Protocol. Clients without the header still connect.
  • oxidb migrate CLI — new subcommand on oxidb-cli: migrate inspect --data <PATH> walks a data directory and reports each file's on-disk format version (OXWA / OXTX / OXBT / OXIX / blob format_version). migrate run validates versions and is the scaffold for future v2 migrations (ADR-0003 Phase 4).

Performance

  • Bytes-first find path for OxiWire responses — new jsonb_oxiwire module converts JSONB to OxiWire bytes via a custom serde Visitor, skipping the serde_json::Value tree intermediate (~20 µs/doc saving on cache miss). A new doc_bytes_cache (env-tunable via OXIDB_DOC_BYTES_CACHE_SIZE, default 1M) keeps pre-encoded bytes around.
  • Composite-index fast path — find queries that are exactly covered by a composite index's fields now route through find_prefix directly, skipping post-filter and Value materialisation.
  • Partial-JSONB filter — new helper evaluates top-level $eq/$ne/$gt/$gte/$lt/$lte/$in conditions plus $and / $or / dot-paths directly against JSONB bytes using codec::extract_field. Wired into the aggregation pipeline's $match step AND the find full-scan rayon path; reserves the full JSONB→Value decode for queries with predicates the partial matcher can't evaluate.
  • Doc cache capacity is env-tunable — OXIDB_DOC_CACHE_SIZE overrides the 100K default. Production hardware with more RAM can hold the full working set.

Benchmark

  • OxiDB sweeps MongoDB at 1M docs. The full tests/comparison-mongodb bench at 1M-document scale (in-network Docker harness, no port-forward artifact) goes OxiDB 24 – MongoDB 0 across 24 measured workloads. Largest wins: count-all 2189×, Top-5 cities aggregation 1262×, composite-indexed compound 4.1×. Smallest wins: bulk insert 1.1×, range-10K-rows-each 1.2×. Resource footprint at peak: OxiDB 1.71 GiB RSS / 741 MB disk vs MongoDB 1.00 GiB / 626 MB.

.NET clients (developer-friendly rework)

  • 4 NuGet packages published at v0.28.18 — OxiDb.Client.Tcp, OxiDb.Client.Embedded, OxiDb.EntityFrameworkCore, and NEW: OxiDb.Linq (LINQ provider, previously source-only).
  • Exception hierarchy — OxiDbException base + OxiDbDuplicateKeyException, OxiDbTransactionConflictException, OxiDbAuthenticationException, OxiDbNotFoundException, OxiDbImmutableException (WORM), OxiDbConnectionException, OxiDbProtocolException. Server error strings routed to the right subclass via FromServerMessage. Legacy OxiDbTcpException retained as [Obsolete] alias.
  • HelloAsync + HelloResponse record — wire-protocol handshake returning server version, supported wire versions, stable + experimental feature sets, auth methods.
  • Typed CRUD overloads — FindAsync<T>, FindOneAsync<T>, InsertReturningIdAsync (returns long), InsertManyReturningIdsAsync (returns long[]). Eliminate the JsonElement→parse dance.
  • StreamAsync<T> — IAsyncEnumerable<T> over paginated LIMIT/SKIP batches for million-row result sets.
  • DI integration — services.AddOxiDbTcp(opts => opts.Host(…)) registers IOxiDbClient as a singleton.
  • Type-safe query builder — Query.Eq, Query.Gte, Query.In, Query.And, Query.Or, Query.Range … for runtime-constructed queries that don't fit LINQ.

1.0 prep docs

  • docs/SEMVER.md, docs/STABILITY.md, docs/DEPRECATION.md, docs/SECURITY.md — Phase 5 of ADR-0003. Translate the ADR-0004 release-policy decisions into operational docs (24-month LTS, additive-only minor releases, GitHub Security Advisories channel, etc.).
  • docs/PHASE3-SDK-FREEZE.md + Python client api/v1.json snapshot + CI gate script (template for the other 9 Tier-A clients).
  • docs/format/compat-matrix.md — Phase 2 cross-version compat matrix (OxiWire / REST / WebSocket).

v0.28.12

2026-05-24

Added

  • Audit log rotation — RotationPolicy in oxidb-server/src/audit.rs supports size-based (OXIDB_AUDIT_MAX_BYTES), age-based (OXIDB_AUDIT_MAX_AGE_SECS), and calendar-aligned UTC rotation (OXIDB_AUDIT_CALENDAR=hourly|daily), with optional gzip compression of rotated files (OXIDB_AUDIT_COMPRESS=true). Wired into both standalone and cluster modes.
  • CERN-grade testing program — 9 cargo-fuzz targets (RESP, pg_wire, OxiWire, MsgPack, differential vs Redis & Postgres), OSS-Fuzz integration scaffolding, coverage reporting, ACID isolation-anomaly suite, HEP-shaped scale workload, encrypted-backup DR drill, upgrade-chain fixture corpus, and 39 authn/authz/SCRAM/canonicalisation/audit attack patterns — all rejected by the server.
  • Format version headers — OXTX for _tx_commit_log, OXWA for .wal, OXBT for .btree; explicit format_version in blob .meta JSON. Establishes the 1.0 on-disk-format contract.

Fixed

  • Unauthenticated DoS bugs found by fuzzing — RESP multi-byte UTF-8 line splitter panic, RESP CR-truncation + allocator-bomb, pg_wire message length unbounded allocation (now capped at 16 MiB), pg_wire i16-overflow + empty-body panic, OxiWire array/map pre-allocation now bounded by remaining bytes. Server versions < 0.28.3 vulnerable.

Changed

  • Julia client surface — find / aggregate now return a Tables.jl-compatible row collection (DataFrames, CSV, MLJ, GLM accept it directly). SQL exports removed from Julia clients — OxiDB is a document database; Tables.jl covers the data-frame integration story.

v0.25.3

2026-04-25

Changed

  • Raft persistence: O(1) per mutation — rewrote oxidb-server/src/raft/log_store.rs to split state into a small raft_meta.json (vote / committed / sm_data) and an append-only raft_log.jsonl (one entry per line).
  • Append-only log writes — append_to_log is now a single line append per entry instead of rewriting the entire log; delete_conflict_logs_since and purge_logs_upto rewrite only on those rare events.
  • Transparent migration from the v0.25.2 single-file raft_state.json on first boot.
  • Unblocked 1M-record load tests under failover — 22.4 s end-to-end, 44,701 rec/s avg, zero records lost. The previous single-file snapshot stalled the cluster at ~52% complete due to 14 MB-per-mutation rewrites.

v0.25.2

2026-04-25

Added

  • Persistent Raft state for cluster mode — OxiDbStore in oxidb-server/src/raft/log_store.rs was previously in-memory only; nodes that restarted came back as Learner term=0 and lost cluster membership, breaking failover scenarios.
  • New OxiDbStore::open(db, &data_dir) constructor — loads existing Raft state on startup; OxiDbStore::new(db) retained as in-memory variant for tests.
  • Atomic write-through on every mutation — save_vote, save_committed, append_to_log, delete_conflict_logs_since, purge_logs_upto, apply_to_state_machine, install_snapshot.
  • ShardReplicaRealWorldTest harness — 14-service docker-compose: 9 oxidb-server nodes (3 Raft groups), 3 per-shard oxipool master/replica routers, 1 top-tier shard-routing oxipool, Go API tier, one-shot cluster-init bootstrapper.
  • End-to-end test suites — Go smoke harness (5 assertions), Python integration tests (8 cases: CRUD + sharding + aggregation), Python failover scenarios (5: network partition, follower down, recovery catch-up, two followers down, leader down), parameterized load test with mid-stream failover (validated against 10K, 100K, 1M record loads).

v0.25.1

2026-04-18

Added

  • Eight new query operators — $not, $nor, $all, $size, $type, $mod, $expr, $elemMatch.
  • $not field operator — negate any field condition; missing fields evaluate to true (MongoDB-compatible).
  • $nor top-level operator — match documents where none of the listed conditions are true.
  • $all array operator — array must contain all specified values.
  • $size operator — match arrays with an exact length.
  • $type operator — match by JSON type (string, number, bool, array, object, null, int).
  • $mod operator — modulo arithmetic on numeric fields ([divisor, remainder]).
  • $expr top-level operator — cross-field comparisons, e.g. {"$expr": {"$gt": ["$sold", "$stock"]}}.
  • $elemMatch operator — match array elements against sub-queries with AND semantics.
  • Go client additions — stored procedures (CreateProcedure, CallProcedure, ListProcedures...), CreateTTLIndex, retention policies, alerting methods, ExtractText, Backup/Restore, SetDialect.

Fixed

  • Array dot-notation in $set / $inc / $unset — variants.0.stock no longer corrupts arrays.

Changed

  • Refactored matches_doc and matches_value into a shared eval_field_op helper.

v0.24.0

2026-04-10

Added

  • WebAssembly support -- New oxidb-wasm crate compiles OxiDB to wasm32 and runs entirely in the browser.
  • In-memory browser mode -- No server needed. JSON queries, SQL, and aggregation all work client-side in the browser.
  • wasm-bindgen API -- Full JavaScript API surface: init, insert, find, update, delete, count, sql, aggregate.
  • ~1.5 MB gzipped WASM binary -- Compact binary size suitable for production web applications.
  • TypeScript types included -- Full type definitions shipped with the WASM package for editor autocompletion and type safety.
  • Cross-platform lock shim (src/locks.rs) -- Uses parking_lot on native targets, spin locks on wasm32.

Changed

  • Native-only dependencies made target-specific -- rayon, memmap2, zstd, and other native-only crates moved to target-specific dependencies to enable WASM compilation.
  • #[cfg(not(target_arch = "wasm32"))] guards throughout core engine -- Platform-incompatible code paths conditionally compiled out for the WASM target.

v0.18.0

2026-03-05

Added

  • OxiWire binary protocol -- Custom wire format with 1-byte type tags, 4-byte LE lengths, 8-byte LE numbers. Magic byte 0xDB. Replaces MsgPack for all request/response paths. Encoder + decoder in Rust and Go
  • .NET EF Core provider -- Full Entity Framework Core support with LINQ queries, transactions, and both TCP and embedded modes. d7d5a05
  • .NET NuGet packages -- OxiDb.Client.Tcp, OxiDb.Client.Embedded, OxiDb.EntityFrameworkCore. d7d5a05
  • Composite index tests -- 9 subtests covering exact match, prefix match, count, sort, update, delete, aggregate, drop, and triple-field composite indexes in Go.
  • Parallel OxiWire serialization -- Result sets >= 5,000 docs are serialized across up to 8 CPU cores. Chunk-based, zero per-doc allocation.
  • OxiDB vs MongoDB benchmark suite -- 22 tests across 7 categories. Score: OxiDB 19 -- MongoDB 1.
  • OxiDB vs PostgreSQL benchmark suite -- 20 tests comparing document workloads. Score: OxiDB 10 -- PostgreSQL 10. d7d5a05
  • OxiDB vs SQLite benchmark -- 100K document embedded benchmark. ce8db5f

Changed

  • Aggregation indexed-path threshold -- Changed from 10% to 50% selectivity. Indexed aggregation path now preferred when candidate set is less than 50% of collection size.
  • Go client rewritten for OxiWire -- All requests/responses use OxiWire binary format. MsgPack dependency removed entirely.
  • Pipeline handler updated -- Sub-responses decoded from OxiWire and re-encoded for composite pipeline responses.

Removed

  • MsgPack support -- Removed from server (Rust), Go client, and all benchmark tests. OxiWire is the sole binary protocol.
  • github.com/vmihailenco/msgpack/v5 -- Removed from Go module dependencies.

v0.17.0

2026-02-23

Added

  • LRU document cache -- Per-collection in-memory cache with configurable capacity. JSON deserialized once, then Arc-refcounted. 0997d8e
  • Streaming scan for non-indexed finds -- Avoids loading all documents into memory for large unindexed queries. 39cd803
  • Lock-free pread -- Separate read-only file handle uses pread for concurrent reads without locking the write path. 63093a3
  • Sorted-offset batch reads -- Indexed finds sort offsets before reading to minimize disk seeks. 63093a3
  • Zero-decode aggregation -- Extract only needed fields from raw JSONB, skip full document deserialization. 4a6f696
  • Batch pread for indexed $match aggregations -- Combine pread with zero-decode for indexed aggregation paths. 4b610ea
  • Zero-decode index creation -- Extract only _id and the indexed field from raw JSONB during index build. beb3f49
  • DocIdSet optimization -- Inline storage for single-document index entries saves ~80 bytes per entry. 4897f86
  • Zero-decode filter for unindexed scans -- JSONB keypath extraction avoids full JSON parse on scan. 7b9c639
  • Parallel segmented scan -- Large unindexed queries split across CPU cores for parallel processing. b339e5a
  • Index-only count for aggregations -- $group with $sum: 1 on indexed fields returns set size without touching documents. b339e5a

Changed

  • Memory consumption reduced -- Skip bulk cache during insert, drop unused Value clones, use DocIdSet instead of BTreeSet for single-entry indexes. 4897f86
  • Streaming I/O throughout -- Replaced collect-then-process patterns with streaming iterators for finds, aggregations, and index creation.

v0.16.0

2026-02-23

Added

  • Core document database engine -- Append-only storage, WAL with CRC32 checksums, per-collection locking.
  • JSON query language -- $eq, $ne, $gt, $gte, $lt, $lte, $in, $exists, $regex, $and, $or. Dot notation for nested fields.
  • Update operators -- $set, $unset, $inc, $mul, $min, $max, $rename, $currentDate, $push, $pull, $addToSet, $pop.
  • Aggregation pipeline -- $match, $group, $sort, $project, $limit, $skip, $unwind, $addFields, $lookup, $count. Accumulators: $sum, $avg, $min, $max, $count, $first, $last, $push.
  • Single-field, unique, and composite indexes -- BTreeMap-backed with index-only count and index-backed sort.
  • Full-text search -- TF-IDF ranking with HTML, XML, JSON, PDF, DOCX, XLSX, and OCR support.
  • Vector search -- HNSW index with cosine, euclidean, and dot product distance metrics.
  • ACID transactions -- OCC with 3-phase commit, per-document versioning, deadlock-free sorted locking.
  • SQL support -- SELECT, INSERT, UPDATE, DELETE, CREATE/DROP INDEX, CREATE/DROP TABLE, JOINs, GROUP BY, aggregate functions.
  • Blob storage -- S3-style bucket/object API with metadata, ETags, content types.
  • Encryption at rest -- AES-256-GCM with random 12-byte nonce per document.
  • Zstd compression -- Level 3, transparent per-document, thread-local context reuse.
  • Change streams -- Watch collections for insert/update/delete events. Resumable with 4096-event replay buffer.
  • Stored procedures -- Named multi-step operations with parameter substitution.
  • Scheduled tasks -- Background job scheduling with enable/disable control.
  • Multi-database support -- Isolated databases within a single server instance.
  • Backup & restore -- Compressed full backups with all data, indexes, and metadata.
  • TCP server -- Length-prefixed JSON over TCP (max 16 MiB). Tokio-based async runtime.
  • SCRAM-SHA-256 authentication -- Salted challenge-response, no plaintext passwords on wire.
  • RBAC -- Admin, ReadWrite, Read roles with per-command authorization.
  • TLS/SSL -- Certificate-based encryption for all traffic.
  • Audit logging -- GELF format for centralized logging.
  • Raft clustering -- Multi-node replication via openraft (optional cluster feature flag).
  • Client libraries -- Python, Go, Julia, .NET (TCP + Embedded), Swift (C FFI).
  • C FFI -- oxidb-client-ffi (cdylib) and oxidb-embedded-ffi (staticlib + cdylib) for language bindings.
Report Issue