weida

weida is a messaging framework for Rust built from first principles around QUIC. Every user data flow maps to one QUIC stream, so a 40-byte transfer and a 40-GB transfer use the same protocol semantics and neither requires the payload to be materialized in memory.

It is built in layers. L0 is a stream core: ZeroMQ's idea rebuilt on QUIC, whose primitives are unidirectional and bidirectional streams with exactly the guarantees QUIC gives — ordered bytes within a stream, none across streams, flow control, a transport delivery receipt, and cancellation by reset. L1 is the ZeroMQ/nanomsg pattern family as thin wrappers over L0: Req/Rep, Push/Pull, Pub/Sub, PAIR, SURVEY and BUS — the whole nanomsg set, and none of them adds wire vocabulary. L2 is a RabbitMQ-analog broker, weida-broker: queues at endpoint paths, publisher confirms, and an absolute per-subscription credit. Consumer acknowledgement and redelivery are the slice still to come.

A completion is a cursor, not a verdict: a level plus an absolute byte offset, reported on a unidirectional stream of its own that never shares a stream with payload. So a Push producer gets a reliable Accepted without an exchange, a reader learns how far the far end got rather than only whether it finished, and no pattern changes shape to gain any of it.

Status: alpha. Published on crates.io: weida, weida-core, weida-protocol, weida-runtime and weida-winpipe at 0.1.0-alpha.1 and 0.1.0-alpha.2, and weida-broker, weida-raft and weida-openbao at 0.1.0-alpha.2. The other seventeen members are not published: weida-py-core, every Python binding and all five protocol libraries with their codecs. Nothing is on PyPI and there are no prebuilt binaries. Wire protocol version 0 is experimental and breaking changes are permitted within 0.x. Phases 0-3 are implemented — the docs, the core model, the native QUIC transport, the in-process, AF_UNIX and named-pipe transports, all six patterns, the raw L0 stream API, peer identity by public-key fingerprint, opt-in per-producer ordering and bounded deduplication, a bounded drain, the cursor back channel, a synchronous facade (weida::blocking), a weida binary and a Python binding (weida-py) — and the first two broker slices with them. Beside weida the repository ships five protocol libraries, none of which has weida in its picture: ZeroMQ (weida-zmtp + weida-zmq + weida-zmq-py), the nanomsg SP protocols (weida-sp + weida-nng + weida-nng-py), MQTT 5 (weida-mqtt-codec + weida-mqtt + weida-mqtt-py), AMQP 1.0 (weida-amqp-codec + weida-amqp + weida-amqp-py) and Core NATS (weida-nats-codec + weida-nats + weida-nats-py), each with a parity document under docs/libraries/ and each measured against a real peer where one exists. weida-zmq is complete against decisions/0013 §4.7's definition of first-class — every socket type of zmq_socket(3) bar ZMQ_STREAM, tcp/ipc/inproc, NULL/PLAIN/CURVE with ZAP, the option table honoured or refused row by row, the monitor and the devices, the zguide's canonical recipes as examples that assert the guide's own claims, and interop against libzmq 4.3.5 and the pure-Rust zeromq crate in both roles (docs/libraries/zmq.md). Its typed proxy and proxy_steerable devices include the canonical ROUTER/DEALER queue, XSUB/XPUB forwarder and PULL/PUSH streamer without involving the weida protocol.

Documents

Start with the guide. The rest of this table specifies; docs/GUIDE.md teaches, in the shape ZeroMQ's zguide established — every program in it is a file in this repository and a test asserts the claim the text makes. It is organised around one question, C8B: how do you scale a software system to eight billion people?, which is this project's C10K: utopian on purpose, and useful because the arithmetic settles structural questions before anything is built (decisions/0026).

Crates

This repository holds two kinds of product: weida itself, and standalone implementations of foreign protocols that are usable with no weida in the picture, with a Python binding beside each finished library. A user may compose those public APIs explicitly. The workspace does not define a general translation from foreign socket types onto weida patterns. The kind column says which product a crate belongs to (decisions/0013 §4). The table lists every one of the twenty-five [workspace] members and is read off cargo metadata, not kept by hand.

Quick start, without writing a program

$ cargo run -q -p weida --bin weida -- serve --echo 'weida://127.0.0.1:7443/echo'
weida://sha256:22ed30a8…9f25@127.0.0.1:7443/echo

$ printf 'hello weida' | cargo run -q -p weida --bin weida -- request 'weida://sha256:22ed30a8…9f25@127.0.0.1:7443/echo'
hello weida

In a program with no reactor, the same shape is a library call: weida::blocking behind the non-default blocking feature wraps every endpoint, one block_on each, over a reactor it owns — and refuses to be called from inside a Tokio runtime, where blocking a worker would deadlock.

Four verbs: serve (with --echo, --sink or --pub), request, send and sub. The payload is on stdin and stdout and nothing else is, so weida request … | sha256sum means what it looks like; addresses, receipts and topics go to stderr. The address serve printed is the whole client configuration, exactly as in the section below. Every transport works the same way — weida+unix://%2Ftmp%2Fs.sock/echo for a socket, weida+pipe://NAME/echo for a Windows pipe — and the exit code names the error a script can act on: 3 refused, 4 unknown endpoint, 5 no reply, 6 untrusted, 7 connection lost. weida --help lists every option, including sub --framing line|raw|nul|length, because a newline after each payload is readable and is not framing.

Identity in one line

A weida peer is its public key. The server generates an identity and prints an address that names it; the client trusts that address and nothing else:

$ cargo run -p weida --example transform_server -- --bind 127.0.0.1:7443
weida://sha256:22ed30a8…9f25@127.0.0.1:7443/transform
weida://sha256:22ed30a8…9f25@127.0.0.1:7443/echo

$ printf 'hello weida' | cargo run -p weida --example transform_client -- 'weida://sha256:22ed30a8…9f25@127.0.0.1:7443/transform'
HELLO WEIDA

No certificate file changes hands, no CA exists, and a peer with any other key is refused with Untrusted(sha256:…) naming the key that answered. --identity PATH keeps the server's key across restarts so the address stays stable; --cert-out PATH plus transform_client --ca PATH is the same exchange through a trusted certificate and a plain address instead. In code: Identity::generate(), Trust::by_address() / Trust::pin(fp) / Trust::anchor(pem), ServerTls::require_client(trust) for mutual identity, and IncomingMeta::peer to see who sent what.

Expected on stderr: delivered — QUIC's transport receipt for the request, not an application acknowledgement — and a trace id that also appears in the server's log line for the request.

Other patterns

Push/Pull and Pub/Sub each have a self-contained example: both halves run in one process on loopback, so there is nothing to configure.

cargo run -p weida --example push_pull   # fire-and-forget, and one transport receipt
cargo run -p weida --example pub_sub     # prefix-filtered topics, two subscribers

Messages arrive out of order in both. Each transfer is its own QUIC stream, so ordering is None for these patterns — see docs/PATTERNS.md §1.7.

No reactor of your own

weida does not require you to be inside #[tokio::main]. quinn needs a Tokio reactor and nothing else in the library does, so Runtime::owned creates and owns one — one worker thread by default — and hands back ordinary futures that any executor can drive. Transfers implement both the tokio::io and the futures-io trait pairs for the same reason.

cargo run -p weida --example owned_runtime   # a full exchange from a plain `fn main`

Runtime::new takes the ambient reactor and fails with Error::Runtime when there is none; Runtime::with_handle takes somebody else's. See docs/ARCHITECTURE.md §5.

Build and test

cargo test --workspace
cargo test -p weida --test streams               # QUIC stream mechanics, measured
cargo test -p weida --test transports            # one pattern suite over QUIC, inproc and AF_UNIX
cargo test -p weida --test identity              # pins, anchors, addresses, client identity
cargo test -p weida --test large -- --ignored     # 1 GiB echo, asserts bounded peak RSS
cargo test -p weida-zmtp                         # ZMTP golden vectors and hostile input
cargo test -p weida-zmq                         # native ZeroMQ sockets, devices and interop
cargo bench                                      # codec and loopback QUIC throughput
cargo test -p weida-nng                         # native SP sockets and protocol semantics

The website

weida.doodleshnookie.net is this document set, rendered. It is built in a separate repository that imports README.md and docs/ at build time and changes nothing in them but links and code-fence labels, so every sentence on a reference page is a sentence in this tree; the landing page is the only page written there, and it quotes this file, cargo metadata and crates.io (decisions/0033). The loop's bookkeeping, third parties' requirements, the research sheets and the master plan are not published there; links to them lead into this repository.

License

Dual-licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
• MIT License (LICENSE-MIT or https://opensource.org/licenses/MIT)

at your option. This is the Rust ecosystem's own form: the MIT arm is the permissive minimum, and the Apache-2.0 arm carries an explicit patent grant.

Unless you state otherwise, any contribution intentionally submitted for inclusion in this work, as defined in the Apache-2.0 license, shall be dual-licensed as above, with no additional terms or conditions.