(Rust) Async state management, Multiplex p2p Networking, gRPC, Websockets

Instead of my current approach of setting up the network using a set of rt::spawn threads, as well as in using the current actix-web ecosystem, I should consider setting up a Multiplex system that will allow communication via gRPC (setting up Protobuf) as well as Websockets and an HTTP(S) REST API.

Ultimately, I’ll be rewriting my monolithic application to remove the templated code I pulled from the libp2p examples so that I can structure the state management within it more appropriately (since that’s what triggered this whole discussion). I’ll also be switching to Axum instead of using Actix as it’s closer to the Tokio ecosystem, and I don’t see any real downsides for it, especially since I have some really good examples of how to structure it.

For handling the additional async threads, I’ll be switching to an architecture that creates the threads and pins them to be used in the tokio::select! macro rather than blindly spawning them, as is implemented in the websocket handler for disoxide. This should make the state management a lot more stable across the different threads. Something to note here is that this websocket handler is passing in a global state that contains the state given from the jedi temple, which seems to be including the redis pool as well as the mpsc/pubsub declarations and the JoinHandle for the tasks spawned from this thread. So it does have nested multithreading which is going all the way to the top and then shared with the Arc<Mutex<_>> which is nice.

So… a lot of work to do, but having any path forward is better than mindlessly grasping for stability.

Reference material:

• Multiplex Service example with REST and gRPC
• KBVE’s Jedi Rust Package that serves as an example on better async multithreaded state management. Created/used in disoxide which is an Axum-based API.
• ”Askama” page from KBVE’s astro package which gets rewritten by Axum. The rawHTMLString is rewritten on the call to basically act like server side rendering, then it’s cached on the frontend to reduce API calls.
• Axum-Connect crate that brings the protobuf-based “Connect-Web RPC framework” (linked site down as of writing) to Rust via Axum
• Tungstenite crate for websockets in Rust
• Protobufs for KBVE’s Jedi

Additional Misc. Reference Material:

• tonic-web-wasm-client for websockets in a wasm architecture mentioned when I talked about setting up my features as modular wasms instead of as part of the monolithic application as different crates.