endo daemon with rspace / rhovm storage

[dckc]

initial work on pet-named workers, storage, and values to the Endo Daemon and expose them with its CLI
endojs/endo#1413

w.r.t. storage... is it just bytes?
I wonder about rspace / rhovm integration... not all of rholang, but just the 3-opcode rhoVM

also seems related to the way I'd like to see SOLID pods evolve:

I wonder what would happen if we mixed the Agoric platform's ability to scale down to clusters and single machines with the notion of a SOLID pod. -- https://www.madmode.com/2021/next-gig-agoric.html

cc @kriskowal @jimscarver @timbl

[kriskowal]

For Endo-as-Docker-but-JavaScript-but-actually-confined, in the first cut, I’m raking my cues from early Git. .git/refs are pet names. .git/objects is a merkelized CAS. So, for Endo, I’m persisting blobs under $XDG_STATE_HOME/endo/sha-512/$SHA512, storing refs under …/pet-name/$PET_NAME.json, and derivation paths for live, restorable values under …/value-uuid/$VALUE_UUID.json, restorable worker stuff under …/worker-uuid/…, specifically restorable capabilities granted to workers under …/worker-uuid/$WORKER_UUID/pet-names/$PET_NAME.json. That last bit is analogous to Agoric’s durable objects, but in ephemeral execution contexts that are not replayed from a full message transcript. My intent is to stand that model on top of this one in the fullness of time. I also expect, that if Endo ends up suffering the Good Problem, it will have to trace Git‘s footsteps and ”pack” this files-and-bytes database into something faster.

…

On Wed, Dec 21, 2022 at 8:12 AM Dan Connolly ***@***.***> wrote: initial work on pet-named workers, storage, and values to the Endo Daemon and expose them with its CLI endojs/endo#1413 <endojs/endo#1413> w.r.t. storage... is it just bytes? I wonder about rspace / rhovm integration... not all of rholang, but just the 3-opcode rhoVM (IOU pointer to boulder talk) also seems related to the way I'd like to see SOLID pods evolve: I wonder what would happen if we mixed the Agoric platform's ability to scale down to clusters and single machines with the notion of a SOLID pod. -- https://www.madmode.com/2021/next-gig-agoric.html cc @kriskowal <https://github.com/kriskowal> @jimscarver <https://github.com/jimscarver> @timbl <https://github.com/timbl> — Reply to this email directly, view it on GitHub <#35>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAOXBVB5SEPMRVMJ6R4CS3WOMT6HANCNFSM6AAAAAATFZJIFM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dckc]

@leithaus 's Boulder 2018 talk, to wit:

• Day 4 - RChain VM - Greg Meredith & VM Developer Team - YouTube

cited from: my notes on RChain Devcon Boulder

which is just a couple hops from:

I'm beginning to see reflection (the R in RChain) as a possible solution to the quoting and diagonalization issues TimBL and I struggled with in the Semantic Web. -- MadMode: dreaming big at the RChain Governance Forum

[dckc]

For Endo-as-Docker-but-JavaScript-but-actually-confined, in the first cut, I’m raking my cues from early Git. .git/refs are pet names. .git/objects is a merkelized CAS.

you can certainly do worse than .git ... fully groking merkelized stores is an achievement I have not yet unlocked, but I've been playing with bup lately, and it seems like an answer to much of my angst about storage. Plus, .git stores seem to make their way into all sorts of places like keybase, s3, etc.

[dckc]

endo store BYTES petname is like rholang new petname in { petname!(BYTES) }, which corresponds to a rhoVM put of BYTES at some new key, and then binding petname to that key.

[dckc]

Now how do endo spawn and endo eval relate to rhoVM? Let's see...

The rhoVM get (aka consume) instruction includes a key and a continuation. If there's a value waiting at the key, a new process is added to the process pool in which the continuation is applied to the value.

If not, the continuation goes into the store at that key, standing by...

A wrinkle: get and put (aka consume and produce) are duals: if, when we put x at k, there's already a continuation f waiting at k, that also adds a process to the pool running f(x)

In rholang, get shows up as a for comprehension: for (@petname <- key) { ...petname... }. Another wrinkle: the @petname there is a destructuring pattern. If I skip the @ there and write for (x <- key) { ... x ... }, x is bound to a quote of the value. If I want the value back, I write *x aka "evaluate x". (In rholang, the evaluate operator is applied to something closer to an AST than to the source text string. I suppose it's largely irrelevant, except that it makes for more powerful matching / destructuring in rholang.)

So endo spawn w1 seems pretty close to new w1 in { for (expr <- w1) { *expr } }.
And endo eval w1 expr seems pretty close to w1 ! (expr).

To do endo eval w1 expr --name b, we need to get the results of *expr back:

new ch in {
  for (@b <- ch) {
    w1 ! (expr, *ch)
  }
}

and refine endo spawn w1 to support that: new w1 in { for (expr, ret <- w1) { ret!(*expr) } }

Also, we should use the <<- persistent receive operator, since otherwise the for comprehension is a one-shot (goes away after firing), and endo workers are more like persistent receives:

new w1 in { for (expr, ret <<- w1) { ret!(*expr) } }

[dckc]

hm... running in cloudflare workers?

Do the storage APIs line up?

[leithaus]

Dear Dan, i have expanded the RSpace verbs. There are a total of 16 verbs based on 4 bits. - bit 1 - produce / consume - bit 2 - ephemeral / persistent - bit 3 - transacted / in-memory - bit 4 - client is sequential / client is concurrent These 4 verbs cover all proposed models of data access -- especially all those in production systems (in classical compute settings). In particular, they allow for an optimized compilation strategy for rholang 1.1. For example, the let construct insists that the client access is sequential, and the syntax is the annotation that lets the compiler know that this guarantee is in place. The enhanced for-comprehension and output syntax provide the information necessary to determine what exactly is transacted (made durable by recording to store that persists even after the rholang interpreter process is gone) versus what is in-memory only and doesn't have to go to store (only lasts as long as the rholang process). Love to all Beings,

…

--greg

On Sat, Jan 28, 2023 at 10:45 AM Dan Connolly ***@***.***> wrote: hm... running in cloudflare workers? Do the storage APIs line up? — Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAASW2WB4OHY7RVXNFYGMA3WUVSNDANCNFSM6AAAAAATFZJIFM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- L.G. Meredith 9336 California Ave SW Seattle, WA 98136 +1 206.650.3740 r <http://biosimilarity.blogspot.com>chain.coop <http://rchain.coop>

[dckc]

hm... running in cloudflare workers?

@zarutian has some related work: durableVats.mjs and such.