Messaging Layer

[Nintorac]

Hi,

What are the team's thoughts on https://zeromq.org/?

"The Guide" is a really great resource https://zguide.zeromq.org/

[yanchengnv]

NVFLARE's design position is "all inclusive". Today our system is hard-wired with gRPC, but that will change. NVFLARE will be based on "specs" (the so-called componentized architecture where each type component has a spec). We will make "messaging layer" spec-based such that anyone can implement their favorite messaging layer based on the spec, and just plug it in.

In general, NVFLARE wants to remain neutral, and leave the decisions to customers; because NVFLARE is ultimately to be used, deployed, and operated by customers.

[Nintorac]

That's great, is there any documentation around this I could see?

Specifically I would be interested to see the minimal set of interfaces that are needed to implement nvflare and perhaps some architecture diagrams around control flow if there is anything?

[yanchengnv]

For NVFLARE 2.0, here are the APIs that you can use/customize:

https://github.com/NVIDIA/NVFlare/tree/main/nvflare/app_common/abstract
https://github.com/NVIDIA/NVFlare/tree/main/nvflare/apis

In NVFLARE 2.1, we'll introduce two important features:
High Availability - automatically cut over to a backup server when the FL server crashes;
Multi Job - run multiple jobs concurrently based on resource availability.

As you might imagine, in both cases, there are customer-specific decisions: where to persist the state data (NFS, S3, SQL, ...)? What exactly are resources and how to make resource-based decision whether a job can be run on a client?

There are other examples of such dilemmas. Our strategy:

• Define an abstraction of the behavior (the spec)
• Create some implementations following the spec
• Encourage customers to implement their own solutions following the same spec
• NVFLARE uses the components purely based on specs

Here is an example of specs for HA: https://docs.google.com/document/d/1KzSmwFpcEqnfpjtTHtFe6MmEmhdw2rdvuJwGH42VmPc/edit#

BTW, at NVIDIA GTC (https://www.nvidia.com/gtc/), there will be a dedicated session for NVFLARE. You are encourage to attend.

[Nintorac]

Thanks this helped, I did a deep dive into the nvflare/apis and untangled some things that have been confusing me.

As I see it there are several different applications all combined into this folder.

I will try to explain what I mean from my current understanding. I will try and relate the various API's to components of the http stack that perform similar functions. I'm not sure how much I have correct here so please correct anywhere that is wrong.

First there are the core components of nvflare. This is DXO, FLComponent and FLContext, the latter two are slightly misnamed as they are not components required for FL, but are base components of a Federated Compute environment. eg. the system should be more general in that it can perform Analysis, Inference, batch jobs..etc. This relates to the protocol level, eg TCP, UDP

Next we have a shared components layer. This includes things like Responder, Shareable, Task, Signal and aux_spec.py. This defines a shared set of concepts on which you can build a Client/Server architecture. This relates to CRUD/GraphQL layer of the HTTP stack

Next you have the client/server layer. Here there are two applications being developed. The client and the server. I will consider them separately.

• The client act's like a server in the HTTP stack would, it has a specific set of skills that it can perform and requests are made against it to make it perform those functions. In the HTTP stack this would be server libraries like Flask or Django. This includes the components ClientEngineSpec, ClientTask

• The server acts like a client in the HTTP stack. It makes requests against a set of servers and then does some processing on the results of all the requests. This is like a web browser in the HTTP stack. This includes ServerEngineSpec -- maybe Workspace and Run's come in here but I think not really. The server is just to chain a bunch of requests together in a meaningful way.

The rest is application layer, an example application would be Federated Learning, in a federated learning app you want to be able to manage experiments, configuration and models in a consistent way.

BTW, at NVIDIA GTC (https://www.nvidia.com/gtc/), there will be a dedicated session for NVFLARE. You are encourage to attend.

Cool I have signed up

[Nintorac]

@yanchengnv Does this match your understanding? Is there anything I am misunderstanding?

[YuanTingHsieh]

@Nintorac I think you are right on your understanding.

We introduced a new concept of Job and Job scheduling in NVFlare 2.1.

The current client/server layer works like the following:

When a job is scheduled to run, the app folders inside it will be deployed to server and clients (based on its configurations).
Then we will start the application on each site.
On the start time, each site will spawn a child process ("runner_process/worker_process") that is specific for this job.
And then these child processes will be responsible of executing logic for this job.
This parent-child process separation allows us to clean up (delete the process) the job more easily.