Large data transfer

(Deprecated) Large data transfer, streaming, rsh, tunneling

Tunneling

Tunneling is implemented as responses with same request id. cid field must be filled in correctly to enable broker to deliver responses to clients on both tunnel ends.

Basically, each RpcRequest can be answered with the CreateTunnelRequest, if called method should open tunnel. Caller can acknowledge this request by replay with CreateTunnelResponse and tunnel is created.

Response with CloseTunnel set will close tunnel.

Clients can send arbitrary data though tunnel, SHV doesn't need to understand to it.

Tunnel life cycle

Tunnel has following states TunnelCtl::State:

1. FindTunnelRequest
2. FindTunnelResponse
3. CreateTunnelRequest
4. CreateTunnelResponse
5. CloseTunnel

Complex example

Here is journal of tunneling where:

1. shvrsh asks shvagent to open tunneled remote shell session
2. shvagent asks shvbroker to find tunelling broker with public IP
3. shvagent exec shvrexec
4. shvrexec creates tunnel with IP and secret found by shvagent
5. shvrsh acknowledges create tunnel response
6. shvrexec sends bash output to the tunnel as RpcResponse with correct cid
7. shvrsh sends bash input to the tunnel as RpcResponse with correct cid

• client 1: shvagent
• client 2: shvrsh
• client 3: shvrexec

shvagent login

1 <== <T:RpcMessage,id:1,method:"hello">i{}                                                                                                       
1 ==> <T:RpcMessage,id:1>i{result:{"nonce":"1437608631"}}                                                                                         
1 <== <T:RpcMessage,id:2,method:"login">i{params:{"login":{"password":"73f422e5fb395a0ad8af042402e835b73d9218a7","type":"SHA1","user":"iot"},"opti ons":{"device":{"mountPoint":"test/agent"},"idleWatchDogTimeOut":180}}}
1 ==> <T:RpcMessage,id:2>i{result:{"clientId":1}}

shvrsh login

2 <== <T:RpcMessage,id:1,method:"hello">i{}                                                                                                       
2 ==> <T:RpcMessage,id:1>i{result:{"nonce":"1336980497"}}                                                                                         
2 <== <T:RpcMessage,id:2,method:"login">i{params:{"login":{"password":"d622fa9568f5dda56ea22bd25526b4c7b778e13d","type":"SHA1","user":"iot"},"opti ons":{"idleWatchDogTimeOut":0}}}
2 ==> <T:RpcMessage,id:2>i{result:{"clientId":2}}

shvrsh calls method runPtyCmd on shvagent to create tunnel

2 <== <T:RpcMessage,id:3,shvPath:"test/agent",method:"launchRexec">i{params:{"method":"runPtyCmd","params":["/bin/bash",123,34]}}
1 ==> <T:RpcMessage,id:3,shvPath:"",method:"launchRexec",cid:2,protocol:1u,grant:"rd">i{params:{"method":"runPtyCmd","params":["/bin/bash",123,34]}}

shvagent asks shvbroker to find node with public IP to open tunnel, there might be more brokers in broker tree and the TunnelCtl::State::FindTunnelRequest tries to find one with public IP closest to client opening tunnel.

1 <== <T:RpcMessage,id:3,cid:2,rcid:null,tctl:<T:TunnelCtl>i{State:1}>i{}

shvbroker response with tunneling broker IP and secret to login, this is the same broker in this case. Note TunnelCtl::State::FindTunnelResponse

1 ==> <T:RpcMessage,id:3,tctl:<T:TunnelCtl>i{State:2,Host:"10.0.3.93",Port:3755,Secret:"ef9b3cd47426dfbfb295a63df5e0e8bf57ffafae",CallerIds:2}>i{}

shvrexec logins to broker with secret to create tunnel

3 <== <T:RpcMessage,id:1,method:"hello">i{}
3 ==> <T:RpcMessage,id:1>i{result:{"nonce":"1524437174"}}
3 <== <T:RpcMessage,id:2,method:"login">i{params:{"login":{"password":"6299fa3137d1615cbc93a7de35edd3b6de0bfaea","type":"SHA1","user":""},"options":{"idleWatchDogTimeOut":0,"tunnel":{"secret":"ef9b3cd47426dfbfb295a63df5e0e8bf57ffafae"}}}}
3 ==> <T:RpcMessage,id:2>i{result:{"clientId":3}}

shvrexec opens tunnel as response to request id 3 (previous method call runPtyCmd), response contains tunneling broker public IP and also the secret to login. Note rcid field, this enables to make response to this response. Note TunnelCtl::State::CreateTunnelRequest

3 <== <T:RpcMessage,id:3,cid:2,rcid:null,tctl:<T:TunnelCtl>i{State:3,Host:"10.0.3.93",Port:3755,Secret:"ef9b3cd47426dfbfb295a63df5e0e8bf57ffafae",RequestId:3,CallerIds:2}>i{}
2 ==> <T:RpcMessage,id:3,protocol:1u,rcid:3,tctl:<T:TunnelCtl>i{State:3,Host:"10.0.3.93",Port:3755,Secret:"ef9b3cd47426dfbfb295a63df5e0e8bf57ffafae",RequestId:3,CallerIds:2}>i{}

shvrsh answers with TunnelCtl::State::CreateTunnelResponse and rcid field to record client ids for response to this response

2 <== <T:RpcMessage,id:3,cid:3,rcid:null,tctl:<T:TunnelCtl>i{State:4}>i{}
3 ==> <T:RpcMessage,id:3,protocol:1u,rcid:2,tctl:<T:TunnelCtl>i{State:4}>i{}

the tunnel is created now

shvrsh and shvrexec are writing to the tunnel as responses with rq id == 3

3 <== <T:RpcMessage,id:3,cid:2>i{result:[1,"fanda@tecka:~/proj/_build/shv/bin$ "]}
2 ==> <T:RpcMessage,id:3,protocol:1u>i{result:[1,"fanda@tecka:~/proj/_build/shv/bin$ "]}
2 <== <T:RpcMessage,id:3,cid:3>i{result:[0,"l"]}
3 ==> <T:RpcMessage,id:3,protocol:1u>i{result:[0,"l"]}
3 <== <T:RpcMessage,id:3,cid:2>i{result:[1,"l"]}
2 ==> <T:RpcMessage,id:3,protocol:1u>i{result:[1,"l"]}
2 <== <T:RpcMessage,id:3,cid:3>i{result:[0,"s"]}
3 ==> <T:RpcMessage,id:3,protocol:1u>i{result:[0,"s"]}
3 <== <T:RpcMessage,id:3,cid:2>i{result:[1,"s"]}
2 ==> <T:RpcMessage,id:3,protocol:1u>i{result:[1,"s"]}
2 <== <T:RpcMessage,id:3,cid:3>i{result:[0,"\r"]}
3 ==> <T:RpcMessage,id:3,protocol:1u>i{result:[0,"\r"]}
3 <== <T:RpcMessage,id:3,cid:2>i{result:[1,"\r\n"]}
2 ==> <T:RpcMessage,id:3,protocol:1u>i{result:[1,"\r\n"]}
3 <== <T:RpcMessage,id:3,cid:2>" ... 362 bytes of data ... "
2 ==> <T:RpcMessage,id:3,protocol:1u>" ... 362 bytes of data ... "
3 <== <T:RpcMessage,id:3,cid:2>" ... 766 bytes of data ... "
2 ==> <T:RpcMessage,id:3,protocol:1u>" ... 766 bytes of data ... "
3 <== <T:RpcMessage,id:3,cid:2>i{result:[1,"fanda@tecka:~/proj/_build/shv/bin$ "]}
2 ==> <T:RpcMessage,id:3,protocol:1u>i{result:[1,"fanda@tecka:~/proj/_build/shv/bin$ "]}