Skip to content

Latest commit

 

History

History
126 lines (106 loc) · 4.4 KB

VSocks.md

File metadata and controls

126 lines (106 loc) · 4.4 KB
Raw

Kata Containers and vsocks

Introduction

There are two different ways processes in the virtual machine can communicate with processes in the host. The first one is by using serial ports, where the processes in the virtual machine can read/write data from/to a serial port device and the processes in the host can read/write data from/to a unix socket. Most GNU/Linux distributions have support for serial ports, making it the most portable solution. However, the serial link limits read/write access to one process at a time. To deal with this limitation the resources (serial port and unix socket) must be multiplexed. In Kata Containers those resources are multiplexed by using kata-proxy and yamux, the following diagram shows how it's implemented.

proxy communication diagram

.----------------------.
| .------------------. |
| | .-----.  .-----. | |
| | |cont1|  |cont2| | |
| | `-----'  `-----' | |
| |       \   /      | |
| |    .---------.   | |
| |    |  agent  |   | |
| |    `---------'   | |
| |         |        | |
| |    .-----------. | |
| |POD |serial port| | |
| `----|-----------|-' |
|      |  socket   |   |
|      `-----------'   |
|           |          |
|       .-------.      |
|       | proxy |      |
|       `-------'      |
|           |          |
|  .------./ \.------. |
|  | shim |   | shim | |
|  `------'   `------' |
| Host                 |
`----------------------'

A newer, simpler method is vsocks, which can accept connections from multiple clients and does not require multiplexers (kata-proxy and yamux). The following diagram shows how it's implemented in Kata Containers.

vsock communication diagram

.----------------------.
| .------------------. |
| | .-----.  .-----. | |
| | |cont1|  |cont2| | |
| | `-----'  `-----' | |
| |       |   |      | |
| |    .---------.   | |
| |    |  agent  |   | |
| |    `---------'   | |
| |       |   |      | |
| | POD .-------.    | |
| `-----| vsock |----' |
|       `-------'      |
|         |   |        |
|  .------.   .------. |
|  | shim |   | shim | |
|  `------'   `------' |
| Host                 |
`----------------------'

System requirements

The host Linux kernel version must be greater than or equal to v4.8, and the vhost_vsock module must be loaded or built-in (CONFIG_VHOST_VSOCK=y). To load the module run the following command:

$ sudo modprobe -i vhost_vsock

The Kata Containers version must be greater than or equal to 1.2.0 and use_vsock must be set to true in the runtime configuration file.

Advantages of using vsocks

High density

Using a proxy for multiplexing the connections between the VM and the host uses 4.5MB per POD. In a high density deployment this could add up to GBs of memory that could have been used to host more PODs. When we talk about density each kilobyte matters and it might be the decisive factor between run another POD or not. For example if you have 500 PODs running in a server, the same amount of kata-proxy processes will be running and consuming for around 2250MB of RAM. Before making the decision not to use vsocks, you should ask yourself, how many more containers can run with the memory RAM consumed by the kata-proxies?.

Reliability

kata-proxy is in charge of multiplexing the connections between virtual machine and host processes, if it dies all connections get broken. For example if you have a POD with 10 containers running, if kata-proxy dies it would be impossible to contact your containers, though they would still be running. Since communication via vsocks is direct, the only way to lose communication with the containers is if the VM itself or the shim dies, if this happens the containers are removed automatically.