Removing support for sharing file on disk between tracee-rules and tracee-ebpf, and EOT

[grantseltzer]

As part of working on #500 I am thinking about the support for interaction between tracee-ebpf and tracee-rules. In particular, right now tracee-ebpf has an option for transmitting an 'EOT' event when it has finished creating events. This feature exists solely for the purpose of an interaction between tracee-ebpf and tracee-rules where they share a file on disk for reading and writing from.

I don't believe this is something we should support. One reason is that this is computationally, and memory inefficient. There are many things that can go wrong, for example tracee-ebpf producing far more events than can be consumed, or the file not being set up to support asynchronous r/w leading to data corruption.

I propose the following:

• Remove the option for tracee-ebpf to transmit EOT
• tracee-rules should always close the input stream if it ever receives an EOF from the operating system (and not require an option flag to do so). That would occur when reading events from a file or if a pipe is closed by producer. This would not require any changes in tracee-ebpf

[grantseltzer]

@yanivagman is there any reason the research team needs this?

[itaysk]

protip: never call a research person "R&D" 😁😅

[grantseltzer]

Oh woops, I didn't know this. I thought that was the name of the team.

[itaysk]

Thanks Grant, this is a good idea.

In addition to what you wrote, communicating via a file means it will grow indefinitely and cleaning it up is pretty complicated. which makes me think no one will ever use it. Unless there are any objections, let's go for it

[itaysk]

To clarify, tracee-rules can still read from a regular file on disk, as long as it's in "batch" mode, meaning it's not being written to by another process simultanously

[yanivagman]

@grantseltzer, I didn't add the EOT option, but I do remeber what I said to @itaysk before he added this.
First, I agree that communicating through a file is inefficient. I discussed this with @itaysk and we agreed we should used a pipe instead (a shared memory was also an option, but a file was chosen just for the POC).
When I wrote the first prototype of tracee-rules, I also tried to use EOF, which seemed to work at first, but then I noticed that in streaming mode, I got errors after only a few events when not using stdin as the file input, but a regular file. This can be explained in that when we use a file (which is not stdin), and no more events are written to the file (or just in a slower pace than what we read from the file), we quickly reach the EOF. However, events can still occur after this EOF happened.
So, to conclude - if not using a regular file for communication - no problem.
However, we should keep in mind that stdin will not always be an option. In the future, we might add additional sources (for example, network packets which will be streamed as well) - in that case, we should take care of proper communication of different input sources through pipes