Dealing with files in the File Stream Adapter that don't have a timestamp on every line.

[AkthemRehab]

Hello,
I am wondering if the file stream has an option to give a unique timestamp for each line? An example use case is a vibration reading that usually has 'one' original timestamp. Usually the name of the file is the timestamp and the readings within the file are taken within the span of 1 second or less.

Best,
Akthem

[tenthe]

Hi @AkthemRehab,

I find the idea of adding this function to the File Stream Adapter very interesting. The File Stream Adapter is currently only intended for test and development purposes to simulate "real" data streams. For this purpose, a timestamp is currently expected in the events in order to be able to simulate the original playback speed.
(Currently you would have to add the timestamp "manually" to the CSV file before uploading it)

We still need to clarify a few details before we can implement the feature. In particular, what the user has to configure in order to use the functionality.

My suggestion would be to add another option to the Replay Speed item. This could be called adding artificial timestamp. The user would then have the option to define a fixed time interval in ms between events.
Current configuration of File Stream Adapter:

Would this work from your point of view or do we need further settings? And do you think that the configuration is understandable for users?

Once we have clarified the details, we can convert the discussion into an issue and mark it as a good first issue.

[AkthemRehab]

Hi Philipp,
This would work as a workaround, Yes.

Will remain the handling of files in their own as objects having their own pipelines. This is of course without the need of adding a timestamp per line. Perhaps even being able to extract the timestamp from the file name or metadata. However, this is the topic of question #2299 I guess.

Let us proceed with turning this question to an issue. Let me know if I need to initiate this from my end.

Thank you for your guidance and support throughout.

[tenthe]

I think we should provide an input field for the start date instead of using the file name, as this would be a more general solution. What do you think?
I have created an issue from this discussion (#2338).

[AkthemRehab]

Sounds Good, Philipp. The freedom to name the file makes more sense.

[tenthe]

An issue was created for this discussion. The technical implementation details can be discussed there. See #2338