Splitting from one input to two outputs

[CobraCo2000]

Hello,
I have a little problem. I would like to filter an input via Fluentbit and split it into several outputs.

My log file looks like this.

[
{"name":"test1","key":1,"value":"abc"},
{"name":"test2","key":1,"value":"abc"},
{"name":"test3","key":1,"value":"abc"},
{"name":"test1","key":1,"value":"abc"},
{"name":"test1","key":1,"value":"abc"},
{"name":"test3","key":1,"value":"abc"}
]

This should now be split so that I can import all rows with the key (“name”: “test1”) into a different elasticSearch index than (“name”: “test2” or “name”: “test3”)

Here is a small example of my config file. But I cannot create a new tag for the output in the GREP filter.

[INPUT]
    Name   tail
	Path   /temp/logfile.log
	Parser json
    Tag    logfile

[FILTER]
    Name   grep
    Match  *
    Regex  $name  ^test1
    Tag    one.*

[FILTER]
    Name   grep
    Match  *
    Exclude $name  ^?!test1
    Tag    trash.*

[OUTPUT]
    Name   stdout
    Match  one.*

[OUTPUT]
    Name   stdout
    Match  trash.*

The reason why I need this is that I need to archive some data from the log file longer than others.
Is there currently a way to split logs like in my example?

[patrick-stephens]

This is normal behaviour, have a look at the routing section: https://docs.fluentbit.io/manual/concepts/data-pipeline/router

You do not "split" inputs, you just match multiple outputs to the same record - this can be done any number of times.

Filters can be applied (or not) by ensuring they match to only what you want, you can use regexes and wildcards in match rules.

If you want to create new tags for the same record then use the rewrite_tag filter: https://docs.fluentbit.io/manual/pipeline/filters/rewrite-tag

[CobraCo2000]

Hi patrick,

thanks for your support. With the router, I can only create multiple imports on one output. But with the rewrite_tag I can achieve exactly what I need.
My configuration now looks like this und works perfect.

[INPUT]
Name tail
Path /temp/logfile.log
Parser json
Tag logfile

[FILTER]
Name 		  rewrite_tag
Match 		  *
Rule          $name \btest1\b  from.one.out false
Rule          $name ^(?!.*\btest1\b).+$  from.trash.out false
Emitter_Name  re_emitted

[OUTPUT]
Name stdout
Match one.*

[OUTPUT]
Name stdout
Match trash.*