fluent-bit process memory not being released even if we deleted the complete list of log file path.

[vikas271019]

when deploying fluent-bit sidecar with loggen microservice with the input log file path being set as wildcard /test*.log.
we observed while creating a thousands of log files from test1.log to tet1000.log, fluent-bit process memory increases for instance from 13660 KB to 23792 KB. After sometime we manually deleted all the log files including the default one of test0.log. we found the memory of the fluent-bit process is not decreasing.
When we checked to identify the file descriptor of the deleted file is still open using the below command
lsof | grep deleted
--> output of the command as --1 /tmp/loggen/loggen.bin test0.log (deleted)
that means above output will list all processes with open file descriptor that are linked to deleted files.
Overall this means, deleted file still being open and hence fluent-bit process memory not being released.

we also tried ways or configuration from fluent-bit to release the memory when files are being deleted.
configure Ignore_Older – prevent fluent-bit from tailing old or deleted file
Exit_On_Eof fluent-bit to stop reading a file after deletion

Even after following the above recommendation , there is release in memory of fluent-bit process.

Expected behavior
Ideally there should be a way or parameter from fluent-bit configuration to release the memory.

Your Environment
please refer the below config map of fluent-bit sidecar

Name:         test-loggen-logshipper-sidecar
Namespace:    test-loggen
Labels:       app.kubernetes.io/instance=test-loggen
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/name=test-loggen
              app.kubernetes.io/version=1.0.0
Annotations:  test.com/product-name: test-loggen
              meta.helm.sh/release-name: test-loggen
              meta.helm.sh/release-namespace: test-loggen

Data
====
outputs.conf:
----
[OUTPUT]
    name                 http
    match                event.*
    alias                log_transformer
    json_date_key        false
    host                 test-log-transformer
    port                 9443
    retry_limit          false
    log_response_payload false
    format               json
    tls                  on

parsers.conf:
----
[PARSER]
    name   json
    format json

filters.conf:
----

[FILTER]
    name  modify
    match event.*
    add   metadata.namespace ${NAMESPACE}
    add   metadata.node_name ${NODE_NAME}
    add   metadata.pod_name  ${POD_NAME}
    add   metadata.pod_uid   ${POD_UID}

fluent-bit.conf:
----
@INCLUDE /etc/fluent-bit/inputs.conf
@INCLUDE /etc/fluent-bit/outputs.conf
@INCLUDE /etc/fluent-bit/filters.conf

[SERVICE]
    flush           3
    grace           10
    log_level       info
    parsers_file    /etc/fluent-bit/parsers.conf
    http_server     on
    http_listen     localhost
    http_port       2020
    storage.metrics on

inputs.conf:
----
[INPUT]
    name              tail
    tag               event.fluent-bit
    alias             log_shipper
    buffer_chunk_size 32k
    buffer_max_size   32k
    path              /logs/logshipper.log
    path_key          filename
    read_from_head    true
    refresh_interval  5
    rotate_wait       10
    skip_empty_lines  off
    skip_long_lines   off
    key               message
    db                /logs/logshipper.db
    db.sync           normal
    parser            json
    mem_buf_limit     1MB
[INPUT]
    name              tail
    tag               event.file0
    alias             file0
    buffer_chunk_size 32k
    buffer_max_size   32k
    read_from_head    true
    refresh_interval  5
    rotate_wait       10
    skip_empty_lines  off
    skip_long_lines   off
    key               message
    db                /logs/file0.db
    db.sync           normal
    db.locking        true
    db.journal_mode   off
    path              /logs/test*.log
    path_key          filename
    exclude_path      /logs/logshipper.log
    mem_buf_limit     1MB`

[fluent bit] version=3.0.5