Need help, to hide the line when no data

[thoath2]

Hello.
My heat pump failed on December 15th, 2024 from 6:00 a.m. to 2:00 p.m. and did not deliver any values, but a line was displayed in plotly-graph.
In https://github.com/dbuezas/lovelace-plotly-graph-card/discussions/317 i found a solution.
I have configured the filter how there recommended (entity: "sensor.wp_aussentemperatur"), but it does not work. I do something wrong.
Here is the standard-diagramm of HA:

And this is plotly-Graph:

And here is the yaml-code:

type: custom:plotly-graph
title: Temperaturen (WP-Allgemein)
defaults:
  yaxes:
    fixedrange: true
hours_to_show: current_day
refresh_interval: 10
min_y_axis: -50
max_y_axis: 10000
entities:
  - entity: sensor.wp_aussentemperatur
    name: |
      $ex {
            return (
                    "Außen "
                    + "<span style='font-size: 16px; color: rgb(154 205 50)'>" 
                    + ys[ys.length - 1].toFixed(1) 
                    +  " °C   </span>"
                   )
          }
    showlegend: true
    statistic: mean
    filters:
      - fn: |-
          ({xs, ys}) => ({
              xs: xs.flatMap((x, i) => ys[i+1] === null ? [x,x]:[x]),
              ys: ys.flatMap((y, i) => ys[i+1] === null ? [y,null]:[y])
          })
    extend_to_present: false
    period: 5minute
    fill: null
    fillcolor: rgba(154, 205, 50, 0.5)
    line:
      color: rgba(154, 205, 50, 1)
      width: 2
  - entity: sensor.wmz_solar_forward_water_temperature
    name: |
      $ex {
            return (
                    "Solar-Vorlauf "
                    + "<span style='font-size: 16px; color: rgb(255 165 0)'>" 
                    + ys[ys.length - 1].toFixed(1) 
                    +  " °C   </span>"
                   )
          }
    showlegend: true
    stackgroup: null
    statistic: mean
    filters:
      - map_y: "y < 0 ? 0 : (1 * y)"
    extend_to_present: false
    period: 5minute
    fill: null
    fillcolor: rgba(255, 165, 0, 0.8)
    line:
      color: rgba(255, 165, 0, 1)
      width: 2
  - entity: sensor.wp_hp1_flow_line_temperature
    name: |
      $ex {
            return (
                    "WP-Vorlauf "
                    + "<span style='font-size: 16px; color: rgb(255 69 0)'>" 
                    + ys[ys.length - 1].toFixed(1) 
                    +  " °C   </span>"
                   )
          }
    showlegend: true
    statistic: mean
    filters:
      - map_y: "y < 0 ? 0 : (1 * y)"
    extend_to_present: false
    period: 5minute
    fill: null
    fillcolor: rgba(255, 69, 0, 0.5)
    line:
      color: rgba(255, 69, 0, 1)
      width: 2
  - entity: sensor.wp_hp1_return_line_temperature
    name: |
      $ex {
            return (
                    "WP-Rücklauf "
                    + "<span style='font-size: 16px; color: rgb(70 130 180)'>" 
                    + ys[ys.length - 1].toFixed(1) 
                    +  " °C   </span>"
                   )
          }
    showlegend: true
    statistic: mean
    filters:
      - map_y: "y < 0 ? 0 : (1 * y)"
    extend_to_present: false
    period: 5minute
    fill: null
    fillcolor: rgba(70, 130, 180, 0.5)
    line:
      color: rgba(70, 130, 180, 1)
      width: 2
layout:
  height: 400
  shapes:
    - type: line
      name: |
        $ex {
              var aktDate = new Date(); //Date.now();
              var aktHour = aktDate.getHours();
              var strAktHour = "0" + aktHour 
              strAktHour = strAktHour.substr(strAktHour.length - 2)
              var aktMinute = aktDate.getMinutes();
              var strAktMinute = "0" + aktMinute 
              strAktMinute = strAktMinute.substr(strAktMinute.length - 2)
              var strHM = strAktHour + ":" + strAktMinute
              return (
                      "Akt. Zeitpunkt "                    
                      + [strHM] + " "                   
                     )
            }
      showlegend: true
      x0: $ex Date.now()
      x1: $ex Date.now()
      y0: 0
      y1: 1
      xref: x
      yref: paper
      line:
        color: grey
        dash: dot
        width: 2

Have you an idea, what i make wrong?

[dbuezas]

When you set period you get data from the long term statistics db, and that db doesn't seem to store null when a sensor goes unavailable. Can you try removing period to confirm?

The good news is that one can then compare the dates of consecutive data points and add a null if they are further away than the period

[thoath2]

Hi dbuezas, sorry, I'm not so safe with the filter-syntax. I only understand a little what she does.
If I understand you correctly, I have to use the two-time period-duration.
My period is 5 minutes.
In the filter I should use:

1000*60*5*2

Is this correct?
I used this. But it doesn't work:

filters:
      - fn: |-
          ({xs, ys}) => ({
              xs: xs.flatMap((x, i) => xs[i]+1000*60*5*2 <= xs[i+1]  ? [x,x]:[x]),
              ys: ys.flatMap((y, i) => xs[i]+1000*60*5*2 <= xs[i+1]  ? [y,null]:[y])
          })
    extend_to_present: false
    period: 5minute

[dbuezas]

Then I ran out of ideas. I suggest extract the raw data to the console following the debug section of the readme

[thoath2]

Hi dbuezas,
I think it's too high for me. I was able to open the console. I entered the following into the card code. But what should I see in the console? Or do I have to use another fn: to get the data?

type: custom:plotly-graph
title: Test
defaults:
  yaxes:
    fixedrange: true
hours_to_show: current_month
refresh_interval: 10
entities:
  - entity: sensor.wp_aussentemperatur
    name: Außentemperatur
    showlegend: true
    statistic: mean
    filters:
      - fn: console.log
      - fn: |-
          ({xs, ys}) => ({
              xs: xs.flatMap((x, i) => xs[i]+1000*60*5*2 < xs[i+1]  ? [x,x]:[x]),
              ys: ys.flatMap((y, i) => xs[i]+1000*60*5*2 < xs[i+1]  ? [y,0]:[y])
          })
    extend_to_present: false
    period: 5minute
    fill: null
    line:
      color: rgba(154, 205, 50, 1)
      width: 2

[dbuezas]

Open the chrome developer console and look there for the arrays of data. Google or ask an AI for details on chrome devtools

[thoath2]

Hi dbuezas,
I have the solution, with the help of chatGPT. Its great.
It is a very large code-snippet. But it works fine.

Hier der Code:

type: custom:plotly-graph
title: Test
defaults:
  yaxes:
    fixedrange: true
hours_to_show: current_month
refresh_interval: 10
entities:
  - entity: sensor.wp_aussentemperatur
    name: Außentemperatur
    showlegend: true
    statistic: mean
    filters:
      - fn: |-
          ({ xs, ys }) => {
              const resolution = 1000 * 60 * 5; // 5 Minuten in Millisekunden
              const threshold = resolution * 2; // doppelte Auflösung

              const newXs = [];
              const newYs = [];

              for (let i = 0; i < xs.length - 1; i++) {
                  newXs.push(xs[i]);
                  
                  // Filter für Ausreißer: Wenn der Wert unter -50 ist, setzen wir ihn auf null
                  const filteredY = ys[i] < -50 ? null : ys[i];
                  newYs.push(filteredY);

                  // Prüfe, ob die Zeitdifferenz größer als die Schwelle ist
                  if (xs[i + 1] - xs[i] > threshold) {
                      // Füge Zwischenpunkt für die X-Werte und null für Y-Werte hinzu
                      const middleX = xs[i] + resolution; // oder eine andere Zeit für die Lücke
                      newXs.push(middleX); // Zwischenwert für X (Lücke)
                      newYs.push(null); // Null für Y (stellt Lücke dar)
                  }
              }

              // Verarbeite den letzten Punkt und überprüfe auch hier auf Ausreißer
              newXs.push(xs[xs.length - 1]);
              // Ausreisserkontrolle
              const lastY = ys[ys.length - 1] < -50 ? null : ys[ys.length - 1];
              newYs.push(lastY);

              return { xs: newXs, ys: newYs };
          }
    extend_to_present: false
    period: 5minute
    fill: null
    line:
      color: rgba(154, 205, 50, 1)
      width: 2

[thoath2]

Based on this discussion, I propose adding a new configuration option, such as:

hide_line_when_no_data: true/false

This option would encapsulate the script above currently required to hide a line when there is no data on an entity level. By doing so, it would significantly reduce the amount of YAML code users need to write and maintain. Additionally, this change would free up the filter element to be used for other purposes, such as preventing outliers.

To implement this:

• The filtering for outliers currently included in the script can be removed.
• The new option should dynamically derive the value for the variable:

    const resolution = 1000 * 60 * 5; // 5 minutes in milliseconds

from the period element in the configuration.

This feature would make the card more versatile and user-friendly by providing a cleaner, more streamlined approach to managing the visibility of lines when no data is present.

Thank you for considering this enhancement!

[dbuezas]

The frequency of data updates depends on the sensor and it may be variable too. The integration is the one responsible to send an update as "unavailable" if the sensor is to be considered as such, and when that happens this card does ensure plotlyjs receives a data point with a null value, which is by default plotted as a hole in the line.

Such a filter should therefore be customizable, I would take a PR for it