RPS doesn't decrease when requests stop suddenly

[wozniakjan]

Report

When using RPS as a scaler metric, it works well as long as there is some traffic for a particular HTTPScaledObject. But when the traffic suddenly stops, it takes the duration of the window for the RPS metric to jump to 0 and until then, it remains steady.

Expected Behavior

The RPS should monotonously decrement during the window where there is no traffic.

Actual Behavior

The RPS doesn't decrement if traffic suddenly stops.

Steps to Reproduce the Problem

1. create HTTPScaledObject with scalingMetric requestRate
2. generate traffic for the application
3. observe RPS grow according to the traffic
4. cut the traffic suddenly
5. RPS remains close to the last value, application doesn't get scaled down
6. after the entire window elapses, RPS drops to 0 and application finally scales down

Logs from KEDA HTTP operator

attached as files
keda-add-ons-http-controller-manager.log
keda-add-ons-http-interceptor.log

HTTP Add-on Version

0.8.0

Kubernetes Version

1.28

Platform

Other

Anything else?

after a little bit of digging, I think this might be due to the implementation of counter middleware. The Record for RPS seems to be only called when there is traffic for that particular application and values to the buckets get inserted. It should probably also insert zero values automatically to the buckets when there is no traffic.

http-add-on/pkg/queue/queue.go

Lines 67 to 73 in 1e27b39

	func (r *Memory) Increase(host string, delta int) error {
	r.mut.Lock()
	defer r.mut.Unlock()
	r.concurrentMap[host] += delta
	r.rpsMap[host].Record(time.Now(), delta)
	return nil
	}

[wozniakjan]

ah, never mind, looks like this is by design

http-add-on/pkg/queue/bucketing.go

Lines 105 to 118 in 1e27b39

	case d < t.window:
	// If we haven't received metrics for some time, which is less than
	// the window -- remove the outdated items and divide by the number
	// of valid buckets
	stIdx := t.timeToIndex(t.lastWrite)
	eIdx := t.timeToIndex(now)
	ret := t.windowTotal
	for i := stIdx + 1; i <= eIdx; i++ {
	ret -= t.buckets[i%len(t.buckets)]
	}
	numB := math.Min(
	float64(t.lastWrite.Sub(t.firstWrite)/t.granularity)+1, // +1 since the times are inclusive.
	float64(len(t.buckets)-(eIdx-stIdx)))
	return roundToNDigits(precision, float64(ret)/numB)

I'd still like to make an argument that the RPS should decrease if the traffic goes to zero. At the moment, as implemented, if traffic drops from 10 to 0 RPS, it yields a higher RPS metric compared to traffic dropping from 10 to 1 RPS. This intuitively doesn't feel correct to me.

wdyt @JorTurFer, @zroubalik?