Svelte v5 runes benchmark results & discussion 🔥

[transitive-bullshit]

(lower times are better)

I've been working on a deep-dive comparison of the best TS reactivity libs.

During this, I created a fork of the JS reactivity benchmark and added Svelte v5.

I've been in touch with @trueadm to verify that my Svelte v5 adapter looks good.

To my surprise, Svelte v5 outperformed all other reactivity libs by quite a solid margin on all benchmark tests except the cellx ones.

I have some follow-up questions, so I figured I'd open up the discussion to more of the Svelte community.

1. The cellx benchmark tests are the only ones where Svelte v5 performs significantly worse than all of the other frameworks (~100x slower). They're not included in the average benchmark graph because MobX fails to run them correctly at all. Perhaps this usage pattern is something worth looking into as a potential perf pitfall in Svelte?

2. Do these results match your team's expectations? I was honestly quite surprised and excited by how much faster Svelte v5 was compared with the rest of the field.

3. I'd love to understand why Svelte v5 is performing so well on these benchmarks. Do you have any up front ideas as to why this may be the case, whether they be algorithmic or more practical optimizations?

4. These benchmarks currently only focus on deep dependency trees of shallow primitives. I'd love to add some benchmarks of deeply nested reactive objects / arrays / maps / sets for the frameworks which support them. Thoughts?

5. Lastly and most importantly, I think it's a shame that Svelte v5's reactivity isn't packaged for standalone usage but rather exposed as an unofficial, internal API. Would you consider an RFC to isolate the core reactivity ala @vue/reactivity?

---

For context, my deep-dive on this was inspired by the TC39 Signals Proposal, which is why I'm looking to understand what aspects of the best TS reactivity libs ala Svelte v5 could potentially be backported into the proposal or polyfill.

Thanks! 🙏

[trueadm]

Thanks for looking into this!

The cellx benchmark tests are the only ones where Svelte v5 performs significantly worse than all of the other frameworks (~100x slower). They're not included in the average benchmark graph because MobX fails to run them correctly at all. Perhaps this usage pattern is something worth looking into as a potential perf pitfall in Svelte?

I need to do an investigation into this, as it might be something we're doing wrong or maybe something with how the adapter is setup. I'll get back to you.

Do these results match your team's expectations? I was honestly quite surprised and excited by how much faster Svelte v5 was compared with the rest of the field.

We've spent a lot of time tuning our signals implementation for optimal memory and runtime performance. I've spent a lot of time looking at V8 deopts and the team has spent significant time cleaning up and simplifying hot code paths.

I'd love to understand why Svelte v5 is performing so well on these benchmarks. Do you have any up front ideas as to why this may be the case, whether they be algorithmic or more practical optimisations?

We have our own benchmark on the repo that was adapted from the reactivity benchmark. Just run pnpm bench and you'll see what I mean. These benchmarks also run on the CI for every PR, so we know when we're potentially regressing or how impactful a change is to our signals implementation. This is probably why we're doing well in this benchmark.

These benchmarks currently only focus on deep dependency trees of shallow primitives. I'd love to add some benchmarks of deeply nested reactive objects / arrays / maps / sets for the frameworks which support them. Thoughts?

That would be useful, but ultimately every framework does deep reactivity quite differently so it might not really be a fair comparison.

Lastly and most importantly, I think it's a shame that Svelte v5's reactivity isn't packaged for standalone usage but rather exposed as an unofficial, internal API. Would you consider an RFC to isolate the core reactivity ala @vue/reactivity?

We have no intentions of exposing our internal signals API in the public domain. Signals aren't really the important thing here either, and that's awesome because it allows us to change our signals architecture freely without having to release new majors of Svelte. It also afford us the ability to move to the TC39 signals once it's stable, or to throw our signals entirely in the case we find a better reactivity subsystem in the future.

[Rich-Harris]

I'd love to understand why Svelte v5 is performing so well on these benchmarks

I think it's a shame that Svelte v5's reactivity isn't packaged for standalone usage but rather exposed as an unofficial, internal API

Just to expand on what @trueadm already said: these things are connected! Part of the reason Svelte's signals are fast and memory-efficient is that unlike other libraries, we don't need to design them with ergonomics in mind. So whereas other libraries might give you a pair of closures, or a class with .get() and .set() methods or a .value accessor, a Svelte signal is a cheap, ugly-looking POJO.

At a number of points during Svelte 5's development we've changed the details of this system in ways that would be breaking if the API were publicly exposed. Keeping the guts contained gives us the ability to design things optimally, and evolve the system without worrying about maintaining a contract with the outside world.

[transitive-bullshit]

Makes perfect sense && thanks for the prompt reply, guys 🙏

I did a quick experiment on top of Svelte v5 to see the difference between standalone $.get()/$.set() versus a .value accessor and the results were really interesting.

In this output, "Svelte v6" is my hacky experiment using the .value wrapper which is ~2x slower and just about dead even with https://github.com/milomg/reactively by @milomg. I'm assuming Svelte v5 runes currently uses a similar 2-phase graph coloring algorithm? If so, then it's really interesting that the perf diff is due almost solely to this ergonomic difference and potentially something the Signal Polyfill efforts should consider.

[trueadm]

So I figured the issue, it was indeed in the adapter. Try this one instead;

export const svelteFramework: ReactiveFramework = {
  name: "Svelte v5",
  signal: (initialValue) => {
    const s = $.state(initialValue);
    return {
      write: (v) => $.set(s, v),
      read: () => $.get(s),
    };
  },
  computed: (fn) => {
    const c = $.derived(fn);
    return {
      read: () => $.get(c),
    };
  },
  effect: $.render_effect,
  withBatch: $.flush_sync,
  withBuild: <T>(fn: () => T): T => {
    let res: T | undefined;
	$.effect_root(() => {
		$.render_effect(() => {
			res = fn();
		});
	});
    return res!;
  },
};

[transitive-bullshit]

That makes sense, though this version of the adapter fails the following unit test:

  test(`${name} | effect`, () => {
    const spy = vi.fn();

    framework.withBuild(() => {
      const s = framework.signal(2);
      const c = framework.computed(() => s.read() * 2);

      framework.effect(() => {
        console.log("EFFECT", c.read());
        spy();
      });
      framework.withBatch(() => {
        s.write(3);
      });

      expect(s.read()).toEqual(3);
      expect(c.read()).toEqual(6);
    });

    expect(spy.mock.calls.length).toBe(2);
  });

(I added additional unit tests to the benchmark to make sure that the adapters are working in a more apples-to-apples way)

The issue is that the spy is never called a second time after the write, so spy.mock.calls.length ends up being 1.

[trueadm]

I wonder if #13282 fixes your issue. Also try ensuring JSDOM is enabled in your test env, that's helped in the past too.

[transitive-bullshit]

I'm getting this same unit test failure for the adapter above when using #13282 as main

[trueadm]

Modify your test:

  test(`${name} | effect`, () => {
    const spy = vi.fn();
    const s = framework.signal(2);
    const c = framework.computed(() => s.read() * 2);

    framework.withBuild(() => {
      framework.effect(() => {
        console.log("EFFECT", c.read());
        spy();
      });
    });

    framework.withBatch(() => {
      s.write(3);
    });

    expect(s.read()).toEqual(3);
    expect(c.read()).toEqual(6);

    expect(spy.mock.calls.length).toBe(2);
  });

[transitive-bullshit]

that works; thanks 🙏

[FilipeMarch]

I noticed React.js isn't included. Given its widespread use and importance in the development community, could you consider adding React.js to the benchmarks for a more comprehensive comparison?

[trueadm]

It's a signal reactivity benchmark, and isn't about UI rendering. React doesn't have signals, so that's why it's not in the benchmark.

[FilipeMarch]

Ah, I see now. Thank you for clarifying.

[Ocean-OS]

I would be curious to see how this compares to Svelte 4's reactivity, would that be possible?

[trueadm]

No. Svelte 4 doesn’t use signals. This is not a framework benchmark, it’s a signals implementation benchmark.

[webJose]

Guys, I suppose you all know this page? https://krausest.github.io/js-framework-benchmark/current.html

It was updated to include Svelte v5. As of today, it runs v5.0.5. Unsure if the Svelte team has contributed to this effort, but would be nice if:

1. The version would be updated.
2. Bring back Svelte v4 to the list. I am dying to know the difference between v4 and v5.

Cheers!

[trueadm]

You can always put up a PR to update the version yourself. We're a bit busy with Advent of Svelte. :P

[webJose]

Hello! I probably can update the v5 version, but I suppose that I cannot revive Svelte v4 on my own. I suppose some knowledge of the testing framework is needed? Like in this thread, there is an adapter that must be created.

Anyway, thanks for the attention and the prompt reply. Much appreciated.

[trueadm]

You can probably find the old Svelte 4 implementation in the git history.