Why is Deno that slow?

[deeprobin]

Many systems developed in Rust are very fast and designed for performance.
What is the background that Node and Bun are faster?

Reference: Benchmarks on https://bun.sh/

[aapoalas]

I'm not part of the Deno team, but can give my view on a few things.

FFI

Deno's FFI has recently been changing quite a bit, and moving to a more direct binding. The numbers used on Bun's website (around 350 ns per FFI call) seem in line with what the FFI performance was after my semi-recent addition of FFI callbacks, during which I also ended up adding a lot of JS-side overhead. A later PR by the core team has removed that overhead and further made the previously indirect FFI call binding a direct one, meaning that the current FFI performance is one order of magnitude better, around 33 ns per noop FFI call on my machine.

This will bring Deno's FFI more in line with Node's NAPI (33ns/op comes to about 29,600,00 ops per second) but falls a bit short, which is as expected since while Node's NAPI is not quite a direct FFI call API, it still gets to benefit from customized function bindings for each FFI function. This means that eg. a noop NAPI call doesn't need to check the passed in parameters (as I understand anyway), the bindings already know the parameters that the C function takes which is none in this case.

Bun's FFI is similar to Deno's in that it is bound from the JS side, instead of the JS side loading up ready-built bindings like Node does. What Bun does differently to Deno, however, is that it JIT-compiles a custom binding function for each FFI function call. This means that for the noop FFI call, Bun's binding layer drops down to nearly nothing, it's just the JS engine calling a C function which immediately turns around and calls the C FFI noop function, nothing more. Not even the return value needs to be handled.

This is a great way to lower the overhead of FFI calls. Deno currently has a draft PR open of implementing the same kind of system, so it might well be that Deno's FFI will soonish reach the same heights as Bun's does. With the V8 engine there's even a chance to go beyond what JavaScriptCore enables using the V8 Fast API Calls (though I may be wrong and maybe JSC also has something similar). Look forward to that.

SQLite

While this benchmark is a fair benchmark in the sense of asking "how quickly can these different runtimes load a huge sqlite database", they're not exactly fair if the question is asked in a more limited fashion, eg. "using this given API, how quickly can these different runtimes load a huge sqlite database".

Bun has a big leg up here: They implement their own custom SQLite binding directly into the runtime, which gives them a great advantage in optimizing the performance of the binding. The Node package used, better-sqlite3, is a Node addon based library which then gets to utilize the performance of native code though dampened by the indirection that is the Node addon API.

Finally, the Deno library used is a WebAssembly based sqlite library. While WebAssembly is fast, it cannot quite measure up to pure native code. Of course between Deno and Wasm there is also a binding layer just how there is one between the others, so the Deno library doesn't get any magical wins here.

So, while the benchmark is definitely accurate for usage purposes, it's not really measuring the performance of Bun vs Node vs Deno but just different libraries doing the same thing in different runtimes.

HTTP

With HTTP performance I have less understanding. I could point to Deno's ops (calls from JS code to code running on the Rust side) using a general purpose low-overhead binding layer whereas Bun I think may have a more hand-written binding layer between JS calls and the corresponding Zig functions. However, I think a big part of the reason for this performance result is simply that they've done a great job in writing the HTTP server in Bun.

I guess I'm also somewhat bound to note that there's an ongoing effort to push the HTTP server performance to even greater heights in Deno.

[lucacasonato]

The benchmark is not representative of many real world traffic scenarios. There are certain HTTP benchmarks where Deno is faster than Node, where Node will be faster than bun, or where bun will be fastest. You can not compare the performance of two systems through a single HTTP benchmark. The benchmark does not inspect request headers, request method, request url, or any number of other fields which are important in real traffic scenarios. It doesn't do routing, static file serving, etc.

Additionally these are just very different HTTP servers: Deno supports h2 and TLS on it's http server, while bun supports neither. We also can do automatic response body compression, full-duplex body streaming, etc.

Jarred is doing very interesting work with Bun, and that is pushing us to improve performance, but comparing system performance through a single isolated benchmark is not helpful.

If you are just looking for synthetic benchmarks where Deno is faster, those exist. (They are not useful at all though).

[lucacasonato]

I think you misread my comment. I never said that Bun can not do routing - I said that the referenced benchmark does not do any routing. There are however things that Deno's HTTP server does, that Bun does not:

• The Bun HTTP server does not send the HTTP Date header on responses. This is a violation of HTTP/1.1 spec because it is an origin server with access to a reliable clock.
• Bun does not support HTTP/2.
• Bun does not support serving TLS encrypted HTTP traffic.

I stand by all statements in my original comment. They are all still true as of 14th October 2022.

[lucacasonato]

I don't care about uWebSockets. I care about the API that Bun exposes, that I can access from JS. I am not comparing Hyper and uWebSockets. I am talking about Deno and Bun. And Bun does not support HTTP/2 or TLS termination, which is the only claim I made in my above comment.

And again, I never said that you can not do routing in Bun (or uWebSockets). I said that the linked benchmark does not do routing, and because of that (and other things), I don't think it is a good benchmark. Please read my original comment...

[lucacasonato]

@uNetworkingAB unfortunately deleted their comments. For full transparency, here are the deleted comments:

---

Comment 1 posted at 16:47 UTC+2 by @uNetworkingAB

I've written the web server and I can tell you it supports routing, TLS, http3, and does check important headers. Just to clarify, because that explanation regarding the web server simply isn't true.

---

Comment 3 posted at 19:02 UTC+2 by @uNetworkingAB

"The Bun Http server" is uWebSockets. That's my project so I wanted to correct some misunderstandings.

There's always radix-tree routing performed, even if you don't see it. This includes wildcards and parameter matching. None of that is bypassed in bun (even if bun.serve doesn't expose it).
All headers are lowercased and put in a bloom filter so even if you don't access them they've already been processed and have O(1) access.
True it does not support http 2 but it does support http 3 (no idea why that is an argument but sure)
TLS 1.3 is perfectly supported and very fast
You definitely can stream large responses both in and out - that's not something deno does exclusively which you seem to imply by bringing it up.

---

Comment 5 posted at 19:15 UTC+2 by @uNetworkingAB

Right let's leave it on that and wait until said support is exposed, no interest in arguing just wanted to correct done assumptions. Alright carry on

[uNetworkingAB]

unfortunately deleted

Ah, you're not one of those worried about big tech having too much data. 😉 By all means, if you want to have this text stored we can have it stored but I thought it was nothing to store. Kind of silly thing to store if you ask me. Two men on the internet not reaching an agreement, who could have though - never happened before am I right?

[uNetworkingAB]

@lucacasonato

There are however things that Deno's HTTP server does, that Bun does not

Well for one, Deno is losing in HTTP performance while consuming 183% CPU-time while Bun is winning consuming 99% CPU-time so you're using almost twice the juice for half the journey. That's a pretty huge difference not accounted for. CPU-time normalize the results and you have a clear winner in Bun.

Secondly, just yesterday we saw Bun add both TLS and routing support (canary) where, like I have tried to point out, the routing performance already being accounted for since, like again I tried to point out, uWS always does routing. So there is no loss to be further accounted for there.

I'm sure we can get those 29 bytes of Date added to the response (and yes I agree that should have been done before you pointed it out).

[littledivy]

@uNetworkingAB Fair point and noted. There is also memory usage, latency, etc that are important metrics. Eg: Memory usage under heavy load for Bun is 200mb and Deno is below 35mb.

Both of these issues are not permenant ofc.

uWS always does routing

Sure its built in but that's not exposed to JS. A routing benchmark is when the consumer of the API is doing the routing. There is a routing bench in this repo that uses hono.

https://github.com/denoland/deno/blob/main/cli/bench/http/bun_hono_router.js
https://github.com/denoland/deno/blob/main/cli/bench/http/deno_flash_hono_router.js

[littledivy]

There is no point in a debating a "clear winner". Locking the discussion so its stays on the original topic. Please do report perf issues in the issue tracker.