Automated publish

blog/2024-10-14-llvm-ir/index.html

@@ -65,4 +65,4 @@
 <span class="line"><span style="color:#fd971f">  %3</span><span style="color:#f8f8f2"> = </span><span style="color:#f92672">call</span><span style="color:#66d9ef;font-style:italic"> ptr</span><span style="color:#fd971f"> @jank_deref</span><span style="color:#f8f8f2">(</span><span style="color:#66d9ef;font-style:italic">ptr</span><span style="color:#fd971f"> %2</span><span style="color:#f8f8f2">)</span></span>
 <span class="line"><span style="color:#fd971f">  %4</span><span style="color:#f8f8f2"> = </span><span style="color:#f92672">call</span><span style="color:#66d9ef;font-style:italic"> ptr</span><span style="color:#fd971f"> @jank_call2</span><span style="color:#f8f8f2">(</span><span style="color:#66d9ef;font-style:italic">ptr</span><span style="color:#fd971f"> %3</span><span style="color:#f8f8f2">, </span><span style="color:#66d9ef;font-style:italic">ptr</span><span style="color:#fd971f"> %1</span><span style="color:#f8f8f2">, </span><span style="color:#66d9ef;font-style:italic">ptr</span><span style="color:#fd971f"> %0</span><span style="color:#f8f8f2">)</span></span>
 <span class="line"><span style="color:#f92672">  ret</span><span style="color:#66d9ef;font-style:italic"> ptr</span><span style="color:#fd971f"> %4</span></span>
-<span class="line"><span style="color:#f8f8f2">}</span></span></code></pre><p>There's still more to do before I can get some real numbers for how long it takes to JIT compile LLVM IR, compared to C++. However, I'm very optimistic. By using a C API, instead of our C++ API, handling codegen optimizations like unboxing ends up being even more complex, but we also have even more power.<h2>How this affects interop</h2><p>Currently, jank has two forms of native interop (one in each direction):<ol><li>A special <code>native/raw</code> form which allows embedding C++ within your jank code<li>The ability to require a C++ as though it's a Clojure namespace, where that C++ code then uses jank's runtime to register types/functions</ol><p>When we're generating C++, a <code>native/raw</code> just gets code-generated right into place. However, when we're generating IR, we can't sanely do that without involving a C++ compiler. This means that <code>native/raw</code> will need to go away, to move forward with IR. However, I think this may be a good move. If we buy into the second form of interop more strongly, we can rely on actual native source files to reach into the jank runtime and register their types/functions. Then, in the actual jank code, everything feels like Clojure.<p>This means that we still have a need for JIT compiling C++. Whenever you <code>require</code> a module from your jank code, which is backed by a C++ file, that code is JIT compiled. Generally, what the C++ code will do is register the necessary functions into the jank runtime so that way you can then drive the rest of your program with jank code. I think this is a happy medium, where we still have the full power of C++ at our disposal, but all of our jank code will result in IR, which will JIT compile much more quickly than C++.<p>This means the answer to the question of C++ or IR is: <strong>why not both?</strong><h2>jank as THE native Clojure dialect</h2><p>There's another reason which leads me to explore LLVM IR within jank. While jank is embracing modern C++, it doesn't need to be so tightly coupled to it. By using just the C ABI as our runtime library, everything can talk to jank. You could talk to jank from Ruby, Lua, Rust, and even Clojure JVM. Just as importantly, jank can JIT compile any LLVM IR, which means any language which compiles on the LLVM stack can then be JIT compiled into your jank program.<p>Just as jank can load C++ files as required namespaces, seamlessly, so too could it do the same for Rust, in the future. Furthermore, as the public interface for jank becomes C, the internal representation and implementation can change opaquely, which would also open the door for more Rust within the jank compiler.<p>In short, <em>any native work you want to do in Clojure</em> should be suited for jank. Your jank code can remain Clojure, but you can package C, C++, and later languages like Rust inside your jank projects and require then from your jank code. The jank compiler and runtime will handle JIT compilation and AOT compilation for you.<h2>Community update</h2><p>This has been a long update which hopefully created some more excitement for jank's direction. I want to wrap up with what the community has been up to recently, though, since that alone warrants celebration.<h3>Characters, scientific notation, and to<i>code</i>string</h3><p><a href="https://github.com/Samy-33">Saket</a> has been improving jank's runtime character objects, which he originally implemented, to be more efficient and support Unicode. He also recently added scientific notation for floating point values, as well as an extension of jank's object concept to support <code>to_code_string</code>, which allows us to now implement <code>pr-str</code>.<p>At this point, Saket has the most knowledge of jank's internals, aside from me, so I've been giving him heftier tasks and he's been super helpful.<h3>More robust escape sequences</h3><p>One of my SciCloj mentees, <a href="https://github.com/jianlingzhong">Jianling</a>, recently merged support for all of the ASCII escape sequences for jank's strings. Previously, we only had rudimentary support. Now he's working on support for hexadecimal, octal, and arbitrary radix literals, to further jank's syntax parity with Clojure.<h3>Nix build</h3><p>We have a newcomer to jank, <a href="https://github.com/haruki7049">Haruki</a>, helping to rework the build system and dependencies to allow for easy building with Nix! There's a draft PR <a href="https://github.com/jank-lang/jank/pull/94">here</a>. I'm excited for this, since I'm currently using NixOS and I need to do a lot of jank dev in a distrobox for easy building. This will also help with stable CI builds and ultimately getting jank into nixpkgs (the central package repo for Nix).<h3>LLVM 19 support</h3><p>The last JIT hard crash fix in LLVM is being backported to the 19.x branch, which means we should be able to start using Clang/LLVM binaries starting 19.2! This is going to drastically simplify the developer experience and allow for packaging jank using the system Clang/LLVM install. My <a href="https://github.com/llvm/llvm-project/issues/111068">backport ticket</a> has been closed as complete, though <a href="https://github.com/llvm/llvm-project/pull/111953">the PR</a> into the 19.x branch is still open.<h2>Summary</h2><p>More people are working on jank now than ever have; I expect this number to keep growing in the coming year. I'll see you folks at the Conj and, after that, in my next update during the holiday season, when I'll have some final numbers comparing jank's startup times with LLVM IR vs C++, as well as some updates on other things I've been tackling.<h2>Would you like to join in?</h2><ol><li>Join the community on <a href="https://clojurians.slack.com/archives/C03SRH97FDK">Slack</a><li>Join the design discussions or pick up a ticket on <a href="https://github.com/jank-lang/jank">GitHub</a><li>Considering becoming a <a href="https://github.com/sponsors/jeaye">Sponsor</a> <span class="icon mr-1"style="color:#c96198"><i class="gg-heart"></i></span><li><strong>Hire me full-time to work on jank!</strong></ol></div></div></section></div><footer class="footer"><div class="container"><div class="columns has-text-centered"></div><div class="container has-text-centered"><div class="content is-small"><p>© 2024 Jeaye Wilkerson | All rights reserved.</div></div></div></footer><noscript><p><img src="//matomo.jeaye.com/matomo.php?idsite=1&amp;rec=1"style="border:0"alt=""></p></noscript><script>for(var coll=document.getElementsByClassName("collapsible"),i=0;i<coll.length;i++)coll[i].addEventListener("click",function(){this.classList.toggle("active");var l=this.nextElementSibling;"block"===l.style.display?l.style.display="none":l.style.display="block"})</script>
+<span class="line"><span style="color:#f8f8f2">}</span></span></code></pre><p>There's still more to do before I can get some real numbers for how long it takes to JIT compile LLVM IR, compared to C++. However, I'm very optimistic. By using a C API, instead of our C++ API, handling codegen optimizations like unboxing ends up being even more complex, but we also have even more power.<h2>How this affects interop</h2><p>Currently, jank has two forms of native interop (one in each direction):<ol><li>A special <code>native/raw</code> form which allows embedding C++ within your jank code<li>The ability to require a C++ as though it's a Clojure namespace, where that C++ code then uses jank's runtime to register types/functions</ol><p>When we're generating C++, a <code>native/raw</code> just gets code-generated right into place. However, when we're generating IR, we can't sanely do that without involving a C++ compiler. This means that <code>native/raw</code> will need to go away, to move forward with IR. However, I think this may be a good move. If we buy into the second form of interop more strongly, we can rely on actual native source files to reach into the jank runtime and register their types/functions. Then, in the actual jank code, everything feels like Clojure.<p>This means that we still have a need for JIT compiling C++. Whenever you <code>require</code> a module from your jank code, which is backed by a C++ file, that code is JIT compiled. Generally, what the C++ code will do is register the necessary functions into the jank runtime so that way you can then drive the rest of your program with jank code. I think this is a happy medium, where we still have the full power of C++ at our disposal, but all of our jank code will result in IR, which will JIT compile much more quickly than C++.<p>This means the answer to the question of C++ or IR is: <strong>why not both?</strong><h2>jank as THE native Clojure dialect</h2><p>There's another reason which leads me to explore LLVM IR within jank. While jank is embracing modern C++, it doesn't need to be so tightly coupled to it. By using just the C ABI as our runtime library, everything can talk to jank. You could talk to jank from Ruby, Lua, Rust, and even Clojure JVM. Just as importantly, jank can JIT compile any LLVM IR, which means any language which compiles on the LLVM stack can then be JIT compiled into your jank program.<p>Just as jank can load C++ files as required namespaces, seamlessly, so too could it do the same for Rust, in the future. Furthermore, as the public interface for jank becomes C, the internal representation and implementation can change opaquely, which would also open the door for more Rust within the jank compiler.<p>In short, <em>any native work you want to do in Clojure</em> should be suited for jank. Your jank code can remain Clojure, but you can package C, C++, and later languages like Rust inside your jank projects and require then from your jank code. The jank compiler and runtime will handle JIT compilation and AOT compilation for you.<h2>Community update</h2><p>This has been a long update which hopefully created some more excitement for jank's direction. I want to wrap up with what the community has been up to recently, though, since that alone warrants celebration.<h3>Characters, scientific notation, and <code>to_code_string</code></h3><p><a href="https://github.com/Samy-33">Saket</a> has been improving jank's runtime character objects, which he originally implemented, to be more efficient and support Unicode. He also recently added scientific notation for floating point values, as well as an extension of jank's object concept to support <code>to_code_string</code>, which allows us to now implement <code>pr-str</code>.<p>At this point, Saket has the most knowledge of jank's internals, aside from me, so I've been giving him heftier tasks and he's been super helpful.<h3>More robust escape sequences</h3><p>One of my SciCloj mentees, <a href="https://github.com/jianlingzhong">Jianling</a>, recently merged support for all of the ASCII escape sequences for jank's strings. Previously, we only had rudimentary support. Now he's working on support for hexadecimal, octal, and arbitrary radix literals, to further jank's syntax parity with Clojure.<h3>Nix build</h3><p>We have a newcomer to jank, <a href="https://github.com/haruki7049">Haruki</a>, helping to rework the build system and dependencies to allow for easy building with Nix! There's a draft PR <a href="https://github.com/jank-lang/jank/pull/94">here</a>. I'm excited for this, since I'm currently using NixOS and I need to do a lot of jank dev in a distrobox for easy building. This will also help with stable CI builds and ultimately getting jank into nixpkgs (the central package repo for Nix).<h3>LLVM 19 support</h3><p>The last JIT hard crash fix in LLVM is being backported to the 19.x branch, which means we should be able to start using Clang/LLVM binaries starting 19.2! This is going to drastically simplify the developer experience and allow for packaging jank using the system Clang/LLVM install. My <a href="https://github.com/llvm/llvm-project/issues/111068">backport ticket</a> has been closed as complete, though <a href="https://github.com/llvm/llvm-project/pull/111953">the PR</a> into the 19.x branch is still open.<h2>Summary</h2><p>More people are working on jank now than ever have; I expect this number to keep growing in the coming year. I'll see you folks at the Conj and, after that, in my next update during the holiday season, when I'll have some final numbers comparing jank's startup times with LLVM IR vs C++, as well as some updates on other things I've been tackling.<h2>Would you like to join in?</h2><ol><li>Join the community on <a href="https://clojurians.slack.com/archives/C03SRH97FDK">Slack</a><li>Join the design discussions or pick up a ticket on <a href="https://github.com/jank-lang/jank">GitHub</a><li>Considering becoming a <a href="https://github.com/sponsors/jeaye">Sponsor</a> <span class="icon mr-1"style="color:#c96198"><i class="gg-heart"></i></span><li><strong>Hire me full-time to work on jank!</strong></ol></div></div></section></div><footer class="footer"><div class="container"><div class="columns has-text-centered"></div><div class="container has-text-centered"><div class="content is-small"><p>© 2024 Jeaye Wilkerson | All rights reserved.</div></div></div></footer><noscript><p><img src="//matomo.jeaye.com/matomo.php?idsite=1&amp;rec=1"style="border:0"alt=""></p></noscript><script>for(var coll=document.getElementsByClassName("collapsible"),i=0;i<coll.length;i++)coll[i].addEventListener("click",function(){this.classList.toggle("active");var l=this.nextElementSibling;"block"===l.style.display?l.style.display="none":l.style.display="block"})</script>