I was previously giving updates only in the #jank Slack channel, but some of these are getting large enough to warrant more prose. Thus, happily, I can announce that jank has a new blog and I have a lot of new progress to report! Let's get into the details.
Over the past couple of months, jank has gained initial C++ interop, an upgrade of Cling to resolve Linux crashes, support for multiple fn arities, including variadic arities, support for if and let*, which involved a rework of the C++ codegen, and initial metadata support, and many smaller changes.
Before I get into the details, I'd like to point out that feedback is welcome on all of this. Very little of jank's design is set in stone right now, so that means your feedback can matter a lot.
Initial C++ interop
When digging deep enough into clojure.core implementations, most things eventually just turn into Java. Crucial fns like println, for example, rely on Java streams, as well as direct interop. Something like str, as well, reaches right for StringBuilder. So jank needs a way to call into C++ to get the important things done.
C++ is syntactically and semantically more complicated than Java. Its reflection story is poorer and is made worse by template usage, which requires a C++ compiler to generate the necessary instantiations ahead of time. With all of this in mind, I figured jank's interop is not going to look like Clojure JVM's, or ClojureScript's. I did dive into what Ferret and Carp are doing for this and came up with some design notes that may be of interest. The notes cover both approaches, some pros/cons, and the rationale for jank's approach.
In short, jank is introducing a special native/raw form which accepts inline C++ code. Within that code can be interpolated jank code, escaped using the #{} form.
Example
; No macros yet, so no defn or fn.
+jank development update - Lots of new changes
I was previously giving updates only in the #jank Slack channel, but some of these are getting large enough to warrant more prose. Thus, happily, I can announce that jank has a new blog and I have a lot of new progress to report! Let's get into the details.
Over the past couple of months, jank has gained initial C++ interop, an upgrade of Cling to resolve Linux crashes, support for multiple fn arities, including variadic arities, support for if and let*, which involved a rework of the C++ codegen, and initial metadata support, and many smaller changes.
Before I get into the details, I'd like to point out that feedback is welcome on all of this. Very little of jank's design is set in stone right now, so that means your feedback can matter a lot.
Initial C++ interop
When digging deep enough into clojure.core implementations, most things eventually just turn into Java. Crucial fns like println, for example, rely on Java streams, as well as direct interop. Something like str, as well, reaches right for StringBuilder. So jank needs a way to call into C++ to get the important things done.
C++ is syntactically and semantically more complicated than Java. Its reflection story is poorer and is made worse by template usage, which requires a C++ compiler to generate the necessary instantiations ahead of time. With all of this in mind, I figured jank's interop is not going to look like Clojure JVM's, or ClojureScript's. I did dive into what Ferret and Carp are doing for this and came up with some design notes that may be of interest. The notes cover both approaches, some pros/cons, and the rationale for jank's approach.
In short, jank is introducing a special native/raw form which accepts inline C++ code. Within that code can be interpolated jank code, escaped using the #{} form.
Example
; No macros yet, so no defn or fn.(def str (fn* ([]
diff --git a/blog/2023-01-13-optimizing-sequences/index.html b/blog/2023-01-13-optimizing-sequences/index.html
index 3004c6b..c477bef 100644
--- a/blog/2023-01-13-optimizing-sequences/index.html
+++ b/blog/2023-01-13-optimizing-sequences/index.html
@@ -1,4 +1,4 @@
-jank development update - Optimizing sequences
In this episode of jank's development updates, we follow an exciting few weekends as I was digging deep into Clojure's sequence implementation, building jank's equivalent, and then benchmarking and profiling in a dizzying race to the bottom.
Introduction
Not expecting a rabbit hole, I was originally surprised at how many allocations are involved in a normal sequence iteration in Clojure and thought to optimize that in jank. In fact, Clojure allocates a new sequence for every element over which it iterates!
Clojure's interface for sequences looks like this (link):
public interface ISeq extends IPersistentCollection
+jank development update - Optimizing sequences
In this episode of jank's development updates, we follow an exciting few weekends as I was digging deep into Clojure's sequence implementation, building jank's equivalent, and then benchmarking and profiling in a dizzying race to the bottom.
Introduction
Not expecting a rabbit hole, I was originally surprised at how many allocations are involved in a normal sequence iteration in Clojure and thought to optimize that in jank. In fact, Clojure allocates a new sequence for every element over which it iterates!
Clojure's interface for sequences looks like this (link):
public interface ISeq extends IPersistentCollection{ /* Returns the current front element of the sequence. */ Object first();
diff --git a/blog/2023-04-07-ray-tracing/index.html b/blog/2023-04-07-ray-tracing/index.html
index 1dfe170..e9eb6f2 100644
--- a/blog/2023-04-07-ray-tracing/index.html
+++ b/blog/2023-04-07-ray-tracing/index.html
@@ -1,4 +1,4 @@
-jank development update - Optimizing a ray tracer
After the last post, which focused on optimizing jank's sequences, I wanted to get jank running a ray tracer I had previously written in Clojure. In this post, I document what was required to start ray tracing in jank and, more importantly, how I chased down the run time in a fierce battle with Clojure's performance.
Missing Clojure functions
Coming out of the last blog post, there were quite a few functions which the ray tracer required that jank did not yet have. A lot of this was tedium, but there are some interesting points.
Polymorphic arithmetic
In Clojure JVM, since everything can be an object, and Clojure's dynamically typed, we can't know what something like (+ a b) actually does. For example, it's possible that either a or b is not a number, but it's also possible that they're an unboxed long or double, or a boxed Long or a Double, or maybe even a BigInteger or Ratio. Each of these will handle + slightly differently. Clojure (and now jank) handles this using a neat polymorphic design. In jank, it starts with this number_ops interface:
struct number_ops
+jank development update - Optimizing a ray tracer
After the last post, which focused on optimizing jank's sequences, I wanted to get jank running a ray tracer I had previously written in Clojure. In this post, I document what was required to start ray tracing in jank and, more importantly, how I chased down the run time in a fierce battle with Clojure's performance.
Missing Clojure functions
Coming out of the last blog post, there were quite a few functions which the ray tracer required that jank did not yet have. A lot of this was tedium, but there are some interesting points.
Polymorphic arithmetic
In Clojure JVM, since everything can be an object, and Clojure's dynamically typed, we can't know what something like (+ a b) actually does. For example, it's possible that either a or b is not a number, but it's also possible that they're an unboxed long or double, or a boxed Long or a Double, or maybe even a BigInteger or Ratio. Each of these will handle + slightly differently. Clojure (and now jank) handles this using a neat polymorphic design. In jank, it starts with this number_ops interface:
This quarter, my work on jank is being sponsored by Clojurists Together. The terms of the work are to research a new object model for jank, with the goal of making jank code faster across the board. This is a half-way report and I'm excited to share my results!
The problem
Before getting into any solutions, or celebrating any wins, we need to talk about why this work is being done at all. As you can see in my previous development updates, jank is fast. It can beat Clojure in each benchmark I've published so far. However, some parts of jank's runtime are still quite slow and, unfortunately, the problem is systemic.
Generally speaking, the problem can be boiled down to this: the JVM is ridiculously fast at allocations. That's important, since Clojure is, to put it nicely, very liberal with its allocations. jank overcomes this, in some ways, by just allocating a whole lot less. Still, each allocation pushes Clojure ahead in benchmarks and it adds up.
So, JVM allocations are fast, but why are jank's slow? To understand this requires an understanding of C++ inheritance and virtual function tables (vtables), so let's cover that at an implementation level.
Virtual function tables
Clojure is thoroughly polymorphic. Everything is an Object, which can then have any number of interfaces it implements, all of which can be extended, checked at run-time, etc. To accomplish this, in C++, I modeled the objects quite closely to how they are in Clojure's Java runtime. Let's take a look.
Let's take a stripped down jank base object:
struct jank_object : gc
+jank development update - A faster object model
This quarter, my work on jank is being sponsored by Clojurists Together. The terms of the work are to research a new object model for jank, with the goal of making jank code faster across the board. This is a half-way report and I'm excited to share my results!
The problem
Before getting into any solutions, or celebrating any wins, we need to talk about why this work is being done at all. As you can see in my previous development updates, jank is fast. It can beat Clojure in each benchmark I've published so far. However, some parts of jank's runtime are still quite slow and, unfortunately, the problem is systemic.
Generally speaking, the problem can be boiled down to this: the JVM is ridiculously fast at allocations. That's important, since Clojure is, to put it nicely, very liberal with its allocations. jank overcomes this, in some ways, by just allocating a whole lot less. Still, each allocation pushes Clojure ahead in benchmarks and it adds up.
So, JVM allocations are fast, but why are jank's slow? To understand this requires an understanding of C++ inheritance and virtual function tables (vtables), so let's cover that at an implementation level.
Virtual function tables
Clojure is thoroughly polymorphic. Everything is an Object, which can then have any number of interfaces it implements, all of which can be extended, checked at run-time, etc. To accomplish this, in C++, I modeled the objects quite closely to how they are in Clojure's Java runtime. Let's take a look.
As summer draws to a close, in the Pacific Northwest, so too does my term of sponsored work focused on a faster object model for jank. Thanks so much to Clojurists Together for funding jank's development. The past quarter has been quite successful and I'm excited to share the results.
If you haven't yet read my previous post, which goes over why I'm overhauling jank's object model, and how I'm doing it, take a look! Without that, I suppose you could still continue, if you enjoy looking at the results of unknown problems. Just know the problem is interesting and the results are impressive.
Overview of changes
These changes spanned almost the entire code base. I think only the lexer was left unchanged, since it deals only with tokens and not runtime Clojure objects. From the parser, through semantic analysis and JIT, and into every runtime function, basically every operation on objects needed changes. I've made a pull request on the jank repo so that these changes can be both quantified and reviewed, by the daring reader: here.
Overall, it's currently at 8,634 added lines and 4,380 deleted lines, across 123 files. Indeed, the new object model lends itself to more code, and somewhat longer compile times, but I think the results are worth it.
What follows is a lot of benchmarking graphs, each covering Clojure JVM, jank prior to this quarter's work, and jank after all of this work. For all graphs, lower is better.
Overall ray tracing speeds
The ray tracer used in the past couple of posts has been my primary benchmark for the overall performance of the new object model, since it relies heavily on maps, numbers, sequence traversal and strings (for output). I'm very pleased to report that jank is now nearly twice as fast at running the same ray tracing code as Clojure JVM, with jank clocking in at 36.96ms versus Clojure's 69.44ms. Since jank was only marginally faster than Clojure at the end of the last post, this also means the improvements in the past quarter have been nearly 2x overall.
This is the primary celebration and is the culmination of a handful of months worth of work, spanning back before I started this object model redesign. When I could first run the ray tracer, two blog posts ago (5 months ago), jank took 797.49ms to run the exact same code!
A lot has changed in the past 5 months. Before I get to where jank will be in the next 5 months, though, let's dig deeper into some of the benchmark results.
Maps
The previous post showed that jank had nearly caught up with Clojure in terms of array map allocation speed. This hasn't changed since then, primarily because I had already pushed map allocations as far as I can for now, with my prototype. The final numbers are 16ns for Clojure and 17ns for jank. I'll be following up on this, at a later time, by introducing a new GC (via MMTK), instead of Boehm.
Map lookups were already fast, but have been made twice as fast still.
Vectors
Vector allocation speeds have been improved, but were quite slow to start with. jank's vectors are backed by immer's persistent vectors and this allocation is using the provided initializer list constructor. Clearly some work will be needed here, possibly requiring changes to immer. The improvements we see are solely due to the new object model being faster to allocate, since no other changes were made.
It's also worth noting that Clojure JVM has some very efficient ways to construct vectors which jank does not have. I'm not sure I can do this without exposing some internals of immer, but it will likely be worth it, since those Clojure JVM constructors can run in under 20ns. The one I'm showing here is the constructor closest to what jank is doing (taking in an initializer list).
Similar to maps, vector lookups were already quick and have nearly doubled in speed.
Strings
jank's strings lag significantly behind Clojure JVM's. This is the most glaring performance difference between the two. The new object model improves this, but more work needs to be done. jank is currently using folly's string, which is compliant with std::string but generally faster. However, folly's string is using jemalloc, rather than Boehm, which means both that jank is currently leaking string memory and also that allocations may be slower than with Boehm. On top of that, folly strings have proven to be fast to use, but slow to construct. I have work planned to provide a custom string instead.
I have included both short string and long string benchmarks here, since I know that folly's implementation uses a short string optimization which avoids allocations and stores the string data in situ. Still, it's much slower than Clojure JVM. JVM strings may be magic, but we'll see when I look into it.
Fast math
Math has sped up the most out of anything, which bodes very well for our ray tracing numbers. Here are the results for fully boxed subtraction, where no type info is known, subtraction between an unknown box and an unboxed double, and fully unboxed subtraction. In all cases, jank is now significantly faster than Clojure JVM. These wins apply across the board for all binary math operations.
Next quarter
This is the last performance-oriented bout of work for a while. jank is where it needs to be, I think, in order for me to start investing more in pushing the compiler and runtime features closer to parity with Clojure JVM. I'm very happy to share that Clojurists Together is actually sponsoring jank development again, for the upcoming quarter. The sponsored work will be focused on building out jank's module system, implementing clojure.core/require, preparing for iterative compilation, and setting the stage for AOT compilation and leiningen integration.
After this work, using jank for multi-file projects will be possible. Soon after that, I hope, we can start using leiningen to manage jank projects. This will mean adventurous devs can start actually using jank themselves, which I expect will only add to the momentum I currently have.
As summer draws to a close, in the Pacific Northwest, so too does my term of sponsored work focused on a faster object model for jank. Thanks so much to Clojurists Together for funding jank's development. The past quarter has been quite successful and I'm excited to share the results.
If you haven't yet read my previous post, which goes over why I'm overhauling jank's object model, and how I'm doing it, take a look! Without that, I suppose you could still continue, if you enjoy looking at the results of unknown problems. Just know the problem is interesting and the results are impressive.
Overview of changes
These changes spanned almost the entire code base. I think only the lexer was left unchanged, since it deals only with tokens and not runtime Clojure objects. From the parser, through semantic analysis and JIT, and into every runtime function, basically every operation on objects needed changes. I've made a pull request on the jank repo so that these changes can be both quantified and reviewed, by the daring reader: here.
Overall, it's currently at 8,634 added lines and 4,380 deleted lines, across 123 files. Indeed, the new object model lends itself to more code, and somewhat longer compile times, but I think the results are worth it.
What follows is a lot of benchmarking graphs, each covering Clojure JVM, jank prior to this quarter's work, and jank after all of this work. For all graphs, lower is better.
Overall ray tracing speeds
The ray tracer used in the past couple of posts has been my primary benchmark for the overall performance of the new object model, since it relies heavily on maps, numbers, sequence traversal and strings (for output). I'm very pleased to report that jank is now nearly twice as fast at running the same ray tracing code as Clojure JVM, with jank clocking in at 36.96ms versus Clojure's 69.44ms. Since jank was only marginally faster than Clojure at the end of the last post, this also means the improvements in the past quarter have been nearly 2x overall.
This is the primary celebration and is the culmination of a handful of months worth of work, spanning back before I started this object model redesign. When I could first run the ray tracer, two blog posts ago (5 months ago), jank took 797.49ms to run the exact same code!
A lot has changed in the past 5 months. Before I get to where jank will be in the next 5 months, though, let's dig deeper into some of the benchmark results.
Maps
The previous post showed that jank had nearly caught up with Clojure in terms of array map allocation speed. This hasn't changed since then, primarily because I had already pushed map allocations as far as I can for now, with my prototype. The final numbers are 16ns for Clojure and 17ns for jank. I'll be following up on this, at a later time, by introducing a new GC (via MMTK), instead of Boehm.
Map lookups were already fast, but have been made twice as fast still.
Vectors
Vector allocation speeds have been improved, but were quite slow to start with. jank's vectors are backed by immer's persistent vectors and this allocation is using the provided initializer list constructor. Clearly some work will be needed here, possibly requiring changes to immer. The improvements we see are solely due to the new object model being faster to allocate, since no other changes were made.
It's also worth noting that Clojure JVM has some very efficient ways to construct vectors which jank does not have. I'm not sure I can do this without exposing some internals of immer, but it will likely be worth it, since those Clojure JVM constructors can run in under 20ns. The one I'm showing here is the constructor closest to what jank is doing (taking in an initializer list).
Similar to maps, vector lookups were already quick and have nearly doubled in speed.
Strings
jank's strings lag significantly behind Clojure JVM's. This is the most glaring performance difference between the two. The new object model improves this, but more work needs to be done. jank is currently using folly's string, which is compliant with std::string but generally faster. However, folly's string is using jemalloc, rather than Boehm, which means both that jank is currently leaking string memory and also that allocations may be slower than with Boehm. On top of that, folly strings have proven to be fast to use, but slow to construct. I have work planned to provide a custom string instead.
I have included both short string and long string benchmarks here, since I know that folly's implementation uses a short string optimization which avoids allocations and stores the string data in situ. Still, it's much slower than Clojure JVM. JVM strings may be magic, but we'll see when I look into it.
Fast math
Math has sped up the most out of anything, which bodes very well for our ray tracing numbers. Here are the results for fully boxed subtraction, where no type info is known, subtraction between an unknown box and an unboxed double, and fully unboxed subtraction. In all cases, jank is now significantly faster than Clojure JVM. These wins apply across the board for all binary math operations.
Next quarter
This is the last performance-oriented bout of work for a while. jank is where it needs to be, I think, in order for me to start investing more in pushing the compiler and runtime features closer to parity with Clojure JVM. I'm very happy to share that Clojurists Together is actually sponsoring jank development again, for the upcoming quarter. The sponsored work will be focused on building out jank's module system, implementing clojure.core/require, preparing for iterative compilation, and setting the stage for AOT compilation and leiningen integration.
After this work, using jank for multi-file projects will be possible. Soon after that, I hope, we can start using leiningen to manage jank projects. This will mean adventurous devs can start actually using jank themselves, which I expect will only add to the momentum I currently have.
For those readers interested in my benchmark code, both the C++ (jank) and Clojure JVM versions are provided in this gist: here.
All benchmarks were done on my Arch Linux desktop with a AMD Ryzen Threadripper 2950X using OpenJDK 11 with the G1 GC.
\ No newline at end of file
diff --git a/blog/2023-10-14-module-loading/index.html b/blog/2023-10-14-module-loading/index.html
index e441aec..e432383 100644
--- a/blog/2023-10-14-module-loading/index.html
+++ b/blog/2023-10-14-module-loading/index.html
@@ -1 +1 @@
-jank development update - Module loading
For the past month and a half, I've been building out jank's support for clojure.core/require, including everything from class path handling to compiling jank files to intermediate code written to the filesystem. This is a half-way report for the quarter. As a warm note, my work on jank this quarter is being sponsored by Clojurists Together.
High level requirements
Class paths
Clojure JVM benefits a great deal from being built upon an existing VM. In the native world, we don't have things like class paths. Maybe the closest things would be include paths at compile-time and LD_LIBRARY_PATH at run-time, but neither of those capture the flexibility of JVM's class paths, which work at both compile-time and run-time.
So, to start with, jank needs a similar system. This is a common pattern for jank, requiring me to implement not just Clojure, but a VM of my own, with the necessary parts to reach parity.
Progress
I've built out class path traversing for jank, which supports both directories and JAR files. This will allow jank to work out of the box with Clojure's existing Maven dependencies and file structures, which is of course important.
jank traverses the class path exhaustively on startup and caches what it finds, mapping the module name (ns name or ns name with a nested class, like clojure.core$foo) to the relevant file. When a function like require or compile is called, jank will find the most relevant source to work with.
Core functions
There are a handful of related clojure.core functions for module loading, like require, compile, load-libs, load-lib, load-one, load-all, alias, etc. The next step, after having class path support, is to implement these.
Progress
I have a working implementation of (require 'clojure.core) and (compile 'clojure.core) now! They hook into the class path work and do the necessary work to require or compile. Compilation writes files to a particular directory, which is also in the class path. Requiring a module which is already loaded will not do anything.
There's still a lot of work to do to build out the necessary core functions and have them work the same as in Clojure JVM. The implementations of require and compile that I have right now only accept a single symbol, rather than being variadic, supporting lib specs, flags, etc. So this is still an MVP, right now, but it works!
Class files
There's no such thing as a class file in the native world. Maybe the closest equivalent would be an object file or, for C++20, a pre-compiled module. Those are both more limiting than a class file, though, since they're not portable; compiled native code is generally targeting a specific platform/architecture. Trying to share these in a Maven dependency, for example, is only going to help those who are on the same hardware as you. Even then, we can run into ABI incompatibilities.
So, while I'm interested in exploring support for intermediate object files and pre-compiled modules, I'm starting with intermediate files being just C++ source (which is what the jank compiler outputs for Cling to JIT compile). From there, another step toward machine code will be to target LLVM IR by having Clang compile the C++ source first. This is closer to JVM byte code, but LLVM IR is actually still platform/architecture specific!
Lastly, I'm very hesitant to provide a default of jank dependencies coming in as binary files, even if I can solve the portability problem, simply due to supply chain security concerns. I would rather live in a world where people share source dependencies with pinned versions and transparent updates. I do think that binary intermediate files make sense for local development, though, and they can greatly speed up iteration.
Progress
As of now, I have (compile 'clojure.core) compiling jank source to C++ source, which is being written to the class path. If you then later try to (require 'clojure.core), it will be loaded from the compiled C++ source. If the C++ source was on the class path already, it will be favored over the jank source.
One benefit of this implementation is that jank developers can include arbitrary C++ source along with their jank source and just require it alongside everything else. In order to work with this, the C++ source just needs to follow a particular interface.
A challenge I ran into with this is how to manage module dependencies. For example, if clojure.core depends on clojure.core$take, which depends on a local fn its own, clojure.core$take$fn_478, I need to ensure that all of these are loaded in order of deepest dependency (leaf node) first. I went back on forth on the design for this, but ultimately settled on something similar to what Clojure does. I generate two C++ source modules for clojure.core itself. One is something like classes/clojure.core.cpp and the other is a special classes/clojure.core__init.cpp. When clojure.core is required, it will look for a clojure.core__init module first. Within that module is a special interface with an __init function which has a big list of all of the dependencies needed to actually load clojure.core. The __init function will just iterate through that list and load each one. Finally, we can actually load clojure.core, which runs the top-level effects of creating all of the vars, the value for each being based on new types brought in from the dependencies.
This is different from Clojure, since the JVM has a standard way for one module to depend on another. That dependency is just conveyed, like using import in Java, and then the JVM ensures all dependencies are met before getting to the body of the module. Again, I need to reimplement that portion of the JVM for jank since the native world has no equivalent feature.
What's remaining
Iterative compilation (tracking source timestamps to know how much to recompile) and support for reloading have not been touched yet. Aside from that, most things I have implemented are quite rough and need further polish to meet parity with Clojure. Although I have require and compile working in the simple case, none of the other related core functions have been implemented.
Performance wins so far
By pre-compiling clojure.core to C++ source, and then just requiring it on startup, the time it takes to boot the jank compiler + runtime and print hello world dropped from 8.7 seconds to 3.7 seconds. So that was all time spent compiling jank code to C++ code. What remains is almost entirely just time compiling C++. If I remove clojure.core loading altogether, it takes less than 0.2 seconds to run the same program. I'll be digging more into the performance here, as I get more implemented, but I want to call out a couple of things.
We've already cut 5 seconds down, which is great!
Everyone knows that compiling C++ is not fast and we are set up to be able to start loading LLVM IR instead, after some more work
The creator of Cling informed me that LLVM tends to spend around 50% of its time in the front-end for C++, which means that by using LLVM IR we'll be cutting down our compilation time by around 50%
I haven't done any startup time benchmarking or profiling for jank yet, but if there's time this quarter, you can bet that I'll be digging deep into this
I have some exciting plans for visualizing jank's performance, both the compiler and your application code, in a way which will ship with the compiler itself. More info on this in a later post.
Thanks again
As a reminder, my work on jank this quarter is sponsored by Clojurists Together. Thank you to all of the members there who chose jank for this quarter. Thanks, also, to all of my Github sponsors. Your continued support fuels jank's continued development!
For the past month and a half, I've been building out jank's support for clojure.core/require, including everything from class path handling to compiling jank files to intermediate code written to the filesystem. This is a half-way report for the quarter. As a warm note, my work on jank this quarter is being sponsored by Clojurists Together.
High level requirements
Class paths
Clojure JVM benefits a great deal from being built upon an existing VM. In the native world, we don't have things like class paths. Maybe the closest things would be include paths at compile-time and LD_LIBRARY_PATH at run-time, but neither of those capture the flexibility of JVM's class paths, which work at both compile-time and run-time.
So, to start with, jank needs a similar system. This is a common pattern for jank, requiring me to implement not just Clojure, but a VM of my own, with the necessary parts to reach parity.
Progress
I've built out class path traversing for jank, which supports both directories and JAR files. This will allow jank to work out of the box with Clojure's existing Maven dependencies and file structures, which is of course important.
jank traverses the class path exhaustively on startup and caches what it finds, mapping the module name (ns name or ns name with a nested class, like clojure.core$foo) to the relevant file. When a function like require or compile is called, jank will find the most relevant source to work with.
Core functions
There are a handful of related clojure.core functions for module loading, like require, compile, load-libs, load-lib, load-one, load-all, alias, etc. The next step, after having class path support, is to implement these.
Progress
I have a working implementation of (require 'clojure.core) and (compile 'clojure.core) now! They hook into the class path work and do the necessary work to require or compile. Compilation writes files to a particular directory, which is also in the class path. Requiring a module which is already loaded will not do anything.
There's still a lot of work to do to build out the necessary core functions and have them work the same as in Clojure JVM. The implementations of require and compile that I have right now only accept a single symbol, rather than being variadic, supporting lib specs, flags, etc. So this is still an MVP, right now, but it works!
Class files
There's no such thing as a class file in the native world. Maybe the closest equivalent would be an object file or, for C++20, a pre-compiled module. Those are both more limiting than a class file, though, since they're not portable; compiled native code is generally targeting a specific platform/architecture. Trying to share these in a Maven dependency, for example, is only going to help those who are on the same hardware as you. Even then, we can run into ABI incompatibilities.
So, while I'm interested in exploring support for intermediate object files and pre-compiled modules, I'm starting with intermediate files being just C++ source (which is what the jank compiler outputs for Cling to JIT compile). From there, another step toward machine code will be to target LLVM IR by having Clang compile the C++ source first. This is closer to JVM byte code, but LLVM IR is actually still platform/architecture specific!
Lastly, I'm very hesitant to provide a default of jank dependencies coming in as binary files, even if I can solve the portability problem, simply due to supply chain security concerns. I would rather live in a world where people share source dependencies with pinned versions and transparent updates. I do think that binary intermediate files make sense for local development, though, and they can greatly speed up iteration.
Progress
As of now, I have (compile 'clojure.core) compiling jank source to C++ source, which is being written to the class path. If you then later try to (require 'clojure.core), it will be loaded from the compiled C++ source. If the C++ source was on the class path already, it will be favored over the jank source.
One benefit of this implementation is that jank developers can include arbitrary C++ source along with their jank source and just require it alongside everything else. In order to work with this, the C++ source just needs to follow a particular interface.
A challenge I ran into with this is how to manage module dependencies. For example, if clojure.core depends on clojure.core$take, which depends on a local fn its own, clojure.core$take$fn_478, I need to ensure that all of these are loaded in order of deepest dependency (leaf node) first. I went back on forth on the design for this, but ultimately settled on something similar to what Clojure does. I generate two C++ source modules for clojure.core itself. One is something like classes/clojure.core.cpp and the other is a special classes/clojure.core__init.cpp. When clojure.core is required, it will look for a clojure.core__init module first. Within that module is a special interface with an __init function which has a big list of all of the dependencies needed to actually load clojure.core. The __init function will just iterate through that list and load each one. Finally, we can actually load clojure.core, which runs the top-level effects of creating all of the vars, the value for each being based on new types brought in from the dependencies.
This is different from Clojure, since the JVM has a standard way for one module to depend on another. That dependency is just conveyed, like using import in Java, and then the JVM ensures all dependencies are met before getting to the body of the module. Again, I need to reimplement that portion of the JVM for jank since the native world has no equivalent feature.
What's remaining
Iterative compilation (tracking source timestamps to know how much to recompile) and support for reloading have not been touched yet. Aside from that, most things I have implemented are quite rough and need further polish to meet parity with Clojure. Although I have require and compile working in the simple case, none of the other related core functions have been implemented.
Performance wins so far
By pre-compiling clojure.core to C++ source, and then just requiring it on startup, the time it takes to boot the jank compiler + runtime and print hello world dropped from 8.7 seconds to 3.7 seconds. So that was all time spent compiling jank code to C++ code. What remains is almost entirely just time compiling C++. If I remove clojure.core loading altogether, it takes less than 0.2 seconds to run the same program. I'll be digging more into the performance here, as I get more implemented, but I want to call out a couple of things.
We've already cut 5 seconds down, which is great!
Everyone knows that compiling C++ is not fast and we are set up to be able to start loading LLVM IR instead, after some more work
The creator of Cling informed me that LLVM tends to spend around 50% of its time in the front-end for C++, which means that by using LLVM IR we'll be cutting down our compilation time by around 50%
I haven't done any startup time benchmarking or profiling for jank yet, but if there's time this quarter, you can bet that I'll be digging deep into this
I have some exciting plans for visualizing jank's performance, both the compiler and your application code, in a way which will ship with the compiler itself. More info on this in a later post.
Thanks again
As a reminder, my work on jank this quarter is sponsored by Clojurists Together. Thank you to all of the members there who chose jank for this quarter. Thanks, also, to all of my Github sponsors. Your continued support fuels jank's continued development!
\ No newline at end of file
diff --git a/blog/2023-12-17-module-loading/index.html b/blog/2023-12-17-module-loading/index.html
index cbc0d02..aea4283 100644
--- a/blog/2023-12-17-module-loading/index.html
+++ b/blog/2023-12-17-module-loading/index.html
@@ -1,4 +1,4 @@
-jank development update - Load all the modules!
I've been quiet for the past couple of months, finishing up this work on jank's module loading, class path handling, aliasing, and var referring. Along the way, I ran into some very interesting bugs and we're in for a treat of technical detail in this holiday edition of jank development updates! A warm shout out to my Github sponsors and Clojurists Together for sponsoring this work.
Module loading progress
Ok, first and foremost, where is jank now with regard to module loading? I'm very pleased to say that everything I wanted to tackle this quarter has been finished and even more on top of that. There's a PR up for the full changes here.
Let's break this down by section.
Class paths
jank traverses a user-defined class path, which supports directories and JAR files, and can use that to find modules when you use require and friends. This is specifically designed to be compatible with the JVM, so once we hook in Leiningen or Clojure CLI, your existing dependency management should work just fine.
Necessary core functions
The following functions have all been implemented, which were required for module loading:
require
alias
use
refer
load
These take into account modules that are already loaded, flags for things like reloading, excluding, etc. For most use cases, they're at functional parity with Clojure on the happy path. Error handling will improve once I have some better mechanisms for it.
Still, that's not a very big list of functions, I know. How about this one?
compile
create-ns
find-ns
remove-ns
the-ns
ns-name
ns-map
ns-publics
var?
var-get
keys (note - not using a custom seq yet)
vals (note - not using a custom seq yet)
name
namespace
subs
gensym
concat
contains?
find
select-keys
map (note - not lazy yet, no transducers)
mapv (note - not lazy yet, no transducers)
mapcat (note - not lazy yet, no transducers)
filter (note - not lazy yet, no transducers)
complement
remove
set?
set
vector
doseq (note - not supporting fancy for features yet)
list*
apply
some
not-any?
not=
symbol
var?
cond
and
or
ns
All of these were needed by some of the above necessary functions, so I implemented them as much as possible. Most of them have complete functional parity with Clojure, but a few have interim implementations, especially since jank doesn't yet have have an equivalent object type to Clojure JVM's LazySeq. Still, jank feels, and looks, more and more like a proper Clojure every day.
(Bonus) Initial AOT compilation
You may have noticed, in that list, that compile has been implemented. This is an initial step toward AOT compilation and it compiles jank files into C++ files on the class path. Those can then be loaded in lieu of the jank files for a performance win. I also added a CMake job to jank's build system to build the jank Clojure libs along with the compiler, so we can always have those pre-compiled and also always know they actually compile.
I'm currently working with the Cling developers to get support added to Cling for jank to pre-compile these C++ files into a closer equivalent to JVM class files. In my local testing, the startup time improvements by doing this were 10x. I'll have more info on this once the work picks up.
(Bonus) CLI argument parsing
In order to support things like user-defined class paths, I've added a proper CLI arg parser to jank. You can see the current options in the help output here:
❯ ./build/jank -h
+jank development update - Load all the modules!
I've been quiet for the past couple of months, finishing up this work on jank's module loading, class path handling, aliasing, and var referring. Along the way, I ran into some very interesting bugs and we're in for a treat of technical detail in this holiday edition of jank development updates! A warm shout out to my Github sponsors and Clojurists Together for sponsoring this work.
Module loading progress
Ok, first and foremost, where is jank now with regard to module loading? I'm very pleased to say that everything I wanted to tackle this quarter has been finished and even more on top of that. There's a PR up for the full changes here.
Let's break this down by section.
Class paths
jank traverses a user-defined class path, which supports directories and JAR files, and can use that to find modules when you use require and friends. This is specifically designed to be compatible with the JVM, so once we hook in Leiningen or Clojure CLI, your existing dependency management should work just fine.
Necessary core functions
The following functions have all been implemented, which were required for module loading:
require
alias
use
refer
load
These take into account modules that are already loaded, flags for things like reloading, excluding, etc. For most use cases, they're at functional parity with Clojure on the happy path. Error handling will improve once I have some better mechanisms for it.
Still, that's not a very big list of functions, I know. How about this one?
compile
create-ns
find-ns
remove-ns
the-ns
ns-name
ns-map
ns-publics
var?
var-get
keys (note - not using a custom seq yet)
vals (note - not using a custom seq yet)
name
namespace
subs
gensym
concat
contains?
find
select-keys
map (note - not lazy yet, no transducers)
mapv (note - not lazy yet, no transducers)
mapcat (note - not lazy yet, no transducers)
filter (note - not lazy yet, no transducers)
complement
remove
set?
set
vector
doseq (note - not supporting fancy for features yet)
list*
apply
some
not-any?
not=
symbol
var?
cond
and
or
ns
All of these were needed by some of the above necessary functions, so I implemented them as much as possible. Most of them have complete functional parity with Clojure, but a few have interim implementations, especially since jank doesn't yet have have an equivalent object type to Clojure JVM's LazySeq. Still, jank feels, and looks, more and more like a proper Clojure every day.
(Bonus) Initial AOT compilation
You may have noticed, in that list, that compile has been implemented. This is an initial step toward AOT compilation and it compiles jank files into C++ files on the class path. Those can then be loaded in lieu of the jank files for a performance win. I also added a CMake job to jank's build system to build the jank Clojure libs along with the compiler, so we can always have those pre-compiled and also always know they actually compile.
I'm currently working with the Cling developers to get support added to Cling for jank to pre-compile these C++ files into a closer equivalent to JVM class files. In my local testing, the startup time improvements by doing this were 10x. I'll have more info on this once the work picks up.
(Bonus) CLI argument parsing
In order to support things like user-defined class paths, I've added a proper CLI arg parser to jank. You can see the current options in the help output here:
❯ ./build/jank -hjank compilerUsage: ./build/jank [OPTIONS] SUBCOMMAND
diff --git a/blog/2023-12-30-fast-string/index.html b/blog/2023-12-30-fast-string/index.html
index 6df207a..23e0143 100644
--- a/blog/2023-12-30-fast-string/index.html
+++ b/blog/2023-12-30-fast-string/index.html
@@ -1,4 +1,4 @@
-jank's new persistent string is fast
One thing I've been meaning to do is build a custom string class for jank. I had some time, during the holidays, between wrapping up this quarter's work and starting on next quarter's, so I decided to see if I could beat both std::string and folly::fbstring, in terms of performance. After all, if we're gonna make a string class, it'll need to be fast. :)
The back story here is that jank needs to be able to get a hash from a string, and that hash should be cached along with the string. This means I can't use existing string classes, since C++ doesn't have the duck typing mechanisms needed to add this behavior without completely wrapping the class.
Now, I could just wrap std::string or folly::fbstring, and all their member functions but I had a couple other goals in mind, too. In particular, I want jank's string to be persistent, as the rest of its data structures are. Also, since I know jank is garbage collected, and the string is persistent, I should be able to do substring operations and string copies by sharing memory, rather than doing deep copies. To summarize these goals shortly:
Goals
Be as fast, or faster, than std::string and folly::fbstring
Support hashing, with cached value
Be immutable (i.e. no copy on substrings, writes only done in constructors, no mutators)
Not a goal: Complete standard compliance (which allows me to cheat)
There are three noteworthy C++ string implementations:
std::string from GCC's libstdc++
std::string from LLVM's libc++
folly::fbstring from Facebook's folly
Each of them uses a different memory layout and encoding scheme. GCC's string is the simplest to understand, so I started with that. libc++'s string and folly's string are similar, but folly takes things a step further, so I'm going to skip over talking about the libc++ string entirely. Let's not get ahead of ourselves, though. We're starting with GCC's string.
libstdc++'s string
Each of these strings has some features in common, but they go about it differently. In libstdc++'s string, the overall layout is composed of three things:
A pointer to the char array
The length of the char array
The allocated capacity (which may be more than the length)
However, one commonality each of these strings have is that they employ a "small string optimization" (SSO). SSO is a trick which avoids dynamic allocations by storing small strings within the memory of the string class itself, up to a certain size. To accommodate this, libstdc++'s string has a fourth member, which is a char array of 16 bytes. However, a union is used so that the 16 byte char array actually shares the same memory as the third member listed above, the capacity. Depending on whether or not the string is small (based on its length), the pointer will point at the local buffer or somewhere on the "heap" and the capacity will either actually be the capacity or it'll be part of the memory used to store the small string in-situ (in place).
The code for this would look like:
struct string
+jank's new persistent string is fast
One thing I've been meaning to do is build a custom string class for jank. I had some time, during the holidays, between wrapping up this quarter's work and starting on next quarter's, so I decided to see if I could beat both std::string and folly::fbstring, in terms of performance. After all, if we're gonna make a string class, it'll need to be fast. :)
The back story here is that jank needs to be able to get a hash from a string, and that hash should be cached along with the string. This means I can't use existing string classes, since C++ doesn't have the duck typing mechanisms needed to add this behavior without completely wrapping the class.
Now, I could just wrap std::string or folly::fbstring, and all their member functions but I had a couple other goals in mind, too. In particular, I want jank's string to be persistent, as the rest of its data structures are. Also, since I know jank is garbage collected, and the string is persistent, I should be able to do substring operations and string copies by sharing memory, rather than doing deep copies. To summarize these goals shortly:
Goals
Be as fast, or faster, than std::string and folly::fbstring
Support hashing, with cached value
Be immutable (i.e. no copy on substrings, writes only done in constructors, no mutators)
Not a goal: Complete standard compliance (which allows me to cheat)
There are three noteworthy C++ string implementations:
std::string from GCC's libstdc++
std::string from LLVM's libc++
folly::fbstring from Facebook's folly
Each of them uses a different memory layout and encoding scheme. GCC's string is the simplest to understand, so I started with that. libc++'s string and folly's string are similar, but folly takes things a step further, so I'm going to skip over talking about the libc++ string entirely. Let's not get ahead of ourselves, though. We're starting with GCC's string.
libstdc++'s string
Each of these strings has some features in common, but they go about it differently. In libstdc++'s string, the overall layout is composed of three things:
A pointer to the char array
The length of the char array
The allocated capacity (which may be more than the length)
However, one commonality each of these strings have is that they employ a "small string optimization" (SSO). SSO is a trick which avoids dynamic allocations by storing small strings within the memory of the string class itself, up to a certain size. To accommodate this, libstdc++'s string has a fourth member, which is a char array of 16 bytes. However, a union is used so that the 16 byte char array actually shares the same memory as the third member listed above, the capacity. Depending on whether or not the string is small (based on its length), the pointer will point at the local buffer or somewhere on the "heap" and the capacity will either actually be the capacity or it'll be part of the memory used to store the small string in-situ (in place).
The code for this would look like:
struct string{ char *data; size_t length;
diff --git a/blog/2024-02-23-bindings/index.html b/blog/2024-02-23-bindings/index.html
index 7a42eb4..75e97da 100644
--- a/blog/2024-02-23-bindings/index.html
+++ b/blog/2024-02-23-bindings/index.html
@@ -1,4 +1,4 @@
-jank development update - Dynamic bindings and more!
jank development update - Dynamic bindings and more!
For the past couple of months, I have been focused on tackling dynamic var bindings and meta hints, which grew into much, much more. Along the way, I've learned some neat things and have positioned jank to be ready for a lot more outside contributions. Grab a seat and I'll explain it all! Much love to Clojurists Together, who have sponsored some of my work this quarter.
Vars and roots
I estimated that dynamic bindings would take me a while to implement, mainly because, even after several years of using Clojure, I still didn't know how they worked. It's because of this that I'll now explain it to all of you. We can break it down into two parts:
Var roots
Var bindings (overrides)
So, firstly, each var is a container for an object (though vars may start out unbound, they still hold a special unbound object regardless). That object is known as the var's root. When you define a var with def or defn, you're generally providing its root.
; person is a var and its root is "sally"
+jank development update - Dynamic bindings and more!
jank development update - Dynamic bindings and more!
For the past couple of months, I have been focused on tackling dynamic var bindings and meta hints, which grew into much, much more. Along the way, I've learned some neat things and have positioned jank to be ready for a lot more outside contributions. Grab a seat and I'll explain it all! Much love to Clojurists Together, who have sponsored some of my work this quarter.
Vars and roots
I estimated that dynamic bindings would take me a while to implement, mainly because, even after several years of using Clojure, I still didn't know how they worked. It's because of this that I'll now explain it to all of you. We can break it down into two parts:
Var roots
Var bindings (overrides)
So, firstly, each var is a container for an object (though vars may start out unbound, they still hold a special unbound object regardless). That object is known as the var's root. When you define a var with def or defn, you're generally providing its root.
; person is a var and its root is "sally"(def person "sally"); add is a var and its root is an instance of the fn
diff --git a/blog/2024-03-29-syntax-quoting/index.html b/blog/2024-03-29-syntax-quoting/index.html
index 6dd44dc..b3e4306 100644
--- a/blog/2024-03-29-syntax-quoting/index.html
+++ b/blog/2024-03-29-syntax-quoting/index.html
@@ -1,4 +1,4 @@
-jank development update - Syntax quoting!
This quarter, I'm being funded by Clojurists Together to build out jank's lazy sequences, special loop* form, destructuring, and support for the for and doseq macros. Going into this quarter, I had only a rough idea of how Clojure's lazy sequences were implemented. Now, a month in, I'm ready to report some impressive progress!
Lazy sequences
There are three primary types of lazy sequences in Clojure. I was planning on explaining all of this, but, even better, I can shine the spotlight on Bruno Bonacci's blog, since he's covered all three of them very clearly. In short, we have:
Per-element lazy sequences
Chunked lazy sequences
Buffered lazy sequences
This month, I have implemented per-element lazy sequences, along with partial support for chunked lazy sequences. Chunked lazy sequences will be finished next month. By implementing even per-element lazy sequences, so many new opportunities open up. I'll show what I mean by that later in this post, so don't go anywhere!
Loop
Prior to this month, jank supported function-level recur. As part of this month's work, I also implemented loop* and its related recur. When we look at how Clojure JVM implements loop*, it has two different scenarios:
Expression loops
Statement loops
If a loop is in a statement position, Clojure JVM will code-generate labels with goto jumps and local mutations. If the loop is an expression, Clojure JVM generates a function around the loop and then immediately calls that. There is potentially a performance win of not generating the function wrapper and calling it right away, but note that this particular idiom is commonly identified and elided by optimizing compilers. It even has its own acronym: IIFE. (see this also)
jank, for now anyway, simplifies this by always using the IIFE. It does it in a more janky way, though, which is interesting enough that I'll share it with you all. Let's take an example loop* (note that the special form of loop is actually loop*, same as in Clojure; loop is a macro which provides destructuring on top of loop* – now you know):
(loop* [x 0]
+jank development update - Lazy sequences!
This quarter, I'm being funded by Clojurists Together to build out jank's lazy sequences, special loop* form, destructuring, and support for the for and doseq macros. Going into this quarter, I had only a rough idea of how Clojure's lazy sequences were implemented. Now, a month in, I'm ready to report some impressive progress!
Lazy sequences
There are three primary types of lazy sequences in Clojure. I was planning on explaining all of this, but, even better, I can shine the spotlight on Bruno Bonacci's blog, since he's covered all three of them very clearly. In short, we have:
Per-element lazy sequences
Chunked lazy sequences
Buffered lazy sequences
This month, I have implemented per-element lazy sequences, along with partial support for chunked lazy sequences. Chunked lazy sequences will be finished next month. By implementing even per-element lazy sequences, so many new opportunities open up. I'll show what I mean by that later in this post, so don't go anywhere!
Loop
Prior to this month, jank supported function-level recur. As part of this month's work, I also implemented loop* and its related recur. When we look at how Clojure JVM implements loop*, it has two different scenarios:
Expression loops
Statement loops
If a loop is in a statement position, Clojure JVM will code-generate labels with goto jumps and local mutations. If the loop is an expression, Clojure JVM generates a function around the loop and then immediately calls that. There is potentially a performance win of not generating the function wrapper and calling it right away, but note that this particular idiom is commonly identified and elided by optimizing compilers. It even has its own acronym: IIFE. (see this also)
jank, for now anyway, simplifies this by always using the IIFE. It does it in a more janky way, though, which is interesting enough that I'll share it with you all. Let's take an example loop* (note that the special form of loop is actually loop*, same as in Clojure; loop is a macro which provides destructuring on top of loop* – now you know):
Given this, jank will replace the loop* with a fn* and just use function recursion. Initial loop values just get lifted into parameters. The jank compiler will transform the above code into the following:
((fn* [x]
diff --git a/blog/2024-05-31-new-projects/index.html b/blog/2024-05-31-new-projects/index.html
index f32b1b1..85a2c4b 100644
--- a/blog/2024-05-31-new-projects/index.html
+++ b/blog/2024-05-31-new-projects/index.html
@@ -1 +1 @@
-jank development update - New projects!
Hey folks! I've been building on last month's addition of lazy sequences, loop*, destructuring, and more. This month, I've worked on rounding out lazy sequences, adding more mutability, better meta support, and some big project updates. Much love to Clojurists Together, who are funding my work this quarter.
Chunked sequences
I've expanded the lazy sequence support added last month to include chunked sequences, which pre-load elements in chunks to aid in throughput. At this point, only clojure.core/range returns a chunked sequence, but all of the existing clojure.core functions which should have support for them do.
If you recall from last month, there is a third lazy sequence type: buffered sequences. I won't be implementing those until they're needed, as I'd never even heard of them before researching more into the lazy sequences in Clojure.
Initial quarter goals accomplished
Wrapping up the lazy sequence work, minus buffered sequences, actually checked off all the boxes for my original goals this quarter. There's a bottomless well of new tasks, though, so I've moved onto some others. So, how do I decide what to work on next?
My goal is for you all to be writing jank programs. The most important tasks are the ones which bring me closer to that goal. Let's take a look at what those have been so far.
Volatiles, atoms, and reduced
Most programs have some of mutation and we generally handle that with volatiles and atoms in Clojure. jank already supported transients for most data structures, but we didn't have a way to hold mutable boxes to immutable values. Volatiles are also essential for many transducers, which I'll mention a bit later. This month, both volatiles and atoms have been implemented.
Implementing atoms involved a fair amount of research, since lockless programming with atomics is not nearly as straightforward as one might expect.
As part of implementing atoms, I also added support for the @ reader macro and the overall derefable behavior. This same behavior will be used for delays, futures, and others going forward.
Meta handling for defs
Last quarter, I added support for meta hints, but I didn't actually use that metadata in many places. Now, with defs, I've added support for the optional meta map and doc string and I also read the meta from the defined symbol. This isn't a huge win, but it does mean that jank can start using doc strings normally, and that we can do things like associate more function meta to the var in a defn, which can improve error reporting.
Monorepo
There will be many jank projects and I've known for a while that I want them all to be in one git monorepo. This makes code sharing, searching, refactoring, and browsing simpler. It gives contributors one place to go in order to get started and one place for all of the issues and discussions. It's not my intention to convince you of anything, if you're not a fan of monorepos, but jank is now using one.
This started by bringing in lein-jank, which was initially created by Saket Patel. From there, I've added a couple of more projects, which I'll cover later in this update.
New clojure.core functions
Following last month's theme, which saw 52 new Clojure functions, I have excellent news. We actually beat that this month, adding 56 new Clojure functions! However, I only added 23 of those and the other 33 were added by madstap (Aleksander Madland Stapnes). He did this while also adding the transducer arity into pretty much every existing sequence function. I added volatiles to support him in writing those transducers.
dotimes
chunk
chunk-first
chunk-next
chunk-rest
chunk-cons
chunked-seq?
volatile!
vswap!
vreset!
volatile?
deref
reduced
reduced?
ensure-reduced
unreduced
identical?
atom
swap!
reset!
swap-vals!
reset-vals!
compare-and-set!
keep
completing
transduce
run!
comp
repeatedly
tree-seq
flatten
cat
interpose
juxt
partial
doto
map-indexed
keep-indexed
frequencies
reductions
distinct
distinct?
dedupe
fnil
every-pred
some-fn
group-by
not-empty
get-in
assoc-in
update-in
update
cond->>
as->
some->
some->>
New projects
At this point, I was thinking that jank actually has pretty darn good Clojure parity, both in terms of syntax and essential core functions. So how can I take the best steps toward getting jank onto your computer?
Well, I think the most important thing is for me to start writing some actual projects in jank. Doing this will require improving the tooling and will help identify issues with the existing functionality. The project I've chosen is jank's nREPL server. By the end of the project, we'll not only have more confidence in jank, we'll all be able to connect our editors to running jank programs!
nREPL server
nREPL has some docs on building new servers, so I've taken those as a starting point. However, let's be clear, there are going to be a lot of steps along the way. jank is not currently ready for me to just build this server today and have it all work. I need a goal to work toward, though, and every quest I go on is bringing me one step closer to completing this nREPL server in jank. Let's take a look at some of the things I know I'll need for this.
Module system
jank's module system was implemented two quarters ago, but since there are no real jank projects, it hasn't seen much battle testing. To start with, I will need to work through some issues with this. Already I've found (and fixed) a couple of bugs related to module writing and reading while getting started on the nREPL server. Further improvements will be needed around how modules are cached and timestamped for iterative compilation.
Native interop
Next, jank's native interop support will need to be expanded. I've started on that this month by making it possible to now write C++ sources alongside your jank sources and actually require them from jank! As you may know, jank allows for inline C++ code within the special native/raw form, but by compiling entire C++ files alongside your jank code, it's now much easier to offload certain aspects of your jank programs to C++ without worrying about writing too much C++ as inline jank strings.
jank's native interop support can be further improved by declaratively noting include paths, implicit includes, link paths, and linked libraries as part of the project. This will likely end up necessary for the nREPL server.
AOT compilation
Also required for the nREPL server, I'll need to design and implement jank's AOT compilation system. This will involve compiling all jank sources and C++ sources together and can allow for direct linking, whole-program link time optimizations (LTO), and even static runtimes (no interactivity, but smaller binaries).
Distribution
Finally, both jank and the nREPL server will need distribution mechanisms for Linux and macOS. For jank, that may mean AppImages or perhaps more integrated binaries. Either way, I want this to be easy for you all to use and I'm following Rust/Cargo as my overall inspiration.
I hope I've succeeded in showing how much work still remains for this nREPL server to be built and shipped out. This will take me several months, I'd estimate. However, I think having this sort of goal in mind is very powerful and I'm excited that jank is far enough along to where I can actually be doing this.
nREPL server progress
Since I have C++ sources working alongside jank source now, I can use boost::asio to spin up an async TCP server. The data sent over the wire for nREPL servers is encoded with bencode, so I started on a jank.data.bencode project and I have the decoding portion of that working. From there, I wanted to write my tests in jank using clojure.test, but I haven't implemented clojure.test yet, so I looked into doing that. It looks like clojure.test will require me to implement multimethods in jank, which don't yet exist. On top of that, I'll need to implement clojure.template, which requires clojure.walk, none of which have been started.
I'll continue on with this depth-first search, implementing as needed, and then unwind all the way back up to making more progress on the nREPL server. Getting clojure.test working will be a huge step toward being able to dogfood more, so I don't want to cut any corners there. Once I can test my decode implementation for bencode, I'll write the encoding (which is easier) and then I'll be back onto implementing the nREPL server functionality.
Hang tight, folks! We've come a long way, and there is still so much work to do, but the wheels are rolling and jank is actually becoming a usable Clojure dialect. Your interest, support, questions, and encouragement are all the inspiration which keeps me going.
Hey folks! I've been building on last month's addition of lazy sequences, loop*, destructuring, and more. This month, I've worked on rounding out lazy sequences, adding more mutability, better meta support, and some big project updates. Much love to Clojurists Together, who are funding my work this quarter.
Chunked sequences
I've expanded the lazy sequence support added last month to include chunked sequences, which pre-load elements in chunks to aid in throughput. At this point, only clojure.core/range returns a chunked sequence, but all of the existing clojure.core functions which should have support for them do.
If you recall from last month, there is a third lazy sequence type: buffered sequences. I won't be implementing those until they're needed, as I'd never even heard of them before researching more into the lazy sequences in Clojure.
Initial quarter goals accomplished
Wrapping up the lazy sequence work, minus buffered sequences, actually checked off all the boxes for my original goals this quarter. There's a bottomless well of new tasks, though, so I've moved onto some others. So, how do I decide what to work on next?
My goal is for you all to be writing jank programs. The most important tasks are the ones which bring me closer to that goal. Let's take a look at what those have been so far.
Volatiles, atoms, and reduced
Most programs have some of mutation and we generally handle that with volatiles and atoms in Clojure. jank already supported transients for most data structures, but we didn't have a way to hold mutable boxes to immutable values. Volatiles are also essential for many transducers, which I'll mention a bit later. This month, both volatiles and atoms have been implemented.
Implementing atoms involved a fair amount of research, since lockless programming with atomics is not nearly as straightforward as one might expect.
As part of implementing atoms, I also added support for the @ reader macro and the overall derefable behavior. This same behavior will be used for delays, futures, and others going forward.
Meta handling for defs
Last quarter, I added support for meta hints, but I didn't actually use that metadata in many places. Now, with defs, I've added support for the optional meta map and doc string and I also read the meta from the defined symbol. This isn't a huge win, but it does mean that jank can start using doc strings normally, and that we can do things like associate more function meta to the var in a defn, which can improve error reporting.
Monorepo
There will be many jank projects and I've known for a while that I want them all to be in one git monorepo. This makes code sharing, searching, refactoring, and browsing simpler. It gives contributors one place to go in order to get started and one place for all of the issues and discussions. It's not my intention to convince you of anything, if you're not a fan of monorepos, but jank is now using one.
This started by bringing in lein-jank, which was initially created by Saket Patel. From there, I've added a couple of more projects, which I'll cover later in this update.
New clojure.core functions
Following last month's theme, which saw 52 new Clojure functions, I have excellent news. We actually beat that this month, adding 56 new Clojure functions! However, I only added 23 of those and the other 33 were added by madstap (Aleksander Madland Stapnes). He did this while also adding the transducer arity into pretty much every existing sequence function. I added volatiles to support him in writing those transducers.
dotimes
chunk
chunk-first
chunk-next
chunk-rest
chunk-cons
chunked-seq?
volatile!
vswap!
vreset!
volatile?
deref
reduced
reduced?
ensure-reduced
unreduced
identical?
atom
swap!
reset!
swap-vals!
reset-vals!
compare-and-set!
keep
completing
transduce
run!
comp
repeatedly
tree-seq
flatten
cat
interpose
juxt
partial
doto
map-indexed
keep-indexed
frequencies
reductions
distinct
distinct?
dedupe
fnil
every-pred
some-fn
group-by
not-empty
get-in
assoc-in
update-in
update
cond->>
as->
some->
some->>
New projects
At this point, I was thinking that jank actually has pretty darn good Clojure parity, both in terms of syntax and essential core functions. So how can I take the best steps toward getting jank onto your computer?
Well, I think the most important thing is for me to start writing some actual projects in jank. Doing this will require improving the tooling and will help identify issues with the existing functionality. The project I've chosen is jank's nREPL server. By the end of the project, we'll not only have more confidence in jank, we'll all be able to connect our editors to running jank programs!
nREPL server
nREPL has some docs on building new servers, so I've taken those as a starting point. However, let's be clear, there are going to be a lot of steps along the way. jank is not currently ready for me to just build this server today and have it all work. I need a goal to work toward, though, and every quest I go on is bringing me one step closer to completing this nREPL server in jank. Let's take a look at some of the things I know I'll need for this.
Module system
jank's module system was implemented two quarters ago, but since there are no real jank projects, it hasn't seen much battle testing. To start with, I will need to work through some issues with this. Already I've found (and fixed) a couple of bugs related to module writing and reading while getting started on the nREPL server. Further improvements will be needed around how modules are cached and timestamped for iterative compilation.
Native interop
Next, jank's native interop support will need to be expanded. I've started on that this month by making it possible to now write C++ sources alongside your jank sources and actually require them from jank! As you may know, jank allows for inline C++ code within the special native/raw form, but by compiling entire C++ files alongside your jank code, it's now much easier to offload certain aspects of your jank programs to C++ without worrying about writing too much C++ as inline jank strings.
jank's native interop support can be further improved by declaratively noting include paths, implicit includes, link paths, and linked libraries as part of the project. This will likely end up necessary for the nREPL server.
AOT compilation
Also required for the nREPL server, I'll need to design and implement jank's AOT compilation system. This will involve compiling all jank sources and C++ sources together and can allow for direct linking, whole-program link time optimizations (LTO), and even static runtimes (no interactivity, but smaller binaries).
Distribution
Finally, both jank and the nREPL server will need distribution mechanisms for Linux and macOS. For jank, that may mean AppImages or perhaps more integrated binaries. Either way, I want this to be easy for you all to use and I'm following Rust/Cargo as my overall inspiration.
I hope I've succeeded in showing how much work still remains for this nREPL server to be built and shipped out. This will take me several months, I'd estimate. However, I think having this sort of goal in mind is very powerful and I'm excited that jank is far enough along to where I can actually be doing this.
nREPL server progress
Since I have C++ sources working alongside jank source now, I can use boost::asio to spin up an async TCP server. The data sent over the wire for nREPL servers is encoded with bencode, so I started on a jank.data.bencode project and I have the decoding portion of that working. From there, I wanted to write my tests in jank using clojure.test, but I haven't implemented clojure.test yet, so I looked into doing that. It looks like clojure.test will require me to implement multimethods in jank, which don't yet exist. On top of that, I'll need to implement clojure.template, which requires clojure.walk, none of which have been started.
I'll continue on with this depth-first search, implementing as needed, and then unwind all the way back up to making more progress on the nREPL server. Getting clojure.test working will be a huge step toward being able to dogfood more, so I don't want to cut any corners there. Once I can test my decode implementation for bencode, I'll write the encoding (which is easier) and then I'll be back onto implementing the nREPL server functionality.
Hang tight, folks! We've come a long way, and there is still so much work to do, but the wheels are rolling and jank is actually becoming a usable Clojure dialect. Your interest, support, questions, and encouragement are all the inspiration which keeps me going.
\ No newline at end of file
diff --git a/blog/2024-06-29-multimethods/index.html b/blog/2024-06-29-multimethods/index.html
index 3002a24..11da4f1 100644
--- a/blog/2024-06-29-multimethods/index.html
+++ b/blog/2024-06-29-multimethods/index.html
@@ -1,4 +1,4 @@
-jank development update - Multimethods!
Welcome back to another jank development update! For the past month, I've been pushing jank closer to production readiness primarily by working on multimethods and by debugging issues with Clang 19 (currently unreleased). Much love to Clojurists Together and all of my Github sponsors for their support this quarter.
Multimethods
I thought, going into this month, that I had a good idea of how multimethods work in Clojure. I figured we define a dispatch function with defmulti:
(defmulti sauce-suggestion ::noodle-type)
Then we define our catch-all method for handling types:
(defmethod sauce-suggestion :default [noodle]
+jank development update - Multimethods!
Welcome back to another jank development update! For the past month, I've been pushing jank closer to production readiness primarily by working on multimethods and by debugging issues with Clang 19 (currently unreleased). Much love to Clojurists Together and all of my Github sponsors for their support this quarter.
Multimethods
I thought, going into this month, that I had a good idea of how multimethods work in Clojure. I figured we define a dispatch function with defmulti:
(defmulti sauce-suggestion ::noodle-type)
Then we define our catch-all method for handling types:
(defmethod sauce-suggestion :default [noodle] (println "You can't go wrong with some butter and garlic."))
Then we define some specializations for certain values which come out of our dispatch function.
(defmethod sauce-suggestion ::shell [noodle] (println "Cheeeeeeeese!"))
diff --git a/blog/feed.xml b/blog/feed.xml
index 9a9288f..cdb9369 100644
--- a/blog/feed.xml
+++ b/blog/feed.xml
@@ -1 +1 @@
-2024-07-05T00:10:32.353473744Zjank bloghttps://jank-lang.org/blog/jank development update - Multimethods!2024-06-29T00:00:00Z2024-06-29T00:00:00Zhttps://jank-lang.org/blog/2024-06-29-multimethodsJeaye Wilkerson<p>Welcome back to another jank development update! For the past month, I've been pushing jank closer to production readiness primarily by working on multimethods and by debugging issues with Clang 19 (currently unreleased). Much love to <a href="https://www.clojuriststogether.org/">Clojurists Together</a> and all of my <a href="https://github.com/sponsors/jeaye">Github sponsors</a> for their support this quarter.</p>jank development update - New projects!2024-05-31T00:00:00Z2024-05-31T00:00:00Zhttps://jank-lang.org/blog/2024-05-31-new-projectsJeaye Wilkerson<p>Hey folks! I've been building on last month's addition of lazy sequences, <code>loop*</code>, destructuring, and more. This month, I've worked on rounding out lazy sequences, adding more mutability, better meta support, and some big project updates. Much love to <a href="https://www.clojuriststogether.org/">Clojurists Together</a>, who are funding my work this quarter.</p>jank development update - Lazy sequences!2024-04-27T00:00:00Z2024-04-27T00:00:00Zhttps://jank-lang.org/blog/2024-04-27-lazy-sequencesJeaye Wilkerson<p>This quarter, I'm being funded by <a href="https://www.clojuriststogether.org/">Clojurists Together</a> to build out jank's lazy sequences, special <code>loop*</code> form, destructuring, and support for the <code>for</code> and <code>doseq</code> macros. Going into this quarter, I had only a rough idea of how Clojure's lazy sequences were implemented. Now, a month in, I'm ready to report some impressive progress!</p>jank development update - Syntax quoting!2024-03-29T00:00:00Z2024-03-29T00:00:00Zhttps://jank-lang.org/blog/2024-03-29-syntax-quotingJeaye Wilkerson<p>Oh, hey folks. I was just wrapping up this macro I was writing. One moment.</p>jank development update - Dynamic bindings and more!2024-02-23T00:00:00Z2024-02-23T00:00:00Zhttps://jank-lang.org/blog/2024-02-23-bindingsJeaye Wilkerson<p>For the past couple of months, I have been focused on tackling dynamic var bindings and meta hints, which grew into much, much more. Along the way, I've learned some neat things and have positioned jank to be ready for a lot more outside contributions. Grab a seat and I'll explain it all! Much love to <a href="https://www.clojuriststogether.org/">Clojurists Together</a>, who have sponsored some of my work this quarter.</p>jank's new persistent string is fast2023-12-30T00:00:00Z2023-12-30T00:00:00Zhttps://jank-lang.org/blog/2023-12-30-fast-stringJeaye Wilkerson<p>One thing I've been meaning to do is build a custom string class for jank. I had some time, during the holidays, between wrapping up this quarter's work and starting on next quarter's, so I decided to see if I could beat both <code>std::string</code> and <code>folly::fbstring</code>, in terms of performance. After all, if we're gonna make a string class, it'll need to be fast. :)</p>jank development update - Load all the modules!2023-12-17T00:00:00Z2023-12-17T00:00:00Zhttps://jank-lang.org/blog/2023-12-17-module-loadingJeaye Wilkerson<p>I've been quiet for the past couple of months, finishing up this work on jank's module loading, class path handling, aliasing, and var referring. Along the way, I ran into some very interesting bugs and we're in for a treat of technical detail in this holiday edition of jank development updates! A warm shout out to my <a href="https://github.com/sponsors/jeaye">Github sponsors</a> and <a href="https://www.clojuriststogether.org/">Clojurists Together</a> for sponsoring this work.</p>jank development update - Module loading2023-10-14T00:00:00Z2023-10-14T00:00:00Zhttps://jank-lang.org/blog/2023-10-14-module-loadingJeaye Wilkerson<p>For the past month and a half, I've been building out jank's support for <code>clojure.core/require</code>, including everything from class path handling to compiling jank files to intermediate code written to the filesystem. This is a half-way report for the quarter. As a warm note, my work on jank this quarter is being sponsored by <a href="https://www.clojuriststogether.org/">Clojurists Together</a>.</p>jank development update - Object model results2023-08-26T00:00:00Z2023-08-26T00:00:00Zhttps://jank-lang.org/blog/2023-08-26-object-modelJeaye Wilkerson<p>As summer draws to a close, in the Pacific Northwest, so too does my term of sponsored work focused on a faster object model for jank. Thanks so much to <a href="https://www.clojuriststogether.org/">Clojurists Together</a> for funding jank's development. The past quarter has been quite successful and I'm excited to share the results.</p>jank development update - A faster object model2023-07-08T00:00:00Z2023-07-08T00:00:00Zhttps://jank-lang.org/blog/2023-07-08-object-modelJeaye Wilkerson<p>This quarter, my work on jank is being sponsored by <a href="https://www.clojuriststogether.org/">Clojurists Together</a>. The terms of the work are to research a new object model for jank, with the goal of making jank code faster across the board. This is a half-way report and I'm excited to share my results!</p>jank development update - Optimizing a ray tracer2023-04-07T00:00:00Z2023-04-07T00:00:00Zhttps://jank-lang.org/blog/2023-04-07-ray-tracingJeaye Wilkerson<p>After the <a href="/blog/2023-01-13-optimizing-sequences">last post</a>, which focused on optimizing jank's sequences, I wanted to get jank running a ray tracer I had previously written in Clojure. In this post, I document what was required to start ray tracing in jank and, more importantly, how I chased down the run time in a fierce battle with Clojure's performance.</p>jank development update - Optimizing sequences2023-01-13T00:00:00Z2023-01-13T00:00:00Zhttps://jank-lang.org/blog/2023-01-13-optimizing-sequencesJeaye Wilkerson<p>In this episode of jank's development updates, we follow an exciting few weekends as I was digging deep into Clojure's sequence implementation, building jank's equivalent, and then benchmarking and profiling in a dizzying race to the bottom.</p>jank development update - Lots of new changes2022-12-08T00:00:00Z2022-12-08T00:00:00Zhttps://jank-lang.org/blog/2022-12-08-progress-updateJeaye Wilkerson<p>I was previously giving updates only in the <a href="https://clojurians.slack.com/archives/C03SRH97FDK">#jank</a> Slack channel, but some of these are getting large enough to warrant more prose. Thus, happily, I can announce that jank has a new blog and I have a <i>lot</i> of new progress to report! Let's get into the details.</p>
\ No newline at end of file
+2024-08-31T18:12:37.140621203Zjank bloghttps://jank-lang.org/blog/jank development update - Multimethods!2024-06-29T00:00:00Z2024-06-29T00:00:00Zhttps://jank-lang.org/blog/2024-06-29-multimethodsJeaye Wilkerson<p>Welcome back to another jank development update! For the past month, I've been pushing jank closer to production readiness primarily by working on multimethods and by debugging issues with Clang 19 (currently unreleased). Much love to <a href="https://www.clojuriststogether.org/">Clojurists Together</a> and all of my <a href="https://github.com/sponsors/jeaye">Github sponsors</a> for their support this quarter.</p>jank development update - New projects!2024-05-31T00:00:00Z2024-05-31T00:00:00Zhttps://jank-lang.org/blog/2024-05-31-new-projectsJeaye Wilkerson<p>Hey folks! I've been building on last month's addition of lazy sequences, <code>loop*</code>, destructuring, and more. This month, I've worked on rounding out lazy sequences, adding more mutability, better meta support, and some big project updates. Much love to <a href="https://www.clojuriststogether.org/">Clojurists Together</a>, who are funding my work this quarter.</p>jank development update - Lazy sequences!2024-04-27T00:00:00Z2024-04-27T00:00:00Zhttps://jank-lang.org/blog/2024-04-27-lazy-sequencesJeaye Wilkerson<p>This quarter, I'm being funded by <a href="https://www.clojuriststogether.org/">Clojurists Together</a> to build out jank's lazy sequences, special <code>loop*</code> form, destructuring, and support for the <code>for</code> and <code>doseq</code> macros. Going into this quarter, I had only a rough idea of how Clojure's lazy sequences were implemented. Now, a month in, I'm ready to report some impressive progress!</p>jank development update - Syntax quoting!2024-03-29T00:00:00Z2024-03-29T00:00:00Zhttps://jank-lang.org/blog/2024-03-29-syntax-quotingJeaye Wilkerson<p>Oh, hey folks. I was just wrapping up this macro I was writing. One moment.</p>jank development update - Dynamic bindings and more!2024-02-23T00:00:00Z2024-02-23T00:00:00Zhttps://jank-lang.org/blog/2024-02-23-bindingsJeaye Wilkerson<p>For the past couple of months, I have been focused on tackling dynamic var bindings and meta hints, which grew into much, much more. Along the way, I've learned some neat things and have positioned jank to be ready for a lot more outside contributions. Grab a seat and I'll explain it all! Much love to <a href="https://www.clojuriststogether.org/">Clojurists Together</a>, who have sponsored some of my work this quarter.</p>jank's new persistent string is fast2023-12-30T00:00:00Z2023-12-30T00:00:00Zhttps://jank-lang.org/blog/2023-12-30-fast-stringJeaye Wilkerson<p>One thing I've been meaning to do is build a custom string class for jank. I had some time, during the holidays, between wrapping up this quarter's work and starting on next quarter's, so I decided to see if I could beat both <code>std::string</code> and <code>folly::fbstring</code>, in terms of performance. After all, if we're gonna make a string class, it'll need to be fast. :)</p>jank development update - Load all the modules!2023-12-17T00:00:00Z2023-12-17T00:00:00Zhttps://jank-lang.org/blog/2023-12-17-module-loadingJeaye Wilkerson<p>I've been quiet for the past couple of months, finishing up this work on jank's module loading, class path handling, aliasing, and var referring. Along the way, I ran into some very interesting bugs and we're in for a treat of technical detail in this holiday edition of jank development updates! A warm shout out to my <a href="https://github.com/sponsors/jeaye">Github sponsors</a> and <a href="https://www.clojuriststogether.org/">Clojurists Together</a> for sponsoring this work.</p>jank development update - Module loading2023-10-14T00:00:00Z2023-10-14T00:00:00Zhttps://jank-lang.org/blog/2023-10-14-module-loadingJeaye Wilkerson<p>For the past month and a half, I've been building out jank's support for <code>clojure.core/require</code>, including everything from class path handling to compiling jank files to intermediate code written to the filesystem. This is a half-way report for the quarter. As a warm note, my work on jank this quarter is being sponsored by <a href="https://www.clojuriststogether.org/">Clojurists Together</a>.</p>jank development update - Object model results2023-08-26T00:00:00Z2023-08-26T00:00:00Zhttps://jank-lang.org/blog/2023-08-26-object-modelJeaye Wilkerson<p>As summer draws to a close, in the Pacific Northwest, so too does my term of sponsored work focused on a faster object model for jank. Thanks so much to <a href="https://www.clojuriststogether.org/">Clojurists Together</a> for funding jank's development. The past quarter has been quite successful and I'm excited to share the results.</p>jank development update - A faster object model2023-07-08T00:00:00Z2023-07-08T00:00:00Zhttps://jank-lang.org/blog/2023-07-08-object-modelJeaye Wilkerson<p>This quarter, my work on jank is being sponsored by <a href="https://www.clojuriststogether.org/">Clojurists Together</a>. The terms of the work are to research a new object model for jank, with the goal of making jank code faster across the board. This is a half-way report and I'm excited to share my results!</p>jank development update - Optimizing a ray tracer2023-04-07T00:00:00Z2023-04-07T00:00:00Zhttps://jank-lang.org/blog/2023-04-07-ray-tracingJeaye Wilkerson<p>After the <a href="/blog/2023-01-13-optimizing-sequences">last post</a>, which focused on optimizing jank's sequences, I wanted to get jank running a ray tracer I had previously written in Clojure. In this post, I document what was required to start ray tracing in jank and, more importantly, how I chased down the run time in a fierce battle with Clojure's performance.</p>jank development update - Optimizing sequences2023-01-13T00:00:00Z2023-01-13T00:00:00Zhttps://jank-lang.org/blog/2023-01-13-optimizing-sequencesJeaye Wilkerson<p>In this episode of jank's development updates, we follow an exciting few weekends as I was digging deep into Clojure's sequence implementation, building jank's equivalent, and then benchmarking and profiling in a dizzying race to the bottom.</p>jank development update - Lots of new changes2022-12-08T00:00:00Z2022-12-08T00:00:00Zhttps://jank-lang.org/blog/2022-12-08-progress-updateJeaye Wilkerson<p>I was previously giving updates only in the <a href="https://clojurians.slack.com/archives/C03SRH97FDK">#jank</a> Slack channel, but some of these are getting large enough to warrant more prose. Thus, happily, I can announce that jank has a new blog and I have a <i>lot</i> of new progress to report! Let's get into the details.</p>
\ No newline at end of file
diff --git a/blog/index.html b/blog/index.html
index 936ae71..fc29497 100644
--- a/blog/index.html
+++ b/blog/index.html
@@ -1 +1 @@
-jank - blog
Welcome back to another jank development update! For the past month, I've been pushing jank closer to production readiness primarily by working on multimethods and by debugging issues with Clang 19 (currently unreleased). Much love to Clojurists Together and all of my Github sponsors for their support this quarter.
Hey folks! I've been building on last month's addition of lazy sequences, loop*, destructuring, and more. This month, I've worked on rounding out lazy sequences, adding more mutability, better meta support, and some big project updates. Much love to Clojurists Together, who are funding my work this quarter.
This quarter, I'm being funded by Clojurists Together to build out jank's lazy sequences, special loop* form, destructuring, and support for the for and doseq macros. Going into this quarter, I had only a rough idea of how Clojure's lazy sequences were implemented. Now, a month in, I'm ready to report some impressive progress!
For the past couple of months, I have been focused on tackling dynamic var bindings and meta hints, which grew into much, much more. Along the way, I've learned some neat things and have positioned jank to be ready for a lot more outside contributions. Grab a seat and I'll explain it all! Much love to Clojurists Together, who have sponsored some of my work this quarter.
One thing I've been meaning to do is build a custom string class for jank. I had some time, during the holidays, between wrapping up this quarter's work and starting on next quarter's, so I decided to see if I could beat both std::string and folly::fbstring, in terms of performance. After all, if we're gonna make a string class, it'll need to be fast. :)
I've been quiet for the past couple of months, finishing up this work on jank's module loading, class path handling, aliasing, and var referring. Along the way, I ran into some very interesting bugs and we're in for a treat of technical detail in this holiday edition of jank development updates! A warm shout out to my Github sponsors and Clojurists Together for sponsoring this work.
For the past month and a half, I've been building out jank's support for clojure.core/require, including everything from class path handling to compiling jank files to intermediate code written to the filesystem. This is a half-way report for the quarter. As a warm note, my work on jank this quarter is being sponsored by Clojurists Together.
As summer draws to a close, in the Pacific Northwest, so too does my term of sponsored work focused on a faster object model for jank. Thanks so much to Clojurists Together for funding jank's development. The past quarter has been quite successful and I'm excited to share the results.
This quarter, my work on jank is being sponsored by Clojurists Together. The terms of the work are to research a new object model for jank, with the goal of making jank code faster across the board. This is a half-way report and I'm excited to share my results!
After the last post, which focused on optimizing jank's sequences, I wanted to get jank running a ray tracer I had previously written in Clojure. In this post, I document what was required to start ray tracing in jank and, more importantly, how I chased down the run time in a fierce battle with Clojure's performance.
In this episode of jank's development updates, we follow an exciting few weekends as I was digging deep into Clojure's sequence implementation, building jank's equivalent, and then benchmarking and profiling in a dizzying race to the bottom.
I was previously giving updates only in the #jank Slack channel, but some of these are getting large enough to warrant more prose. Thus, happily, I can announce that jank has a new blog and I have a lot of new progress to report! Let's get into the details.
Welcome back to another jank development update! For the past month, I've been pushing jank closer to production readiness primarily by working on multimethods and by debugging issues with Clang 19 (currently unreleased). Much love to Clojurists Together and all of my Github sponsors for their support this quarter.
Hey folks! I've been building on last month's addition of lazy sequences, loop*, destructuring, and more. This month, I've worked on rounding out lazy sequences, adding more mutability, better meta support, and some big project updates. Much love to Clojurists Together, who are funding my work this quarter.
This quarter, I'm being funded by Clojurists Together to build out jank's lazy sequences, special loop* form, destructuring, and support for the for and doseq macros. Going into this quarter, I had only a rough idea of how Clojure's lazy sequences were implemented. Now, a month in, I'm ready to report some impressive progress!
For the past couple of months, I have been focused on tackling dynamic var bindings and meta hints, which grew into much, much more. Along the way, I've learned some neat things and have positioned jank to be ready for a lot more outside contributions. Grab a seat and I'll explain it all! Much love to Clojurists Together, who have sponsored some of my work this quarter.
One thing I've been meaning to do is build a custom string class for jank. I had some time, during the holidays, between wrapping up this quarter's work and starting on next quarter's, so I decided to see if I could beat both std::string and folly::fbstring, in terms of performance. After all, if we're gonna make a string class, it'll need to be fast. :)
I've been quiet for the past couple of months, finishing up this work on jank's module loading, class path handling, aliasing, and var referring. Along the way, I ran into some very interesting bugs and we're in for a treat of technical detail in this holiday edition of jank development updates! A warm shout out to my Github sponsors and Clojurists Together for sponsoring this work.
For the past month and a half, I've been building out jank's support for clojure.core/require, including everything from class path handling to compiling jank files to intermediate code written to the filesystem. This is a half-way report for the quarter. As a warm note, my work on jank this quarter is being sponsored by Clojurists Together.
As summer draws to a close, in the Pacific Northwest, so too does my term of sponsored work focused on a faster object model for jank. Thanks so much to Clojurists Together for funding jank's development. The past quarter has been quite successful and I'm excited to share the results.
This quarter, my work on jank is being sponsored by Clojurists Together. The terms of the work are to research a new object model for jank, with the goal of making jank code faster across the board. This is a half-way report and I'm excited to share my results!
After the last post, which focused on optimizing jank's sequences, I wanted to get jank running a ray tracer I had previously written in Clojure. In this post, I document what was required to start ray tracing in jank and, more importantly, how I chased down the run time in a fierce battle with Clojure's performance.
In this episode of jank's development updates, we follow an exciting few weekends as I was digging deep into Clojure's sequence implementation, building jank's equivalent, and then benchmarking and profiling in a dizzying race to the bottom.
I was previously giving updates only in the #jank Slack channel, but some of these are getting large enough to warrant more prose. Thus, happily, I can announce that jank has a new blog and I have a lot of new progress to report! Let's get into the details.
jank is a general-purpose programming language which embraces the interactive, value-oriented nature of Clojure as well as the desire for native compilation and minimal runtimes. jank is strongly compatible with Clojure and considers itself a dialect of Clojure. Please note that jank is under heavy development; assume all features are planned or incomplete.
Where jank differs from Clojure JVM is that its host is C++ on top of an LLVM-based JIT. This allows jank to offer the same benefits of REPL-based development while being able to seamlessly reach into the native world and compete seriously with JVM's performance.
Still, jank is a Clojure dialect and thus includes its code-as-data philosophy and powerful macro system. jank remains a functional-first language which builds upon Clojure's rich set of persistent, immutable data structures. When mutability is needed, jank offers a software transaction memory and reactive agent system to ensure clean and correct multi-threaded designs.
Enjoy both REPL iteration with JIT compilation and static AOT compilation to native executables.
01
Iterate like you would with Clojure
Iterate in the REPL and build your program from the ground up without leaving your editor.
(defn -main [& args]
+jank programming language - Clojure/LLVM/C++
The jank programming language
jank is a general-purpose programming language which embraces the interactive, value-oriented nature of Clojure as well as the desire for native compilation and minimal runtimes. jank is strongly compatible with Clojure and considers itself a dialect of Clojure. Please note that jank is under heavy development; assume all features are planned or incomplete.
Where jank differs from Clojure JVM is that its host is C++ on top of an LLVM-based JIT. This allows jank to offer the same benefits of REPL-based development while being able to seamlessly reach into the native world and compete seriously with JVM's performance.
Still, jank is a Clojure dialect and thus includes its code-as-data philosophy and powerful macro system. jank remains a functional-first language which builds upon Clojure's rich set of persistent, immutable data structures. When mutability is needed, jank offers a software transaction memory and reactive agent system to ensure clean and correct multi-threaded designs.
jank is under heavy development. It's safest to assume that any feature advertised is partially developed or in the planning stages. There is no sales pitch here; just a lot of work and some big plans. All development happens on Github, so watch the repo there!
\ No newline at end of file
+jank programming language - Clojure/LLVM/C++
jank is under heavy development. It's safest to assume that any feature advertised is partially developed or in the planning stages. There is no sales pitch here; just a lot of work and some big plans. All development happens on Github, so watch the repo there!
\ No newline at end of file
+