Template inconsistency C# and Extension

[injeniero]

I have been using Prompty to write the prompts for our C# Unity game. A template that does work in VS does not work in c#. After some research, I noticed the problem is the Template engine used.

On VS the extension is using nunjucks while the C# runtime is using scriban with support for (liquid)[https://shopify.github.io/liquid/]. So the problem is all of those template engines share a basic template grammar, but behavior and operators are all over the place.

A few examples:

• Nunjucks only has the filter length, while liquid only has size.
• Nunjucks | join(", "), while liquid | join: ", "
• Nunjucks has the not unary operator, while liquid does not.
• Expressions precedence, I had to add parenthesis for C# to produce the same output. This causes the Editor to complain when trying to extract semantic tokens, but it does render the template correctly.

I have not used python runtime, but I checked it is using jinja2 which I'm sure has its own different behaviors.

[injeniero]

To move forward, I'm using a custom version of Scriban where I added support for the missing filters that I'm currently using and the not operator. I'm not using filters with parameters because I don't know if it is worth the effort to keep modifying Scriban to also support that.

[sethjuarez]

Yeah - we've been noodling over implementing an "uber" template language just for these things so all the languages can be unified. In general, you can override the template language and parser mechanism with custom invokers (here's the Liquid one in the new Core runtime we are working on). Would love your thoughts on how we might move forward on this!

[injeniero]

I was also looking for options and there is no template with logic that works across python, c# and JS.

Possible solutions I see are:

• Make the extension capable of using an installed runtime. Most people would use VS Code to edit their prompts while having the proper runtime in place. So if I'm targeting C#, then I can select C# runtime to be used by the extension, same for python or web. I would make the template field required so you can check if the selected runtime supports that template.

• Work on a template engine that can be run in different runtimes. This is a larger effort. In this case I would try to write the engine in C/C++ so it can be accessed from python, c#, Node with native bindings and also provide Wasm builds to be used in the Web.

• Same as previous but using jinja2. There is a C++ port for Jinja2 port that is working and is actively in development. No idea how easy is to make it Wasm compatible.

[injeniero]

I'm now using Jinja2Cpp in my c# runtime. My c# runtime only needs to worry about parsing and rendering the template, the execution is done in another part of the code. This works a lot better now.

Still, because you are using nunjucks in the extensions, there are other unexpected surprises because nunjucks uses the JS engine for expression evaluation, so empty Objects and Arrays evaluate to true, while in Python they are falsy.

After looking for a lib in JS that actually replicates Jinja2 behavior, I found this from huggingface. It has MIT license and it does replicate Jinja2 behavior. They say it is a minimalist implementation, but it seems to be fairly complete. So far, the only missing thing I noticed is the lack of whitespace management.

[sethjuarez]

OH! - Maybe we can add this as an invoker in the new Core runtime??

[injeniero]

This is the C wrapper I built and currently using https://github.com/injeniero/jinja2c