atom.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <title>Haskell Embedded: Functional Programming and Embedded Systems</title>
    <link href="http://haskellembedded.github.io//atom.xml" rel="self" />
    <link href="http://haskellembedded.github.io/" />
    <id>http://haskellembedded.github.io//atom.xml</id>
    <author>
        <name></name>
        <email></email>
    </author>
    <updated>2016-09-23T00:00:00Z</updated>
    <entry>
    <title>Introducing Ion</title>
    <link href="http://haskellembedded.github.io//posts/2016-09-23-introducing-ion.html" />
    <id>http://haskellembedded.github.io//posts/2016-09-23-introducing-ion.html</id>
    <published>2016-09-23T00:00:00Z</published>
    <updated>2016-09-23T00:00:00Z</updated>
    <summary type="html"><![CDATA[<div class="info">
    Posted on September 23, 2016
    
        by Chris Hodapp
    
</div>
<div class="info">
    
    Tags: <a href="/tags/haskell.html">haskell</a>, <a href="/tags/ivory.html">ivory</a>
    
</div>

<h1 id="really-short-version">Really Short Version</h1>
<p><a href="https://github.com/HaskellEmbedded/ion">Ion</a> is a Haskell EDSL that I wrote for concurrent, realtime, embedded programming, targeting the <a href="https://github.com/GaloisInc/ivory">Ivory</a> EDSL. I finally released it to <a href="https://hackage.haskell.org/package/ion">hackage</a>. It’s still rather experimental.</p>
<h1 id="background">Background: Atom &amp; Ivory</h1>
<p>Last year, I wrote a <a href="./2015-02-17-atom-examples.html">few</a> <a href="./2015-02-20-atom-part-2-probes.lhs">posts</a> on <a href="https://hackage.haskell.org/package/atom">Atom</a>. Remember Atom? If not, those posts might give some useful background.</p>
<p>At <a href="./2015-02-06-how-i-got-here.html">some work</a> at my former job, I was already using Atom in conjunction with <a href="https://github.com/GaloisInc/ivory">Ivory</a>, but those two libraries really weren’t made for interfacing with each other. Atom predates Ivory, but they both model certain features of the C language, and as a result have many near-identical-but-incompatible constructs. For some boring details on this, see the <a href="#hackery">section on this hackery</a>.</p>
<p>Sometime after this, I decided to re-implement Atom’s functionality in a more Ivory-friendly way. I looked around in the Atom source code first with the aim of adding an Ivory backend to it, however, I quickly gave up on this as the internals were a bit too dense for me to follow.</p>
<p>This post is badly-overdue, and for that I apologize. For more information, track me (hodapp) down in <a href="irc://chat.freenode.net/%23haskell-embedded">#haskell-embedded</a> or Ion’s GitHub.</p>
<h1 id="ion">Ion</h1>
<p>Here, then, is where I started writing the <a href="https://github.com/HaskellEmbedded/ion">Ion</a> library from scratch. The name ‘Ion’ is a pun that’s a reference to ‘Atom’ and meaning loosely that while Atom is more standalone, Ion exists bonded to another library, Ivory. (I should probably move away from chemistry puns and leave that up to things like <a href="https://www.rust-lang.org/">Rust</a> and <a href="http://www.redox-os.org/">Redox</a>.)</p>
<p>For the most part, I liked the way that Atom worked, and I wanted Ion to behave very similarly. Particularly, I liked the way that I could modularize a specification with the Atom monad, the way that specs could ‘inherit’ schedule parameters, the single run-time report giving the entire schedule of the system, and the checks that Atom’s compiler did to ensure that specifications were sensible.</p>
<p>Ion started here, but diverged somewhat later on. I didn’t manage to match all the features that are in Atom (and I note some of this in Ion’s documentation), and I started down some other paths.</p>
<h2 id="what-is-ion">What is Ion?</h2>
<p>Ion, in brief, is a Haskell EDSL for concurrent, realtime, embedded programming. It targets the <a href="https://github.com/GaloisInc/ivory">Ivory</a> EDSL and is coupled closely with it.</p>
<p>I made Ion to cover two main cases:</p>
<ul>
<li>Scheduling tasks (“tasks” loosely just meaning “little bits of restricted Ivory code”) that needed to execute on very strict timing.</li>
<li>Using continuating-passing style to compose together asychronous tasks (particularly those that call around with callbacks).</li>
</ul>
<p>I sort of gloss over the “why?” of the first part because it’s almost all of the same reasons of why Atom exists. I try to answer the “why?” of the second part, and to some extent the “how?”, below.</p>
<h1 id="async-cps">Async &amp; CPS</h1>
<p>In the application that was using Ion, I started integrating in some support for network communication via a SIM800 GSM modem. This involved many operations of transmitting a command over a UART, waiting for a reply sometime in the future which contained the result of that command - perhaps an HTTP payload or something.</p>
<p>Or, maybe it didn’t - maybe it just contained some minor error, and the command should be retried.</p>
<p>Or, maybe it was a fatal error, and the only thing left to do was try to close down the connections, power off the modem, and power off the UART.</p>
<p>Or, maybe the reply was just total garbage from the UART.</p>
<p>Or, maybe something left the modem in a weird state, and it sent no reply at all because it’s still waiting for us to do something.</p>
<p>The world of rigid, deterministic timing didn’t really have a place for this sort of uncertainly-timed, non-deterministic, divergent behavior (someone’s probably codified this into a theorem or something). Actually, I had tried my best to make some similar and simpler procedures work in Atom. I made specifications which ran with the same rigid timing regardless of when operations actually finished, and to make this reliable, I set that timing to be very slow, and had parts of the specification disabled if earlier steps failed. It worked, but operations took up far more time than needed, and handling anything more divergent than <em>if this fails, don’t run that</em> would become very messy.</p>
<p>This also is a bit tricky to handle in C in any context without threads or coroutines. It almost always will involve callbacks, interrupts, or events - some scope starts an asynchronous operation (e.g. triggering an ADC measurement), and the result comes in the form of an interrupt handler or callback later being called. That callback/interrupt handler/event handler will have to run in a separate scope - which means that any state that needs to make it ‘across’ to that handler cannot reside on the stack. It must be stored in some other form, and recovered at the handler. (I found out at some point that this has been <a href="https://www.usenix.org/legacy/events/usenix02/full_papers/adyahowell/adyahowell_html/index.html">described already</a>: it is called <em>stack ripping</em>, and it comes with event-driven programming.)</p>
<p>That’s annoying as-is, but in my case, I didn’t even have a heap from which to dynamically allocate, so the only remaining option was static memory.</p>
<p>As a side note, Ivory does provide a nice <a href="https://github.com/GaloisInc/ivory/blob/master/ivory/src/Ivory/Language/Coroutine.hs">coroutines</a> implementation, but I ran into two issues with them: They put every variable (whether ‘live’ across a suspend/yield or not) into static memory, and they were not composable. <a href="#coroutines">An appendix section</a> gives some more details on this.</p>
<p>I wanted coroutines I could parametrize over other coroutines (higher-order coroutines?), which seemed to require something like a coroutine whose ‘resume’ continuation and ‘exit’ continuation both were reified rather than implicit. I also wanted it to be in a form that I could interface it with C APIs that wanted function pointers for callbacks or “normal” functions for interrupts.</p>
<p>I do not have a reference on this, but from memory, one of Oleg Kiselyov’s papers defined a coroutine as something like, “two continuations calling each other.” After thinking on this a bit, I realized that coroutines weren’t really the appropriate abstraction; I needed something more general, perhaps like a continuation, because ultimately what I was dealing with was <a href="https://en.wikipedia.org/wiki/Continuation-passing_style">continuation-passing style</a>, and indeed CPS can express other patterns such as exceptions. (After reading extensively about <a href="https://hackage.haskell.org/package/mtl/docs/Control-Monad-Cont.html">Control.Monad.Cont</a> and trying to discern whether I could use <code>Cont</code>, <code>ContT</code>, <code>MonadCont</code>, or <code>callCC</code> to achieve this, I decided that I had even less of an idea than when I started. I was leaning towards “no,” but I still have no idea.)</p>
<p>In the end, I ignored coroutines completely. I made a couple basic abstractions and some plumbing to make them practically usable, and this worked well for me. The <a href="#cps">CPS</a> section gives some further examples on this.</p>
<h1 id="examples">Examples</h1>
<p>The examples here are all rather contrived. All of the “real” work that I did with Ion was proprietary code that I can’t share, and even if I could, it wouldn’t build to an actual embedded target, except in the context of the much larger <a href="https://hackage.haskell.org/package/shake">Shake</a> build it was in and the entire associated toolchain.</p>
<p>However, I’ve tried to create some representative examples to a hypothetical C API. See <a href="https://github.com/HaskellEmbedded/ion/blob/master/src/Ivory/Language/Ion/Examples/Example.hs">Example.hs</a> for these examples in a more buildable form (at least as far as the Haskell part goes).</p>
<h2 id="scheduling">Scheduling</h2>
<p>First, let’s tell Ivory about some (nonexistent) C calls in <code>something.h</code>, with corresponding C prototypes indicated above for those who have no idea how to grok Ivory declarations:</p>
<div class="sourceCode"><pre class="sourceCode haskell"><code class="sourceCode haskell"><span class="co">-- void foo(int16_t)</span>
<span class="ot">foo ::</span> <span class="dt">Def</span> (<span class="ch">&#39;[Sint16] :-&gt; ())</span>
foo <span class="fu">=</span> importProc <span class="st">&quot;foo&quot;</span> <span class="st">&quot;something.h&quot;</span>

<span class="co">-- void bar(int32_t)</span>
<span class="ot">bar ::</span> <span class="dt">Def</span> (<span class="ch">&#39;[Sint32] :-&gt; ())</span>
bar <span class="fu">=</span> importProc <span class="st">&quot;bar&quot;</span> <span class="st">&quot;something.h&quot;</span>

<span class="co">-- uint16_t get_value(int32_t)</span>
<span class="ot">get_value ::</span> <span class="dt">Def</span> (<span class="ch">&#39;[Uint8] :-&gt; Uint16)</span>
get_value <span class="fu">=</span> importProc <span class="st">&quot;get_value&quot;</span> <span class="st">&quot;something.h&quot;</span>

<span class="co">-- bool get_flag(void)</span>
<span class="ot">get_flag ::</span> <span class="dt">Def</span> (<span class="ch">&#39;[] :-&gt; IBool)</span>
get_flag <span class="fu">=</span> importProc <span class="st">&quot;get_flag&quot;</span> <span class="st">&quot;something.h&quot;</span></code></pre></div>
<p>Here’s a top-level spec. As with Atom, <code>period</code> defines what division of a base rate all of its contents inherit - hence, <code>variousPhases</code> (which I define later) runs at 1/100th of the base rate, unless that’s overridden.</p>
<div class="sourceCode"><pre class="sourceCode haskell"><code class="sourceCode haskell"><span class="ot">simpleSchedule ::</span> <span class="dt">Ion</span> ()
simpleSchedule <span class="fu">=</span> ion <span class="st">&quot;schedule&quot;</span> <span class="fu">$</span> <span class="kw">do</span>
  
  period <span class="dv">100</span> <span class="fu">$</span> <span class="kw">do</span>
    variousPhases

  cond ((<span class="fu">&gt;?</span> <span class="dv">10</span>) <span class="fu">&lt;$&gt;</span> call get_value) <span class="fu">$</span> <span class="kw">do</span>
    ivoryEff <span class="fu">$</span> comment <span class="st">&quot;get_value() &gt; 10&quot;</span>
    cond (call get_flag) <span class="fu">$</span> <span class="kw">do</span>
      ivoryEff <span class="fu">$</span> comment <span class="st">&quot;get_value() &gt; 10 &amp;&amp; get_flag()&quot;</span></code></pre></div>
<p>The entire <code>cond</code> block is there to illustrate that parts of a spec can be made conditional. The argument to <code>cond</code> isn’t exactly a boolean, but rather, is an Ivory effect which returns a boolean. One denotes the Ivory effects to weave into all of this with <code>ivoryEff</code>, and in this case, the code does the very boring effect of inserting a C comment with <code>comment</code>.</p>
<p>Here is <code>variousPhases</code>:</p>
<div class="sourceCode"><pre class="sourceCode haskell"><code class="sourceCode haskell"><span class="ot">variousPhases ::</span> <span class="dt">Ion</span> ()
variousPhases <span class="fu">=</span> <span class="kw">do</span>
    phase <span class="dv">1</span> <span class="fu">$</span> ivoryEff <span class="fu">$</span> <span class="kw">do</span>
      comment <span class="st">&quot;period 100, phase 1&quot;</span>
      call_ foo
    phase <span class="dv">10</span> <span class="fu">$</span> ion <span class="st">&quot;optional_tag&quot;</span> <span class="fu">$</span> ivoryEff <span class="fu">$</span> <span class="kw">do</span>
      comment <span class="st">&quot;period 100, phase 10&quot;</span>
      call_ bar
    disable <span class="fu">$</span> phase <span class="dv">20</span> <span class="fu">$</span> ivoryEff <span class="fu">$</span> <span class="kw">do</span>
      comment <span class="st">&quot;shouldn&#39;t even appear in code&quot;</span>
      call_ foo
      call_ bar
    delay <span class="dv">50</span> <span class="fu">$</span> <span class="kw">do</span>
      p <span class="ot">&lt;-</span> getSched
      ivoryEff <span class="fu">$</span> <span class="kw">do</span>
        comment <span class="st">&quot;Should be phase 100 + 50&quot;</span>
        comment (<span class="st">&quot;Reported sched: &quot;</span> <span class="fu">++</span> show p)
      delay <span class="dv">10</span> <span class="fu">$</span> ion <span class="st">&quot;moreDelay&quot;</span> <span class="fu">$</span> <span class="kw">do</span>
        p <span class="ot">&lt;-</span> getSched
        ivoryEff <span class="fu">$</span> <span class="kw">do</span>
          comment <span class="st">&quot;Should be phase 100 + 50 + 10&quot;</span>
          comment (<span class="st">&quot;Reported sched: &quot;</span> <span class="fu">++</span> show p)
      phase <span class="dv">1</span> <span class="fu">$</span> <span class="kw">do</span>
        ivoryEff <span class="fu">$</span> comment <span class="st">&quot;Should override to phase 1&quot;</span>
    period <span class="dv">1000</span> <span class="fu">$</span> <span class="kw">do</span>
      ivoryEff <span class="fu">$</span> comment <span class="st">&quot;Should override all other period&quot;</span></code></pre></div>
<p>This is full of <code>phase</code>, which behaves more like <code>exactPhase</code> from Atom, specifying that its contents should execute at some specific offset of the current period. I don’t yet have support for the exact semantics of Atom’s <code>phase</code>, which means something more like, “schedule it then, at the earliest”.</p>
<p><code>delay</code> has no exact analogue in Atom, but it simply specifies that something executes at some relative phase past the phase that was inherited. That is, if you nested a series of <code>delay 1</code>, they’d all proceed one tick apart, starting at what phase they inherited.</p>
<p>If we run all this through Ion and Ivory with <code>ionCompile ivoryOpts &quot;simpleSchedule&quot; simpleSchedule</code> then the below C code results:</p>
<div class="sourceCode"><pre class="sourceCode c"><code class="sourceCode c"><span class="co">// module simpleSchedule Source:</span>

<span class="ot">#include &quot;simpleSchedule.h&quot;</span>
<span class="dt">uint8_t</span> counter_schedule_0 = (<span class="dt">uint8_t</span>) 1U;
<span class="dt">uint8_t</span> counter_optional_tag_1 = (<span class="dt">uint8_t</span>) 10U;
<span class="dt">uint8_t</span> counter_schedule_2 = (<span class="dt">uint8_t</span>) 50U;
<span class="dt">uint8_t</span> counter_moreDelay_3 = (<span class="dt">uint8_t</span>) 60U;
<span class="dt">uint8_t</span> counter_schedule_4 = (<span class="dt">uint8_t</span>) 1U;
<span class="dt">uint16_t</span> counter_schedule_5 = (<span class="dt">uint16_t</span>) 0U;
<span class="dt">uint8_t</span> counter_schedule_6 = (<span class="dt">uint8_t</span>) 0U;
<span class="dt">uint8_t</span> counter_schedule_7 = (<span class="dt">uint8_t</span>) 0U;
<span class="dt">void</span> simpleSchedule(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule entry procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: schedule */</span>
    <span class="kw">if</span> ((bool) ((<span class="dt">uint8_t</span>) 0U == counter_schedule_0)) {
        ion_schedule_0();
        counter_schedule_0 = (<span class="dt">uint8_t</span>) 99U;
    } <span class="kw">else</span> {
        counter_schedule_0 = (<span class="dt">uint8_t</span>) (counter_schedule_0 - (<span class="dt">uint8_t</span>) 1U);
    }
    <span class="co">/* Path: schedule.optional_tag */</span>
    <span class="kw">if</span> ((bool) ((<span class="dt">uint8_t</span>) 0U == counter_optional_tag_1)) {
        ion_optional_tag_1();
        counter_optional_tag_1 = (<span class="dt">uint8_t</span>) 99U;
    } <span class="kw">else</span> {
        counter_optional_tag_1 = (<span class="dt">uint8_t</span>) (counter_optional_tag_1 - (<span class="dt">uint8_t</span>) 1U);
    }
    <span class="co">/* Path: schedule */</span>
    <span class="kw">if</span> ((bool) ((<span class="dt">uint8_t</span>) 0U == counter_schedule_2)) {
        ion_schedule_2();
        counter_schedule_2 = (<span class="dt">uint8_t</span>) 99U;
    } <span class="kw">else</span> {
        counter_schedule_2 = (<span class="dt">uint8_t</span>) (counter_schedule_2 - (<span class="dt">uint8_t</span>) 1U);
    }
    <span class="co">/* Path: schedule.moreDelay */</span>
    <span class="kw">if</span> ((bool) ((<span class="dt">uint8_t</span>) 0U == counter_moreDelay_3)) {
        ion_moreDelay_3();
        counter_moreDelay_3 = (<span class="dt">uint8_t</span>) 99U;
    } <span class="kw">else</span> {
        counter_moreDelay_3 = (<span class="dt">uint8_t</span>) (counter_moreDelay_3 - (<span class="dt">uint8_t</span>) 1U);
    }
    <span class="co">/* Path: schedule */</span>
    <span class="kw">if</span> ((bool) ((<span class="dt">uint8_t</span>) 0U == counter_schedule_4)) {
        ion_schedule_4();
        counter_schedule_4 = (<span class="dt">uint8_t</span>) 99U;
    } <span class="kw">else</span> {
        counter_schedule_4 = (<span class="dt">uint8_t</span>) (counter_schedule_4 - (<span class="dt">uint8_t</span>) 1U);
    }
    <span class="co">/* Path: schedule */</span>
    <span class="kw">if</span> ((bool) ((<span class="dt">uint16_t</span>) 0U == counter_schedule_5)) {
        ion_schedule_5();
        counter_schedule_5 = (<span class="dt">uint16_t</span>) 999U;
    } <span class="kw">else</span> {
        counter_schedule_5 = (<span class="dt">uint16_t</span>) (counter_schedule_5 - (<span class="dt">uint16_t</span>) 1U);
    }
    <span class="co">/* Path: schedule */</span>
    <span class="kw">if</span> ((bool) ((<span class="dt">uint8_t</span>) 0U == counter_schedule_6)) {
        ion_schedule_6();
        counter_schedule_6 = (<span class="dt">uint8_t</span>) 0U;
    } <span class="kw">else</span> {
        counter_schedule_6 = (<span class="dt">uint8_t</span>) (counter_schedule_6 - (<span class="dt">uint8_t</span>) 1U);
    }
    <span class="co">/* Path: schedule */</span>
    <span class="kw">if</span> ((bool) ((<span class="dt">uint8_t</span>) 0U == counter_schedule_7)) {
        ion_schedule_7();
        counter_schedule_7 = (<span class="dt">uint8_t</span>) 0U;
    } <span class="kw">else</span> {
        counter_schedule_7 = (<span class="dt">uint8_t</span>) (counter_schedule_7 - (<span class="dt">uint8_t</span>) 1U);
    }
}
<span class="dt">void</span> ion_schedule_0(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: schedule */</span>
    <span class="co">/* Phase: 1 */</span>
    <span class="co">/* Period: 100 */</span>
    <span class="co">/* Action has no conditions */</span>
    <span class="co">/* period 100, phase 1 */</span>
    foo();
}
<span class="dt">void</span> ion_optional_tag_1(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: schedule.optional_tag */</span>
    <span class="co">/* Phase: 10 */</span>
    <span class="co">/* Period: 100 */</span>
    <span class="co">/* Action has no conditions */</span>
    <span class="co">/* period 100, phase 10 */</span>
    bar();
}
<span class="dt">void</span> ion_schedule_2(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: schedule */</span>
    <span class="co">/* Phase: 50 */</span>
    <span class="co">/* Period: 100 */</span>
    <span class="co">/* Action has no conditions */</span>
    <span class="co">/* Should be phase 100 + 50 */</span>
    <span class="co">/* Reported sched: Schedule {schedId = 0, schedName = &quot;schedule&quot;, schedPath = [&quot;schedule&quot;], schedPhase = 50, schedPeriod = 100, schedAction = [], schedCond = []} */</span>
    ;
}
<span class="dt">void</span> ion_moreDelay_3(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: schedule.moreDelay */</span>
    <span class="co">/* Phase: 60 */</span>
    <span class="co">/* Period: 100 */</span>
    <span class="co">/* Action has no conditions */</span>
    <span class="co">/* Should be phase 100 + 50 + 10 */</span>
    <span class="co">/* Reported sched: Schedule {schedId = 0, schedName = &quot;moreDelay&quot;, schedPath = [&quot;schedule&quot;,&quot;moreDelay&quot;], schedPhase = 60, schedPeriod = 100, schedAction = [], schedCond = []} */</span>
    ;
}
<span class="dt">void</span> ion_schedule_4(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: schedule */</span>
    <span class="co">/* Phase: 1 */</span>
    <span class="co">/* Period: 100 */</span>
    <span class="co">/* Action has no conditions */</span>
    <span class="co">/* Should override to phase 1 */</span>
    ;
}
<span class="dt">void</span> ion_schedule_5(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: schedule */</span>
    <span class="co">/* Phase: 0 */</span>
    <span class="co">/* Period: 1000 */</span>
    <span class="co">/* Action has no conditions */</span>
    <span class="co">/* Should override all other period */</span>
    ;
}
<span class="dt">void</span> ion_schedule_6(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: schedule */</span>
    <span class="co">/* Phase: 0 */</span>
    <span class="co">/* Period: 1 */</span>
    <span class="co">/* Action has 1 conditions: */</span>
    ;
    
    <span class="dt">uint16_t</span> n_r0 = get_value();
    
    <span class="kw">if</span> ((bool) (n_r0 &gt; (<span class="dt">uint16_t</span>) 10U)) {
        <span class="co">/* get_value() &gt; 10 */</span>
        ;
    }
}
<span class="dt">void</span> ion_schedule_7(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: schedule */</span>
    <span class="co">/* Phase: 0 */</span>
    <span class="co">/* Period: 1 */</span>
    <span class="co">/* Action has 2 conditions: */</span>
    ;
    
    bool n_r0 = get_flag();
    <span class="dt">uint16_t</span> n_r1 = get_value();
    
    <span class="kw">if</span> ((bool) (n_r0 &amp;&amp; (bool) (n_r1 &gt; (<span class="dt">uint16_t</span>) 10U))) {
        <span class="co">/* get_value() &gt; 10 &amp;&amp; get_flag() */</span>
        ;
    }
}</code></pre></div>
<p>and this header:</p>
<div class="sourceCode"><pre class="sourceCode c"><code class="sourceCode c"><span class="co">// module simpleSchedule Header:</span>

<span class="ot">#include &quot;ivory.h&quot;</span>
<span class="kw">extern</span> <span class="dt">uint8_t</span> counter_schedule_0;
<span class="kw">extern</span> <span class="dt">uint8_t</span> counter_optional_tag_1;
<span class="kw">extern</span> <span class="dt">uint8_t</span> counter_schedule_2;
<span class="kw">extern</span> <span class="dt">uint8_t</span> counter_moreDelay_3;
<span class="kw">extern</span> <span class="dt">uint8_t</span> counter_schedule_4;
<span class="kw">extern</span> <span class="dt">uint16_t</span> counter_schedule_5;
<span class="kw">extern</span> <span class="dt">uint8_t</span> counter_schedule_6;
<span class="kw">extern</span> <span class="dt">uint8_t</span> counter_schedule_7;
<span class="dt">void</span> simpleSchedule(<span class="dt">void</span>);
<span class="dt">void</span> ion_schedule_0(<span class="dt">void</span>);
<span class="dt">void</span> ion_optional_tag_1(<span class="dt">void</span>);
<span class="dt">void</span> ion_schedule_2(<span class="dt">void</span>);
<span class="dt">void</span> ion_moreDelay_3(<span class="dt">void</span>);
<span class="dt">void</span> ion_schedule_4(<span class="dt">void</span>);
<span class="dt">void</span> ion_schedule_5(<span class="dt">void</span>);
<span class="dt">void</span> ion_schedule_6(<span class="dt">void</span>);
<span class="dt">void</span> ion_schedule_7(<span class="dt">void</span>);</code></pre></div>
<p><code>simpleSchedule</code> is the most important of this: This is the main scheduling function (“schedule entry procedure”) that must be called at the base rate through a timer, an interrupt, or something of the sort for anything to work right. Often this must be set up outside of Ivory because Ivory really isn’t interested in whatever C/ASM black magic the timer requires - go do that, hide it, and don’t let Ivory know about it. Also, the name of that function is what we supplied to <code>ionCompile</code>.</p>
<p>It has also produced several variables, all of the <code>counter_</code> ones, which are used for establishing the correct periods and phases. Take a look in <code>simpleSchedule</code> and the pattern should be clear (as well as near-identical to how Atom does it, since I basically copied its method): Each Ivory effect in a particular period/phase context turns into a branch, a function, and a counter variable. The counter variable starts at the respective phase, and each branch is responsible for resetting it to the respective period, decrementing the counter, and calling the function that contains the Ivory effects. Note also that <code>counter_schedule_5</code> uses a <code>uint16_t</code> - it corresponds to the <code>period 1000</code> block, so a larger counter variable is needed to fit that range. Note also in <code>ion_schedule_7</code> that nesting conditions with <code>cond</code> leads to all of them applying (i.e. logical AND).</p>
<p>Read through the code in some more detail, and it should be clear how specifications map and flatten out to generated C code. The names given in <code>ion</code> are for the most part just for documentation purposes; Ion is perfectly content to generate unique C identifiers without help. Note that each point has a sort of “path” that led to it, and that path determines how it is scheduled.</p>
<h2 id="timers">Timers</h2>
<p>Here’s a shorter example that incorporates Ion’s resettable timers (note the use of <code>mdo</code> so we may define the timer first; this will need <code>{-# LANGUAGE RecursiveDo #-}</code>):</p>
<div class="sourceCode"><pre class="sourceCode haskell"><code class="sourceCode haskell"><span class="ot">exampleTimer ::</span> <span class="dt">Ion</span> (<span class="dt">Def</span> (<span class="ch">&#39;[] &#39;</span><span class="fu">:-&gt;</span> ()))
exampleTimer <span class="fu">=</span> ion <span class="st">&quot;timer&quot;</span> <span class="fu">$</span> mdo

  <span class="co">-- Timer is initialized with a Uint16; procedure called at</span>
  <span class="co">-- expiration is fixed at compile-time:</span>
  timer1 <span class="ot">&lt;-</span> period <span class="dv">1</span> <span class="fu">$</span> timer (<span class="dt">Proxy</span><span class="ot"> ::</span> <span class="dt">Proxy</span> <span class="dt">Uint16</span>) expire

  <span class="co">-- Initialization procedure:</span>
  init <span class="ot">&lt;-</span> newProc <span class="fu">$</span> body <span class="fu">$</span> <span class="kw">do</span>
    <span class="co">-- Trigger the timer for 1000 ticks:</span>
    startTimer timer1 <span class="dv">1000</span>
  
  expire <span class="ot">&lt;-</span> newProc <span class="fu">$</span> body <span class="fu">$</span> <span class="kw">do</span>
    call_ printf <span class="st">&quot;Timer expired!\r\n&quot;</span>

  return init</code></pre></div>
<p>This makes use of the <code>Ion</code> monad to return the entry procedure, which is required to start the timer counting in the first place. Also, due to some limitations, the timer’s behavior upon expiration is fixed at compile-time, though its countdown time is not (hence the 1000 we pass to <code>startTimer</code>). Rather than calling some external procedure when the timer expires (as we did in the prior section), we use <code>newProc</code> to define a procedure right there, and Ion takes care of giving it a name and having Ivory include it. To simplify things, we put the timer inside of <code>period 1</code> so it counts at the base rate - but if, for instance, the base rate were 1 millisecond, we might sensibly put the timer inside period 1000 so that its countdowns are all in seconds.</p>
<p>The resultant C source is below:</p>
<div class="sourceCode"><pre class="sourceCode c"><code class="sourceCode c"><span class="co">// module timer Source:</span>

<span class="ot">#include &quot;timer.h&quot;</span>
<span class="dt">uint16_t</span> timer_0 = (<span class="dt">uint16_t</span>) 0U;
<span class="dt">uint8_t</span> counter_decr_0 = (<span class="dt">uint8_t</span>) 0U;
<span class="dt">void</span> timer_1(<span class="dt">void</span>)
{
    timer_0 = (<span class="dt">uint16_t</span>) 1000U;
}
<span class="dt">void</span> timer_2(<span class="dt">void</span>)
{
    printf(<span class="st">&quot;Timer expired!</span><span class="ch">\r\n</span><span class="st">&quot;</span>);
}
<span class="dt">void</span> timer(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule entry procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: timer.timer_0.decr */</span>
    <span class="kw">if</span> ((bool) ((<span class="dt">uint8_t</span>) 0U == counter_decr_0)) {
        ion_decr_0();
        counter_decr_0 = (<span class="dt">uint8_t</span>) 0U;
    } <span class="kw">else</span> {
        counter_decr_0 = (<span class="dt">uint8_t</span>) (counter_decr_0 - (<span class="dt">uint8_t</span>) 1U);
    }
}
<span class="dt">void</span> ion_decr_0(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule procedure from Ion &amp; Ivory */</span>
    <span class="co">/* Path: timer.timer_0.decr */</span>
    <span class="co">/* Phase: 0 */</span>
    <span class="co">/* Period: 1 */</span>
    <span class="co">/* Action has no conditions */</span>
    ;
    
    <span class="dt">uint16_t</span> n_deref0 = timer_0;
    
    <span class="kw">if</span> ((bool) ((<span class="dt">uint16_t</span>) 0U == n_deref0)) { } <span class="kw">else</span> {
        <span class="dt">uint16_t</span> n_cse1 = (<span class="dt">uint16_t</span>) (n_deref0 - (<span class="dt">uint16_t</span>) 1U);
        
        timer_0 = n_cse1;
        <span class="kw">if</span> ((bool) (n_cse1 &gt; (<span class="dt">uint16_t</span>) 0U)) { } <span class="kw">else</span> {
            timer_2();
        }
    }
}</code></pre></div>
<p>and the header:</p>
<div class="sourceCode"><pre class="sourceCode c"><code class="sourceCode c"><span class="co">// module timer Header:</span>

<span class="ot">#include &quot;ivory.h&quot;</span>
<span class="kw">extern</span> <span class="dt">uint16_t</span> timer_0;
<span class="kw">extern</span> <span class="dt">uint8_t</span> counter_decr_0;
<span class="dt">void</span> timer_1(<span class="dt">void</span>);
<span class="dt">void</span> timer_2(<span class="dt">void</span>);
<span class="dt">void</span> timer(<span class="dt">void</span>);
<span class="dt">void</span> ion_decr_0(<span class="dt">void</span>);</code></pre></div>
<p>This should look similar to the last section, but with the addition now of <code>timer_0</code>. Our procedure with <code>newProc</code> was also turned into the C function <code>timer_2</code>.</p>
<h2 id="cps">CPS</h2>
<p>The next example might be a bit trickier to understand, but I hope that it both explains both this functionality and what I mean when I talk about ‘composing’ things.</p>
<p>I’ll start with a smaller piece rather than a larger one this time. Suppose we’re dealing with some kind of async API with <code>uint16_t transmit_async(uint16_t opcode, void (*callback)(uint16_t))</code> - it transmits <code>opcode</code>, it returns some kind of immediate success/error code as a <code>uint16_t</code> (perhaps an error indicates a failure to even transmit), and when the opcode returns a result, then it calls <code>callback</code> with some payload.</p>
<p>Given the right machinery, this actually composes pretty easily with continuation-passing style. Consider the below, which connects this function to two other callbacks - a different “error” callback (perhaps we produce our own error code of another format), and a “success” callback - and returns for us an initialization procedure:</p>
<div class="sourceCode"><pre class="sourceCode haskell"><code class="sourceCode haskell"><span class="ot">exampleSend ::</span> <span class="dt">Word16</span> <span class="co">-- ^ Payload value (or something like that)</span>
               <span class="ot">-&gt;</span> <span class="dt">Def</span> (<span class="ch">&#39;[Uint32] &#39;</span><span class="fu">:-&gt;</span> ()) <span class="co">-- ^ Error callback</span>
               <span class="ot">-&gt;</span> <span class="dt">Def</span> (<span class="ch">&#39;[Uint16] &#39;</span><span class="fu">:-&gt;</span> ()) <span class="co">-- ^ Success callback</span>
               <span class="ot">-&gt;</span> <span class="dt">Ion</span> (<span class="dt">Def</span> (<span class="ch">&#39;[] &#39;</span><span class="fu">:-&gt;</span> ()))
exampleSend payload err succ <span class="fu">=</span> mdo

  <span class="kw">let</span><span class="ot"> transmit_async ::</span> <span class="dt">Def</span> (<span class="ch">&#39;[Uint16, ProcPtr (&#39;</span>[<span class="dt">Uint16</span>] <span class="fu">:-&gt;</span> ())] <span class="fu">:-&gt;</span> <span class="dt">Uint32</span>)
      transmit_async <span class="fu">=</span> importProc <span class="st">&quot;transmit_async&quot;</span> <span class="st">&quot;foo.h&quot;</span>

  write <span class="ot">&lt;-</span> newProc <span class="fu">$</span> body <span class="fu">$</span> <span class="kw">do</span>
    comment <span class="fu">$</span> <span class="st">&quot;Transmit value: &quot;</span> <span class="fu">++</span> show payload
    <span class="co">-- Tell transmit_async to transmit this, and call us back at &#39;recv&#39;</span>
    <span class="co">-- (which we define after):</span>
    errCode <span class="ot">&lt;-</span> call transmit_async (fromIntegral payload) <span class="fu">$</span> procPtr recv
    <span class="co">-- Check for a nonzero error code:</span>
    ifte_ (errCode <span class="fu">/=?</span> <span class="dv">0</span>)
      (call_ err errCode)
      <span class="fu">$</span> return ()

  recv <span class="ot">&lt;-</span> newProc <span class="fu">$</span> \value <span class="ot">-&gt;</span> body <span class="fu">$</span> <span class="kw">do</span>
    <span class="co">-- Say that hypothetically we should have received the same value</span>
    <span class="co">-- back, so check this first:</span>
    ifte_ (value <span class="fu">/=?</span> fromIntegral payload)
      <span class="co">-- If a mismatch, then call the error handler with some code:</span>
      (call_ err <span class="bn">0x12345678</span>)
      <span class="co">-- Otherwise, call the success handler:</span>
      <span class="fu">$</span> call_ succ value

  return write</code></pre></div>
<p>Remember that timer module we used in the last section, and how it was parametrized over a callback to be called upon expiration? We could probably easily rig this into <code>exampleSend</code> so that a failure to receive a reply within N milliseconds would trigger the error callback. (I’m not going to do that - this is just an example of how one can combine callbacks.)</p>
<p>Slight aside: You might notice the use of <code>mdo</code> again. This is because otherwise the definitions would have to go in reverse order of their calls. Consider the following:</p>
<div class="sourceCode"><pre class="sourceCode haskell"><code class="sourceCode haskell">foo <span class="fu">=</span> <span class="kw">do</span>
  call1 <span class="ot">&lt;-</span> newProc <span class="fu">$</span> body <span class="fu">$</span> call_ call2
  call2 <span class="ot">&lt;-</span> newProc <span class="fu">$</span> body <span class="fu">$</span> call_ call3
  call3 <span class="ot">&lt;-</span> newProc <span class="fu">$</span> body <span class="fu">$</span> call_ call4
  call4 <span class="ot">&lt;-</span> newProc <span class="fu">$</span> body <span class="fu">$</span> call_ call5
  <span class="fu">...</span>
  callN <span class="ot">&lt;-</span> newProc <span class="fu">$</span> body <span class="fu">$</span> retVoid</code></pre></div>
<p><code>call1</code> calls <code>call2</code>, which calls <code>call3</code>, which calls <code>call4</code>, and so on. They’re in the order in which they’d be called. However, this is invalid code, because the definition of <code>call1</code> relies on a definition that comes later. If we actually reordered these to be valid, then definitions would be backwards from the order in which they’re called. The simplest solution I found was just to use <code>mdo</code>.</p>
<p>Back to <code>exampleSend</code>: Now that it’s defined in this format, we can build it up into larger things. Consider the below definition which invokes it three times:</p>
<div class="sourceCode"><pre class="sourceCode haskell"><code class="sourceCode haskell"><span class="ot">exampleChain ::</span> <span class="dt">Ion</span> (<span class="dt">Def</span> (<span class="ch">&#39;[] &#39;</span><span class="fu">:-&gt;</span> ()))
exampleChain <span class="fu">=</span> mdo
  <span class="kw">let</span><span class="ot"> error ::</span> <span class="dt">Def</span> (<span class="ch">&#39;[Uint32] :-&gt; ())</span>
      error <span class="fu">=</span> importProc <span class="st">&quot;assert_error&quot;</span> <span class="st">&quot;something.h&quot;</span>

  init <span class="ot">&lt;-</span> exampleSend <span class="bn">0x1234</span> error <span class="fu">=&lt;&lt;</span>
          adapt_0_1 <span class="fu">=&lt;&lt;</span> exampleSend <span class="bn">0x2345</span> error <span class="fu">=&lt;&lt;</span>
          adapt_0_1 <span class="fu">=&lt;&lt;</span> exampleSend <span class="bn">0x3456</span> error <span class="fu">=&lt;&lt;</span>
          adapt_0_1 <span class="fu">=&lt;&lt;</span> exampleSend <span class="bn">0x4567</span> error success

  success <span class="ot">&lt;-</span> newProc <span class="fu">$</span> \_ <span class="ot">-&gt;</span> body <span class="fu">$</span> <span class="kw">do</span>
    call_ printf <span class="st">&quot;All calls succeeded!\r\n&quot;</span>

  return init</code></pre></div>
<p>This chains together 4 calls to <code>exampleSend</code> to create a procedure that will try to send opcodes 0x1234, 0x2345, 0x3456, and 0x4567. If any step fails, <code>assert_error</code> is called (though we could just as easily parametrize <code>exampleChain</code> over a separate error callback). If every step succeeds, it calls the internal procedure <code>success</code> (of course, again, we could parametrize over this callback too). It returns the entry procedure which starts this entire process.</p>
<p>I haven’t explained <code>adapt_0_1</code> yet, but it’s just a piece of plumbing to stick together mismatched C functions. The success callback takes a single <code>Uint16</code>, but we for whatever reason don’t need it, so <code>adapt_0_1</code> throws away that value and just calls the entry procedure of <code>exampleSend</code> (which takes no arguments).</p>
<p>This produces the below C code:</p>
<div class="sourceCode"><pre class="sourceCode c"><code class="sourceCode c"><span class="co">// module exampleChain Source:</span>

<span class="ot">#include &quot;exampleChain.h&quot;</span>
<span class="dt">void</span> exampleChain_0(<span class="dt">void</span>)
{
    <span class="co">/* Transmit value: 17767 */</span>
    ;
    
    <span class="dt">uint32_t</span> n_r0 = transmit_async((<span class="dt">uint16_t</span>) 17767U, exampleChain_1);
    
    <span class="kw">if</span> ((bool) ((<span class="dt">uint32_t</span>) 0U != n_r0)) {
        assert_error(n_r0);
    }
}
<span class="dt">void</span> exampleChain_1(<span class="dt">uint16_t</span> n_var0)
{
    <span class="kw">if</span> ((bool) (n_var0 != (<span class="dt">uint16_t</span>) 17767U)) {
        assert_error((<span class="dt">uint32_t</span>) 305419896U);
    } <span class="kw">else</span> {
        exampleChain_11(n_var0);
    }
}
<span class="dt">void</span> exampleChain_2(<span class="dt">uint16_t</span> n_var0)
{
    exampleChain_0();
}
<span class="dt">void</span> exampleChain_3(<span class="dt">void</span>)
{
    <span class="co">/* Transmit value: 13398 */</span>
    ;
    
    <span class="dt">uint32_t</span> n_r0 = transmit_async((<span class="dt">uint16_t</span>) 13398U, exampleChain_4);
    
    <span class="kw">if</span> ((bool) ((<span class="dt">uint32_t</span>) 0U != n_r0)) {
        assert_error(n_r0);
    }
}
<span class="dt">void</span> exampleChain_4(<span class="dt">uint16_t</span> n_var0)
{
    <span class="kw">if</span> ((bool) (n_var0 != (<span class="dt">uint16_t</span>) 13398U)) {
        assert_error((<span class="dt">uint32_t</span>) 305419896U);
    } <span class="kw">else</span> {
        exampleChain_2(n_var0);
    }
}
<span class="dt">void</span> exampleChain_5(<span class="dt">uint16_t</span> n_var0)
{
    exampleChain_3();
}
<span class="dt">void</span> exampleChain_6(<span class="dt">void</span>)
{
    <span class="co">/* Transmit value: 9029 */</span>
    ;
    
    <span class="dt">uint32_t</span> n_r0 = transmit_async((<span class="dt">uint16_t</span>) 9029U, exampleChain_7);
    
    <span class="kw">if</span> ((bool) ((<span class="dt">uint32_t</span>) 0U != n_r0)) {
        assert_error(n_r0);
    }
}
<span class="dt">void</span> exampleChain_7(<span class="dt">uint16_t</span> n_var0)
{
    <span class="kw">if</span> ((bool) (n_var0 != (<span class="dt">uint16_t</span>) 9029U)) {
        assert_error((<span class="dt">uint32_t</span>) 305419896U);
    } <span class="kw">else</span> {
        exampleChain_5(n_var0);
    }
}
<span class="dt">void</span> exampleChain_8(<span class="dt">uint16_t</span> n_var0)
{
    exampleChain_6();
}
<span class="dt">void</span> exampleChain_9(<span class="dt">void</span>)
{
    <span class="co">/* Transmit value: 4660 */</span>
    ;
    
    <span class="dt">uint32_t</span> n_r0 = transmit_async((<span class="dt">uint16_t</span>) 4660U, exampleChain_10);
    
    <span class="kw">if</span> ((bool) ((<span class="dt">uint32_t</span>) 0U != n_r0)) {
        assert_error(n_r0);
    }
}
<span class="dt">void</span> exampleChain_10(<span class="dt">uint16_t</span> n_var0)
{
    <span class="kw">if</span> ((bool) (n_var0 != (<span class="dt">uint16_t</span>) 4660U)) {
        assert_error((<span class="dt">uint32_t</span>) 305419896U);
    } <span class="kw">else</span> {
        exampleChain_8(n_var0);
    }
}
<span class="dt">void</span> exampleChain_11(<span class="dt">uint16_t</span> n_var0)
{
    printf(<span class="st">&quot;All calls succeeded!</span><span class="ch">\r\n</span><span class="st">&quot;</span>);
}
<span class="dt">void</span> exampleChain(<span class="dt">void</span>)
{
    <span class="co">/* Auto-generated schedule entry procedure from Ion &amp; Ivory */</span>
    ;
}</code></pre></div>
<p>and the below C header:</p>
<div class="sourceCode"><pre class="sourceCode c"><code class="sourceCode c"><span class="co">// module exampleChain Header:</span>

<span class="ot">#include &quot;ivory.h&quot;</span>
<span class="dt">void</span> exampleChain_0(<span class="dt">void</span>);
<span class="dt">void</span> exampleChain_1(<span class="dt">uint16_t</span> n_var0);
<span class="dt">void</span> exampleChain_2(<span class="dt">uint16_t</span> n_var0);
<span class="dt">void</span> exampleChain_3(<span class="dt">void</span>);
<span class="dt">void</span> exampleChain_4(<span class="dt">uint16_t</span> n_var0);
<span class="dt">void</span> exampleChain_5(<span class="dt">uint16_t</span> n_var0);
<span class="dt">void</span> exampleChain_6(<span class="dt">void</span>);
<span class="dt">void</span> exampleChain_7(<span class="dt">uint16_t</span> n_var0);
<span class="dt">void</span> exampleChain_8(<span class="dt">uint16_t</span> n_var0);
<span class="dt">void</span> exampleChain_9(<span class="dt">void</span>);
<span class="dt">void</span> exampleChain_10(<span class="dt">uint16_t</span> n_var0);
<span class="dt">void</span> exampleChain_11(<span class="dt">uint16_t</span> n_var0);
<span class="dt">void</span> exampleChain(<span class="dt">void</span>);</code></pre></div>
<p>Note that this is all done with no static variables. It has no need to store a continuation, call stack, or anything of the sort. Of course, it’s not magic. You still need to use static variables for state that lives across calls, and external APIs have to store their callbacks someplace.</p>
<p>Note one particular limitation here (the same one as with <code>timer</code>): When you parametrize an <code>Ion</code> over procedures, those procedures are fixed at C compile-time. Ivory may now allow some means of storing function pointers in such a manner that you could work past this limitation, but if memory serves me, when I wrote this Ivory had no way to store a function pointer in static memory. I accepted this limitation because it made for much better-defined behavior, as I saw it.</p>
<p>This is fairly representative of the way that I used this functionality, just a bit simpler. When I used it for network communications, it took a form more akin to exception handling, as it would run through a sequence of steps for which the proper way to handle a failure differed depending on step. In that instance, I had an entire chain of shutdown steps (e.g. close TCP connection; drop modem connection; turn off modem; turn off UART; and disable power source), and callbacks would jump to different parts of it.</p>
<h1 id="conclusions">Conclusions</h1>
<p>Ion helped me immensely with generating large amounts of C plumbing whose behavior was very easy to reason about. It also cut both ways, as it made it trivial to very quickly occupy a lot of RAM, generate a lot of unnecessary code, and produce huge binaries.</p>
<p>Much of the aforementioned plumbing was to bundle together the boilerplate that Ivory requires - things like <code>incl</code> for every single function that actually needs to be a C function by way of an Ivory procedure. In Haskell-world, one doesn’t normally think of having to explicitly declare each function, but Ivory inherits this C limitation. When Ion has to generate a lot of Ivory procedures and variables (particularly when I’m using it to parametrize these definitions over something), it has to accumulate all of this somehow and get it to Ivory’s compiler.</p>
<p>I tried to fix the bugs in Ion that pertained to correctness, but it still has a lot of room for improvement when it comes to efficiency and comprehensibility of the generated code for scheduling, and part of that is because it errs on the side of paranoia.</p>
<h1 id="hackery"><em>Appendix 1: Atom &amp; Ivory hackery</em></h1>
<p>Atom and Ivory both generate C code, and to that end, both express many of the same features of the C programming language - very central things like expressions, variables, conditionals, and the C type system. However, they have different purposes in mind (otherwise, why would I be using both?) and were never really meant to interface with each other. The way they model those features then is identical or similar, but incompatible.</p>
<p>However, I needed the two libraries to generate C code in the same build, to be able to refer to Atom variables from Ivory (and vice versa), and to have something in Ivory responsible for calling Atom’s main tick function.</p>
<p>I ended up resorting to hacks like giving Atom and Ivory variables the same C name and relying on some incidental details of how to refer to functions. This worked, but it was ugly and error-prone, and it also bypassed proper typing. If I mismatched the names, the C code would probably fail to build. If I mismatched the types (for instance, my Atom variable is a <code>uint32_t</code> by way of <a href="http://hackage.haskell.org/package/atom-1.0.12/docs/Language-Atom-Expressions.html#t:V">V Word32</a> and my Ivory variable is a <code>int16_t</code> by way of <a href="https://hackage.haskell.org/package/ivory-0.1.0.0/docs/Ivory-Language.html#t:Sint16">Sint16</a>), the generated C code might have had some subtle errors. In either case, Haskell saw no problems in type-checking, because I was only coincidentally coupling the two variables via the generated C code. This could be particularly nasty when I was trying to match pointer types properly, and cheating a little by writing the variable’s name as <code>&amp;foo</code> or <code>*foo</code>.</p>
<p>As an aside, if I remember right, a fair number of the bugs discovered in the code were a direct result of me bypassing the type system in this manner.</p>
<h1 id="coroutines"><em>Appendix 2: Limitations on Coroutines</em></h1>
<p>Two pernicious limitation I ran into on Ivory’s coroutines were an inability to build up coroutines out of smaller parts, and an inability to take the <em>yield</em> escape hatch that a coroutine provided and pass it around like a first-class value. This wasn’t a slight against them - they are coroutines, behaving like coroutines but inheriting the C-derived limitations that Ivory purposely has, and I was trying to make them behave like something else.</p>
<p>Consider coroutine A and coroutine B. As far as control flow goes, coroutine A can do a few things:</p>
<ul>
<li>return back to caller</li>
<li>suspend itself with <em>yield</em></li>
<li>resume coroutine B (or call it in the first place)</li>
</ul>
<p><em>(It can’t call itself, but that’s incidental here; Ivory’s coroutines store a continuation in static memory, so only one can be “live” at once.)</em></p>
<p>However, returning and yielding don’t have any meaningful first-class form. Coroutine A can’t pass its own “return” to coroutine B, and let B return a value to its caller, nor can it pass its own “yield” elsewhere and let another context suspend it.</p>
<p>In other words: these coroutines don’t provide their own continuations or an “exit” continuation; that’s all handled implicitly.</p>
]]></summary>
</entry>
<entry>
    <title>ARMing Haskell</title>
    <link href="http://haskellembedded.github.io//posts/2015-12-15-arm.html" />
    <id>http://haskellembedded.github.io//posts/2015-12-15-arm.html</id>
    <published>2015-12-15T00:00:00Z</published>
    <updated>2015-12-15T00:00:00Z</updated>
    <summary type="html"><![CDATA[<div class="info">
    Posted on December 15, 2015
    
        by Calvin Beck
    
</div>
<div class="info">
    
    Tags: <a href="/tags/Haskell.html">Haskell</a>, <a href="/tags/ARM.html">ARM</a>, <a href="/tags/QEMU.html">QEMU</a>, <a href="/tags/Raspberry%20Pi.html">Raspberry Pi</a>
    
</div>

<p>I have spent the last few weeks desperately trying to get Haskell working on a Raspberry Pi 2 with Raspbian (Jessie). I have had… <a href="https://twitter.com/Chobbez/status/672209942686752768">some</a> <a href="https://twitter.com/Chobbez/status/672178036612005889">problems</a></p>
<p><em>Edit 2015-12-16: Thanks to the help of the wonderful Haskell community, my issues have been resolved. Make sure you have LLVM 3.5.2, or you may encounter the same problems that I did!</em></p>
<p><em>Edit 2015-12-17: Thanks to slyfox on #ghc on freenode we now have a sort of working frankenstein’s cross-compiler!</em></p>
<h1 id="cross-compilation">Cross-Compilation</h1>
<p>There are a few useful guides available for setting up cross-compilation:</p>
<ul>
<li><a href="https://github.com/ku-fpg/raspberry-pi/wiki/GHC-Cross-Compiler-for-Raspberry-Pi" class="uri">https://github.com/ku-fpg/raspberry-pi/wiki/GHC-Cross-Compiler-for-Raspberry-Pi</a></li>
<li><a href="https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/RaspberryPi" class="uri">https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/RaspberryPi</a></li>
</ul>
<p>It’s fairly easy to get a cross-compiler up and running, as long as you carefully follow instructions. This is all well and good, but there are still issues with Template Haskell. Template Haskell essentially requires that you can run code built from the compiler on the system running the compiler. Roughly speaking Template Haskell uses Haskell code to generate more Haskell during compilation. Since you can’t run an ARM executable on x86 / x86_64, you can’t compile anything that uses Template Haskell with the cross-compiler as of yet. More on this later, but <a href="https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/CrossCompilation">here’s some more information</a>.</p>
<p>In addition to the lack of Template Haskell, there are <a href="https://ghc.haskell.org/trac/ghc/ticket/9689">problems with certain FFI libraries, like zlib</a>. So, while my 7.10.2 can produce an acceptable ARM “Hello, World!”, I can’t compile my more complicated project. Thus I am forced to look for another option!</p>
<h2 id="cross-compilation-with-qemu">Cross-Compilation with QEMU</h2>
<p>So, let’s say you have <a href="#qemu-user-emulation">read the section on using a QEMU ARM user chroot with binfmt_misc to run ARM code on your non-ARM machine</a>. If we can 1) have ARM libraries on our machine, and 2) run ARM executables, then it seems like we should be able to run the <a href="#cross-compilation">cross compiler from above</a>, and then when we execute ARM code, drop into <code>qemu-arm</code>.</p>
<p>The only thing we should have to do is tell <code>qemu-arm</code> to load dynamic libraries from somewhere else, so that we can load ARM libraries, and not x86 / x86_64 libraries. Turns out <a href="http://wiki.qemu.org/download/qemu-doc.html#Command-line-options">there is an option for this</a>:</p>
<pre><code>$ qemu-arm

...
-L path       QEMU_LD_PREFIX    set the elf interpreter prefix to &#39;path&#39;
...
</code></pre>
<p>Yeah, that will work. Using the cross compiler above, and the ARM Gentoo environment set up below I was able to compile my program just fine using:</p>
<pre><code>export QEMU_LD_PREFIX=$HOME/arm-chroot

cabal sandbox init
cabal --with-ghc=arm-unknown-linux-gnueabihf-ghc --with-ghc-pkg=arm-unknown-linux-gnueabihf-ghc-pkg --with-ld=arm-linux-gnueabihf-ld --with-strip=arm-linux-gnueabihf-strip install</code></pre>
<p>Which is <em>AWESOME</em> because the native compiler is <em>so</em> much faster than the one emulated with QEMU. This is not without caveats. For instance when compiling with multiple jobs it took forever and then ran out of memory, setting <code>-j1</code> with <code>cabal install</code> fixed this particular issue.</p>
<p>A bigger problem is that there appears to be issues with the C FFI. I’m not yet sure how this works, but when I try to run an ARM binary which uses JuicyPixels to write a PNG (which relies upon zlib), I get this error:</p>
<pre><code>user error (Codec.Compression.Zlib: incompatible zlib version)</code></pre>
<p>but otherwise this works surprisingly well.</p>
<p>After a bit of digging I have found that this error comes from <a href="https://github.com/haskell/zlib/blob/85f2d95396ff884ff22af62e5e0d6029c789db63/Codec/Compression/Zlib/Stream.hsc#L549">here</a>, and since we have got an actual failure, it’s probably from calling the <code>failIfError</code> function. That means our issue is from <a href="https://github.com/haskell/zlib/blob/85f2d95396ff884ff22af62e5e0d6029c789db63/Codec/Compression/Zlib/Stream.hsc#L953">here</a>.</p>
<p>I have checked the zlib version on all of my environments, and the version is 1.2.8 everywhere. So the <em>version</em> isn’t the issue. In fact if we look at the <a href="https://github.com/madler/zlib/blob/e8fee0ea7bf62e595bd5518b7b22e3e16397278c/deflate.c#L233">code</a> for <code>c_deflateInit2_</code> we’ll notice that it’s only checking the first version number anyway:</p>
<div class="sourceCode"><pre class="sourceCode c"><code class="sourceCode c"><span class="kw">if</span> (version == Z_NULL || version[<span class="dv">0</span>] != my_version[<span class="dv">0</span>] ||
    stream_size != <span class="kw">sizeof</span>(z_stream)) {
    <span class="kw">return</span> Z_VERSION_ERROR;
}</code></pre></div>
<p>But there’s our problem. The size of our <code>z_stream</code> must differ somehow. If we check the <a href="https://github.com/haskell/zlib/blob/85f2d95396ff884ff22af62e5e0d6029c789db63/Codec/Compression/Zlib/Stream.hsc#L1039">Haskell Zlib library</a> we notice the following:</p>
<div class="sourceCode"><pre class="sourceCode haskell"><code class="sourceCode haskell">
<span class="ot">c_deflateInit2 ::</span> <span class="dt">StreamState</span>
               <span class="ot">-&gt;</span> <span class="dt">CInt</span> <span class="ot">-&gt;</span> <span class="dt">CInt</span> <span class="ot">-&gt;</span> <span class="dt">CInt</span> <span class="ot">-&gt;</span> <span class="dt">CInt</span> <span class="ot">-&gt;</span> <span class="dt">CInt</span> <span class="ot">-&gt;</span> <span class="dt">IO</span> <span class="dt">CInt</span>
c_deflateInit2 z a b c d e <span class="fu">=</span>
  withCAString <span class="fu">#</span>{const_str <span class="dt">ZLIB_VERSION</span>} <span class="fu">$</span> \versionStr <span class="ot">-&gt;</span>
    c_deflateInit2_ z a b c d e versionStr (<span class="fu">#</span>{const sizeof(z_stream)}<span class="ot"> ::</span> <span class="dt">CInt</span>)</code></pre></div>
<p>I’m on an x86_64 machine, so it’s pretty much guaranteed that the <code>z_stream</code> structure on my machine differs in size to the one on the 32 bit ARM machines. This should be processed by <code>hsc2hs</code>, so that’s where the issue is. I need to be running an <code>hsc2hs</code> that targets ARM, not my native one. I had an <code>arm-unknown-linux-gnueabihf-hsc2hs</code>, but it seemed to loop forever, as did the <code>hsc2hs</code> from my chroot (but not when running it in my chroot). So that’s no good either.</p>
<p><code>hsc2hs</code> by default generates a C program, which when run spits out the appropriate Haskell file. So we just need to get <code>hsc2hs</code> to generate ARM code. I can pass <code>hsc2hs</code> options with cabal, which means I can tell the native <code>hsc2hs</code> to compile using a cross compiler and linker.</p>
<pre><code>--hsc2hs-option=&quot;-c arm-linux-gnueabihf-gcc -l arm-linux-gnueabihf-ld&quot;</code></pre>
<p>Unfortunately that still doesn’t work! We’re on the right track, but not quite there yet. The problem now is that <code>arm-linux-gnueabihf-ld</code> doesn’t actually know where to look for libc (we need the ARM one). We can use the <code>--sysroot</code> option to make it use the ARM chroot. I actually had to change to <code>gcc</code> for the linker as well, because it handles linking with the C runtime much more nicely.</p>
<pre><code>--hsc2hs-option=&quot;-c arm-linux-gnueabihf-gcc -l arm-linux-gnueabihf-gcc -C &quot;--sysroot=$HOME/arm-chroot/&quot; -L &quot;--sysroot=$HOME/arm-chroot/&quot;&quot;</code></pre>
<p>This loops forever with <code>qemu-arm</code>, however the executable when run on the Raspberry Pi 2 works perfectly fine. This seems to be a QEMU bug.</p>
<p>Note that <code>hsc2hs</code> actually has cross compilation options (<code>-x</code>). When these are used instead of creating a C program <code>hsc2hs</code> uses <a href="https://github.com/ghc/hsc2hs/blob/master/CrossCodegen.hs#L6">tricks to figure out what’s going on with the target</a>. Unfortunately this doesn’t handle <code>const_str</code>, which we <a href="https://github.com/haskell/zlib/blob/master/Codec/Compression/Zlib/Stream.hsc#L1017">need</a>, so this won’t work for us.</p>
<h1 id="running-on-the-raspberry-pi">Running on the Raspberry Pi</h1>
<p>Raspbian has an old version of GHC in its repos, GHC 7.6.3, which works with simple pieces of code. For instance “Hello, World!” might compile and run perfectly well. However, my small image processing program encountered nasty, randomly changing, run time errors, and segmentation faults. Sometimes, if the stars aligned, the program would run to completion producing correct results, other times it seemed to loop forever. This is not good. One such error that I received was:</p>
<pre><code>allocGroup: free list corrupted</code></pre>
<p>which, if we look at the <a href="https://github.com/ghc/ghc/blob/ghc-7.6/rts/sm/BlockAlloc.c#L383">source code</a>, should definitely never happen. Everything that is happening here <em>SCREAMS</em> that memory is getting stomped on somewhere. This is not something which is going to be easy to debug as the problem could quite literally be anywhere in the code, or in fact in a different library entirely. Not good.</p>
<h2 id="what-about-a-newer-compiler">What about a newer compiler?</h2>
<p>So, the Raspbian compiler is horribly ill, and we have to try something different. I attempted to compile GHC from scratch, but this is an effort which takes a very long time on a Raspberry Pi, and after fixing build errors I still had issues. Fortunately since GHC 7.10.2 there are <a href="https://www.haskell.org/ghc/download_ghc_7_10_3#linux_armv7">binaries for ARMv7</a>. Additionally, I found a guide which suggested that there were no issues on a <a href="https://www.scaleway.com/">Scaleway server</a>:</p>
<p><a href="http://statusfailed.com/blog/2015/11/29/haskell-and-servant-on-scaleway-arm-servers.html" class="uri">http://statusfailed.com/blog/2015/11/29/haskell-and-servant-on-scaleway-arm-servers.html</a></p>
<p>Following these suggestions I was able to install 7.10.2, and 7.10.3 on the Raspberry Pi. Unfortunately these compilers have given me quite a bit of grief, as I am incapable of producing a working “Hello, World!” with them (despite the 7.10.2 cross-compiler working just fine for this):</p>
<p><a href="https://ghc.haskell.org/trac/ghc/ticket/11190" class="uri">https://ghc.haskell.org/trac/ghc/ticket/11190</a></p>
<p>Cabal-install hangs, and this too is a dead end for me.</p>
<p>Or it would be if it wasn’t for Ben Gamari, who replied to <a href="https://ghc.haskell.org/trac/ghc/ticket/11190">my issue in the GHC trac</a>. As it turns out the LLVM version available with Raspbian is 3.5.0. This version has an issue on ARM which breaks the binaries that GHC spits out. Upgrading to 3.5.2 should fix this problem!</p>
<h1 id="qemu">QEMU</h1>
<p>QEMU is an attractive option. We can emulate an ARM machine on a fast x86 processor, which may be faster. At any rate it’s nice to be able to compile, and even test ARM binaries, on your development system, without having to dedicate a Raspberry Pi as a build bot. I only have access to one Raspberry Pi 2 at the moment, and it’s occupied with work stuff. QEMU would make things easier, and hell, it might even work.</p>
<h1 id="qemu-system-emulation">QEMU System Emulation</h1>
<p>This is also a dead end. I am horrendously bad at getting QEMU machine emulation to work. I got it working with Raspbian Wheezy, but it wouldn’t work with Jessie for some reason. I spent a lot of time trying to get an ARM machine emulated, but all of the guides are filled with archaic options, vmlinuz kernels, and magic. Ultimately nothing worked, and this was a huge waste of time that would be slow anyway. <a href="https://twitter.com/Chobbez/status/675573353898315776">Here be dragons. All who dare enter should beware.</a></p>
<h1 id="qemu-user-emulation">QEMU User Emulation</h1>
<p>Initially I thought that system emulation would be much easier to set up and get working than user emulation. I was wrong. QEMU system emulation will emulate an entire ARM machine, whereas user emulation essentially just lets you run ARM binaries on a different machine as a regular user process. User emulation translates all of the instructions, and sys-calls, but uses the same kernel and filesystem as before. This is cool. What’s especially cool is that Linux has a feature called <a href="https://en.wikipedia.org/wiki/Binfmt_misc">binfmt_misc</a>, which lets you run arbitrary executable formats. So, we can actually get our normal x86 / x86_64 Linux system to transparently execute ARM binaries. Essentially we use <code>binfmt_misc</code> to tell the kernel to run ARM executables with the <code>qemu-arm</code> program. This is pretty awesome.</p>
<p>There are some interesting possibilities with this, as we might be able to leverage this to make the cross compiler work with Template Haskell. After turning this on, instead of getting <code>Exec format error</code>’s with Template Haskell and the cross compiler, it was instead complaining about missing ARM libraries. Cool. This is something that might be worth exploring at a later date, but I don’t really want to mess with mixing ARM and x86_64 binaries on my computer. That’s bad mojo. Worth messing with in the future, because it has the possibility of being faster, but it’s probably filled with caveats and it seems a bit scary.</p>
<p>So, what’s the next best option? How about a <code>chroot</code>!? Yeah. That’ll work. So, here’s the plan! In order to make sure we get all of the ARM libraries we need, we’ll just install an ARM Linux distribution in a <code>chroot</code>, and use <code>binfmt_misc</code> and <code>qemu-arm</code> to transparently execute any ARM binaries in the <code>chroot</code>. I got some information for how to set up <code>binfmt_misc</code> on Gentoo here:</p>
<p><a href="https://wiki.gentoo.org/wiki/Crossdev_qemu-static-user-chroot" class="uri">https://wiki.gentoo.org/wiki/Crossdev_qemu-static-user-chroot</a></p>
<p>This guide seems to be doing more complicated stuff with LXC, which I don’t really understand, so I have deviated and just went with a simple <code>chroot</code> Linux install. I’m going to install Gentoo, because it’s what I know, and I find that it doesn’t hide problems from me, and it will let me try different versions of LLVM and other libraries which might be causing problems quite easily.</p>
<p>We need access to <code>qemu-arm</code> in our chroot if we want to use it to run ARM binaries. The simple way to do this is to statically link <code>qemu-arm</code>, which will allow us to copy the <code>qemu-arm</code> binary into the chroot, and not have to worry about copying any dynamically loaded libraries as well. In order to do this on Gentoo we simply add the <a href="https://wiki.gentoo.org/wiki/QEMU#Miscellaneous">static-user</a> use flag to the <code>qemu</code> package, your distro may vary. We can check that we are good to go with <code>ldd</code>:</p>
<pre><code>$ ldd `which qemu-arm`
    not a dynamic executable</code></pre>
<p>Looking good! Make sure you set up <code>binfmt_misc</code> first, but now we can make our chroot.</p>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash"><span class="co"># First we make the directory for our chroot:</span>
<span class="kw">mkdir</span> ~/arm-chroot
<span class="kw">cd</span> ~/arm-chroot

<span class="co"># Get the Gentoo stage3</span>
<span class="kw">wget</span> http://distfiles.gentoo.org/releases/arm/autobuilds/20151116/stage3-armv7a_hardfp-20151116.tar.bz2
<span class="kw">tar</span> -xjf stage3-armv7a_hardfp-20151116.tar.bz2

<span class="co"># Copy qemu-arm into the chroot, so we can use ARM binaries when chrooted.</span>
<span class="kw">cp</span> <span class="kw">`which</span> qemu-arm<span class="kw">`</span> ~/arm-chroot/usr/bin/qemu-arm

<span class="co"># Then we do the magical mounting: https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Base#Mounting_the_necessary_filesystems</span>
<span class="kw">mount</span> -t proc proc proc
<span class="kw">mount</span> --rbind /sys /sys
<span class="kw">mount</span> --make-rslave sys
<span class="kw">mount</span> --rbind /dev dev
<span class="kw">mount</span> --make-rslave dev

<span class="co"># And now we chroot!</span>
<span class="kw">sudo</span> -s
<span class="kw">chroot</span> . /bin/bash

<span class="co"># At this point you can pretty much just follow the Gentoo Handbook https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Base#Configuring_Portage</span>
<span class="kw">emerge-webrsync</span>
<span class="kw">emerge</span> --sync

<span class="co"># ...</span>
<span class="co"># Configure the system how you want...</span>
<span class="co"># ...</span>

<span class="co"># Note that we need LLVM version 3.5.2, which you may need to add to package.accept_keywords</span>
<span class="kw">echo</span> <span class="st">&quot;=sys-devel/llvm-3.5.2 **&quot;</span> <span class="kw">&gt;&gt;</span> /etc/portage/package.accept_keywords

<span class="co"># Now we can install whatever we need for GHC, like LLVM</span>
<span class="kw">emerge</span> -a llvm binutils zlib

<span class="co"># Grab and install GHC ARMv7 binaries...</span>
<span class="kw">wget</span> http://downloads.haskell.org/~ghc/7.10.3/ghc-7.10.3-armv7-deb8-linux.tar.bz2
<span class="kw">tar</span> -xjf ghc-7.10.3-armv7-deb8-linux.tar.bz2
<span class="kw">cd</span> ghc-7.10.3
<span class="kw">./configure</span>
<span class="kw">make</span> install</code></pre></div>
<p>The first time I did this I still encountered <a href="https://ghc.haskell.org/trac/ghc/ticket/11190">the same problem as on the Raspberry Pi</a> with the <code>qemu-arm</code> GHC, however after installing LLVM 3.5.2 I have had no more problems. You can even install Cabal and cabal-install pretty easily. Just download the tarballs from <a href="https://www.haskell.org/cabal/download.html">here</a>, and follow the instructions in the README files. It really is that simple. Make sure to move the <code>cabal</code> executable somewhere in your path, and run <code>cabal update</code>, and then you should be set!</p>
<p>I have since been able to compile my image processing program, and stuff it on the Raspberry Pi 2. No problems at all. QEMU is a bit slow, but it’s nice to not have to worry about compiling on the Pi, and now I can do everything from my development machine. This is quite a bit slower than the Pi 2 itself, but works very well.</p>
<h1 id="current-status">Current Status</h1>
<p>After much pain and suffering I have a way to do Haskell development on ARM! Others have had success come quite a bit more easiely. I have had a conversation with <a href="https://twitter.com/dhess/status/675142967158571009">somebody on Twitter whom found success</a> with <a href="https://www.scaleway.com/">Scaleway servers</a> and a <a href="http://beagleboard.org/black">Beaglebone Black</a>. The difference here seems to be that whatever image they were using on Scaleway had LLVM 3.5.2, and not the broken LLVM 3.5.0.</p>
<p>Good luck to anybody trying to do the same! Ask questions, or boast about your success in the comments.</p>
<h1 id="the-future">The Future</h1>
<p>So with that done and working, here’s some stuff that could greatly improve the Haskell on ARM development experience:</p>
<ul>
<li>It would be really nice to have binary packages for ARM, which is something that <a href="https://nixos.org/nix/">nix</a> could provide. <a href="https://twitter.com/a_cowley/status/677212533854466048">Anthony Cowley thinks this is a good idea too</a>.</li>
<li>It might be possible to build a GHC cross compiler, and have it transparently execute ARM binaries with binfmt_misc and <code>qemu-arm</code> for Template Haskell.
<ul>
<li>There would still be caveats.</li>
<li>This would be much faster than emulating the compiler, though.</li>
<li>Perhaps there is a way to get <code>qemu-arm</code> to use a different path for dynamic libraries, so you can have ARM libraries somewhere nice and separate.</li>
</ul></li>
<li><a href="https://wiki.gentoo.org/wiki/Project:Prefix">Gentoo Prefix</a> might be a bit nicer than a chroot.</li>
<li><a href="https://github.com/commercialhaskell/stack/issues/1332">Stack currently has issues on ARM</a>. It would be nice to get Stack working somehow.</li>
</ul>
]]></summary>
</entry>
<entry>
    <title>Embedding Haskell: Compilers, and compiling compilers</title>
    <link href="http://haskellembedded.github.io//posts/2015-10-09-compiler-compilers.html" />
    <id>http://haskellembedded.github.io//posts/2015-10-09-compiler-compilers.html</id>
    <published>2015-10-09T00:00:00Z</published>
    <updated>2015-10-09T00:00:00Z</updated>
    <summary type="html"><![CDATA[<div class="info">
    Posted on October  9, 2015
    
        by Chris Hodapp
    
</div>
<div class="info">
    
    Tags: <a href="/tags/haskell.html">haskell</a>, <a href="/tags/ramblings.html">ramblings</a>
    
</div>

<p>My <a href="../posts/2015-06-09-atom-cincyfp-slides.html">last post</a> mentioned that some things need some better explanation, because I’m always trying to re-explain and clarify.</p>
<p>This blog is devoted to the use of Haskell with embedded systems. What does that even mean? We see a couple broad categories (which the slides on the last page, as well as our <a href="../pages/links.html">Links</a> page, mirror):</p>
<ol style="list-style-type: decimal">
<li><em>Full Compilation:</em> Compiling Haskell code to an embedded target.</li>
<li><em>Limited Compilation:</em> Compiling some limited subset of Haskell code to an embedded target.</li>
<li><em>Hosted EDSL &amp; Compiler:</em> Hosting, in Haskell, an EDSL and a compiler to an embedded target.</li>
</ol>
<p>As far as I know, I made these categories up. If anyone happens to know a more established classification, better names, or an example of who wrote about it first, please tell me.</p>
<p>This might look like a lopsided, arbitrary grouping; it sort of is. The commonality is that in all cases one uses Haskell to express something (a program, a circuit, specifications, call it what you will) for an embedded target. More on that follows.</p>
<p>I exclude things like Cryptol and Idris from this because - while implemented in Haskell and usable for embedded platforms - they are different languages unto themselves. I might arbitrarily drop that distinction in the future if I feel like it…</p>
<h1 id="full-compilation">Full Compilation</h1>
<p>This is what normally comes to mind when people hear about using Haskell with embedded systems - compiling the Haskell code to run directly on an embedded target, bringing along the normal runtime with it (plus whatever required bootstrapping and support). The <a href="../pages/links.html#compiling-to-embedded-targets">Compiling to Embedded Targets</a> section of the Links page is concerned particularly with this.</p>
<p>However, this actually appears to be pretty rare. The nature of the Haskell language brings some formidable challenges. Particularly, one must make the Haskell runtime fit on the target and make the garbage collection and lazy evaluation behave in predictable and sane ways.</p>
<p><a href="https://github.com/ajhc/ajhc">Ajhc</a>, a <a href="http://repetae.net/computer/jhc/">JHC</a>-derived compiler from Kiwamu Okabe of METASEPI, is the only example of this I found - it could compile and execute on ARM Cortex-M3/M4. Kiwamu has written a lot on his experiences with making Haskell run in this footprint. His subsequent switch to the ATS language may be a hint.</p>
<p><a href="https://github.com/galoisinc/halvm">HaLVM</a> from Galois might arguably fit in this category.</p>
<h1 id="limited-compilation">Limited Compilation</h1>
<p>This uses an existing compiler for certain stages (such as the parsing and type-checking), but a custom back-end to actually produce code, often with a lot of static analysis. This may adapt or disallow certain constructs (for instance, floating-point, recursive functions, recursive datatypes: <a href="http://hackage.haskell.org/package/clash-prelude-0.10/docs/CLaSH-Tutorial.html#unsupported">CλaSH Unsupported Haskell features</a>).</p>
<p>GHC accomodates this by allowing developers to invoke GHC functionality, from Haskell, <a href="https://wiki.haskell.org/GHC/As_a_library">as a library</a>.</p>
<p>The <a href="../pages/links.html#compiling-for-fpgaasic">Compiling for FPGA/ASIC</a> section of the Links page has a few examples of this.</p>
<h1 id="hosted-edsl-compiler">Hosted EDSL &amp; Compiler</h1>
<p>The <a href="../pages/links.html#code-generation-edsls">Code Generation EDSLs</a> and <a href="../pages/links.html#circuit-design-edsls">Circuit Design EDSLs</a> sections of the Links page cover the copious examples of this. Atom, the topic of a few of my <a href="../posts/2015-02-17-atom-examples.lhs">prior posts</a>, is in this category.</p>
<p>This category is the one I am most often having to explain. It typically uses an EDSL (embedded domain-specific language) inside of Haskell to direct the process of code generation to a lower-level representation. This is otherwise called: <em>compiling</em>.</p>
<p>To emphasize: The code that runs on the target is <em>entirely decoupled from the Haskell runtime</em>. The Haskell compiler here isn’t compiling anything for the target - it’s compiling another compiler and the input to that compiler. That input happens to be specifications of what <em>will</em> run on the target.</p>
<p>This is a limitation of one sort:</p>
<ul>
<li>Basically all notions of ‘runtime’ on the embedded target must be handled separately. (Ivory works with this still, for instance with the <a href="https://hackage.haskell.org/package/base/docs/Prelude.html#t:Num">Num</a> typeclass, in some surprising ways. More on that will follow in a future post!)</li>
<li>This adds the confusion and complication of another stage (possibly multiple stages) to the process of bringing code/specifications to the embedded target. This is why I use <a href="http://shakebuild.com/">Shake</a>.</li>
</ul>
<p>It’s also a benefit of another sort:</p>
<ul>
<li>Any Haskell environment compatible with the libraries in question should produce the same results (as far as the embedded target cares). Its runtime does not matter, nor whether the environment has any knowledge of the architecture of the embedded target.</li>
<li>That stage separation also adds a nice opportunity for static analysis and optimization. For instance, Copilot makes use of this to add an interpreter/simulator, SBV uses it to prove or disprove given properties about the code, and Atom uses it to verify some timing constraints.</li>
</ul>
<p>I said in the <a href="../posts/2015-06-09-atom-cincyfp-slides.html">last post</a> that in this category Haskell takes on the role of a metaprogramming or template language. While this may be true, I sort of ignored that it’s less relevant, because it’s the same in all the other categories.</p>
<h1 id="commonality">Commonality</h1>
<p>Lumping together these categories might seem like a stretch, especially considering that the last category involves extra stages and a shift in how one thinks about the software.</p>
<p>Ponder the following, though:</p>
<ul>
<li>A “normal” Haskell program interacts through what is sequenced in the famous <a href="https://hackage.haskell.org/package/base/docs/System-IO.html">IO</a> monad (particularly, the value called <code>main</code>).</li>
<li>An Atom specification interacts through what is sequenced in the <a href="https://hackage.haskell.org/package/atom-1.0.13/docs/Language-Atom.html#t:Atom">Atom</a> monad (particularly, whichever values one passes to the <a href="https://hackage.haskell.org/package/atom-1.0.13/docs/Language-Atom.html#g:2">Atom compiler</a>).</li>
<li>An Ivory program interacts through what is sequenced in the <a href="https://hackage.haskell.org/package/ivory-0.1.0.0/docs/Ivory-Language.html#t:Ivory">Ivory eff</a> and <a href="https://hackage.haskell.org/package/ivory-0.1.0.0/docs/Ivory-Language.html#t:Module">Module</a> monads (particularly, whichever values one passes to the <a href="https://hackage.haskell.org/package/ivory-backend-c-0.1.0.1/docs/Ivory-Compile-C.html">Ivory compiler</a>).</li>
<li>A <a href="http://www.clash-lang.org/">CλaSH</a> description interacts through the <a href="https://hackage.haskell.org/package/clash-prelude-0.10/docs/CLaSH-Signal.html#t:Signal">Signal</a> applicative (particularly, the value called <a href="https://hackage.haskell.org/package/clash-prelude-0.10/docs/CLaSH-Annotations-TopEntity.html">topEntity</a>).</li>
</ul>
<p>Is a trend clear? (No, it’s not monads. <a href="https://hackage.haskell.org/package/clash-prelude-0.10/docs/CLaSH-Signal.html#t:Signal">Signal</a> is only applicative, and I suspect Lava behaves similarly.)</p>
<p>That list spans our three categories. In each of them, one builds up a program (in a very broad sense) simply by building up a value in Haskell. Beyond that, the only real differences are,</p>
<ul>
<li>the type of that value,</li>
<li>what system handles it (the Haskell compiler and runtime, some other compiler and maybe runtime, or a combination thereof),</li>
<li>and the eventual output (native binary, LLVM bitcode, C code, VHDL code, assembly language, input to a model checker, etc.).</li>
</ul>
<p>Ignoring the vague nature of the term, “declarative,” this relates pretty directly to the <em>declarative</em> nature of Haskell programs.</p>
<p>Seen from this perspective, one is still <em>compiling Haskell</em> to run on some embedded target. The compilation just might continue outside of the system’s Haskell compiler, and the running might not involve its runtime.</p>
]]></summary>
</entry>
<entry>
    <title>Presentation at CincyFP</title>
    <link href="http://haskellembedded.github.io//posts/2015-06-09-atom-cincyfp-slides.html" />
    <id>http://haskellembedded.github.io//posts/2015-06-09-atom-cincyfp-slides.html</id>
    <published>2015-06-09T00:00:00Z</published>
    <updated>2015-06-09T00:00:00Z</updated>
    <summary type="html"><![CDATA[<div class="info">
    Posted on June  9, 2015
    
        by Chris Hodapp
    
</div>
<div class="info">
    
    Tags: <a href="/tags/haskell.html">haskell</a>, <a href="/tags/atom.html">atom</a>
    
</div>

<p><em>Later note: This post is back-dated because while I’d put the slides in the repository already, I had not actually linked to them or said anything about them.</em></p>
<p>The fine people at <a href="https://cincyfp.wordpress.com/">CincyFP</a>, the local (to me) functional programming group, requested that I give a presentation on some of the embedded Haskell stuff I was doing, and I did this at the <a href="https://cincyfp.wordpress.com/2015/06/04/june-meeting-4/">June meeting</a> this year.</p>
<p>Slides from the presentation are up <a href="../slides/20150609_CincyFP/Slides.html">here</a> (and <a href="../slides/20150609_CincyFP/Slides.md">Markdown source</a>; or look <a href="https://github.com/HaskellEmbedded/HaskellEmbedded.github.io/tree/master/slides/20150609_CincyFP">in the repo</a>). CincyFP seems to have mostly Clojure aficionados, so in terms of Haskell-heavy details I kept this presentation as a “part 1” to some possible future presentations. Actually, not even that, since the title was “EDSLs &amp; Metaprogramming” and I only scratched the surface of that.</p>
<p>I start walking through an Atom example at slide 9. While I live-coded this at the presentation, I have no video or audio. However, I put my example code online <a href="../slides/20150609_CincyFP/Example.hs">here</a> (and <a href="../slides/20150609_CincyFP/StringEmbed.hs">StringEmbed.hs</a>).</p>
<p>The slides are fairly terse, and I intend to explain bits of them in the next few posts. Particularly, around slide 2, I try to explain a few different broad ways Haskell might be used on embedded systems, and this refers to similar categories as on the <a href="../pages/links.html">Links</a> page. I find myself having to explain these distinctions quite often, so I am going to try to explain this better in posts to come.</p>
<p>Slide 6 is a little bit misleading. The next post tries to patch this up a bit.</p>
]]></summary>
</entry>
<entry>
    <title>How I Got Here (i.e. to using Haskell for embedded code)</title>
    <link href="http://haskellembedded.github.io//posts/2015-02-06-how-i-got-here.html" />
    <id>http://haskellembedded.github.io//posts/2015-02-06-how-i-got-here.html</id>
    <published>2015-02-06T00:00:00Z</published>
    <updated>2015-02-06T00:00:00Z</updated>
    <summary type="html"><![CDATA[<div class="info">
    Posted on February  6, 2015
    
        by Chris Hodapp
    
</div>
<div class="info">
    
    Tags: <a href="/tags/haskell.html">haskell</a>, <a href="/tags/atom.html">atom</a>, <a href="/tags/ivory.html">ivory</a>, <a href="/tags/copilot.html">copilot</a>, <a href="/tags/idris.html">idris</a>, <a href="/tags/sbv.html">sbv</a>, <a href="/tags/python.html">python</a>
    
</div>

<p>One work project since near the begining of 2014 has been the design of an embedded system for which power usage and stability are two primary concerns. It’s not controlling an airplane or a nuclear plant - malfunctions will not be catastrophic - but stability still matters.</p>
<p>My boss had decided that for various reasons too numerous and extensive to list here, we needed to have support for Bluetooth Low Energy. Our system at the time ran on the venerable TI MSP430 and I started looking at ways to support BLE from this. All of TI’s BLE offerings at the time ran on the 8051 and appeared to require proprietary toolchains. After surveying what else was available, we had eventually decided on the <a href="https://www.nordicsemi.com/eng/Products/Bluetooth-Smart-Bluetooth-low-energy/nRF51822">Nordic nRF51822</a>, a microcontroller based on an ARM Cortex-M0, supported by standard GCC, and providing all the necessary peripherals for BLE - and simply eliminated the MSP430 altogether.</p>
<p>It quickly evolved to a distributed system - some parts were sensors or actuators that were BLE peripherals, another part ran constantly as a BLE central, another part ended up as a more powerful, Internet-connected Linux machine that ran only sparingly.</p>
<p>A few patterns emerged:</p>
<ul>
<li>All parts needed to be speaking the same protocol, which necessitated keeping various structures and constants in sync at the software each part ran.</li>
<li>The nRF51822 board acting as the BLE central was responsible for juggling a lot of concurrent operations, because nearly all API calls were async.</li>
<li>The nRF51822 boards ran different software, but still shared various bits of code and constants.</li>
</ul>
<p>The two languages here were C for the nRF51822, and Python for the Linux machine. I tried to build abstractions that could sensibly represent these patterns, but were sufficiently lightweight in C. One was in Python to keep data types in sync; for Linux, it handled it at runtime in Python, and for the microcontroller, it generated C code with the requisite <code>memcpy</code> calls and proper offsets and lengths. The C code also had various FSMs (some explicit, some not) for managing the concurrent operations. At later examination they were nearly all ad-hoc coroutines that were ‘hiding’ a very simple concurrent description in their details.</p>
<p>Seeing that I had ad-hoc coroutines hidden everywhere in my code, I tried to integrate in some lightweight RTOSes that provided some form of cooperative thread or coroutine. I looked at <a href="http://atomthreads.com/">Atomthreads</a>, <a href="http://www.riot-os.org/">RIOT OS</a>, <a href="http://www.contiki-os.org/">Contiki</a>, <a href="http://www.freertos.org/">FreeRTOS</a>, and <a href="http://www.chibios.org/dokuwiki/doku.php">ChibiOS/RT</a>, but the issue that stopped me in all cases was that they appeared they would interfere with Nordic’s firmware, which I required for the Bluetooth Low Energy functionality. Finally, I discovered <a href="http://www.cocoos.net/intro.html">cocoOS</a> which provided just the cooperative threads and basic messaging primitives, and due to its simplicity required only minimal implementation and nothing very low-level. I quickly ported it and it was working seemingly wonderfully.</p>
<p>I had recently read Joe Armstrong’s famous <a href="http://www.erlang.org/download/armstrong_thesis_2003.pdf">thesis on Erlang</a>, and found it very cogent and pragmatic, but still accessible. He defines on page 19 <em>Concurrency Oriented Programming</em>. I had a thought that perhaps cocoOS gave me the primitives I needed to let me apply the methodology that Armstrong described, and express the system as a design around concurrent processes. With clean system design in hand and hopeful visions of elegance in my head, I set about working on this - until reaching an abrupt ending when I ran out of memory with around 2 cooperative tasks and almost nothing else compiled in.</p>
<p>Around the same point, I had also extended my Python code generation to generate code for some more complex protocols and refactored it to make use of a very handy library, <a href="https://www.python.org/about/success/cog/">Cog</a>.</p>
<p>At some point I saw that these all were sides of the same ugly dilemma (trilemma?):</p>
<ul>
<li>I could write larger amounts of code that would ‘hide’ the simpler representations inside.</li>
<li>I could attempt to streamline the above with code generation via macros.</li>
<li>I could build or use runtime abstractions that helped me approximate those simpler representations, but incurred too much overhead.</li>
</ul>
<p><code>#define</code> and <code>#include</code> existed, but they addressed only the smallest of cases, and I felt like C++ was no cure either. Cog helped streamline the code generation, but my code generation still felt like primitive macros. My view of code generation in general was grim too, from seeing its use at other jobs. In particular:</p>
<ol style="list-style-type: decimal">
<li>The need for automatically-generated code probably means that the underlying language lacks the right abstractions (or that you do not understand them).</li>
<li>Either you incur the complexity of implementing the code generation in that same (lacking or ill-understood) language, or you incur the mental overhead of having to coordinate with another language at the same time - possibly a third language, if you handle specifications in a separate language.</li>
<li>You don’t fix a problem in automatically-generated code - you fix the code generator or the specifications, which is nearly always more difficult and has more consequences.</li>
<li>The code generator’s scope tends to expand anywhere generated code touches manually-written code.</li>
</ol>
<p>Of course, avoiding code generation completely is senseless. Compilers are code generators. They run on CPUs probably created (gate-wise) by some form of code generator. The problem is not code generation, but code generation as ad-hoc automation that buries complexity underneath a layer of fresh complexity.</p>
<p>Luckily, plenty of people (besides compiler writers) seem to get code generation right. After my failure to make any progress with cocoOS, I had started reading about <a href="https://github.com/tomahawkins/atom">Atom</a>, an EDSL in Haskell which generates C code for hard realtime embedded software. It looked very promising and solid, but too narrow in scope for what I needed. Shortly after, I found <a href="https://github.com/leepike/Copilot">Copilot</a>, another Haskell EDSL for some similar purposes which generates C either via Atom or <a href="https://hackage.haskell.org/package/sbv">SBV</a> (also a Haskell EDSL). <a href="http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20120014570.pdf">Experience Report: a Do-It-Yourself High-Assurance Compiler</a> from NASA focuses on Copilot, but identifies some general benefits from the paradigm; it is a short, informative read that I highly recommend.</p>
<p>I stumbled into <a href="http://ivorylang.org/">Ivory</a> from Galois Inc. (another Haskell EDSL, of course) and knew quickly that I had found something very powerful for my own use-case. Their <a href="https://github.com/GaloisInc/smaccmpilot-experiencereport/blob/master/embedded-experience.pdf?raw=true">experience report</a> is a quick read, but a good summary of Ivory (as well as <a href="http://ivorylang.org/tower-overview.html">Tower</a>, yet another Haskell EDSL). It’s still a rather young project, and documentation and examples are sparse. However, it is under active development, and its developers (particularly Lee Pike, who I’ve bothered with plenty of emails) have been very responsive to my inquiries.</p>
<p>(To be clear, these all are Haskell EDSLs oriented to code generation. All of them in some fashion allow the programmer to express a specification within Haskell, and then generate C code at runtime. Haskell code is not actually being built for any embedded target - although, some authors here had interest in doing this via <a href="http://repetae.net/computer/jhc/">JHC</a>, and the <a href="http://www.idris-lang.org/">Idris</a> language extracts the entire runtime and program to C with some recent work going into making this light enough for embedded targets. A certain <a href="http://gauss.ececs.uc.edu/franco_files/franco.html">Dr. Franco</a> at my university insisted to me several years ago that code generation from a specification was the way of the future, and was most pleased when I suggested this year that perhaps he was correct.)</p>
<p>After some more reading and examination here, I started migrating the code over to a combination of Atom and Ivory. Atom quickly proved useful for managing a lot of the concurrency that was present, and generally handling high-level scheduling. The transition from C code to Ivory was rough in spots, but Ivory proved very useful too for representing all kinds of general functionality.</p>
<p>Haskell’s power as a language has been proving immensely useful here. Effectively, the EDSLs have moved the level of meaningful abstraction up into code-generation, and this has both given me much better abstraction and let me eliminate some runtime overhead.</p>
<p>More in-depth examples and explanations on things like Atom, Copilot, Ivory, and SBV will follow on this blog.</p>
<h1 id="references">References</h1>
<h2 id="papers">Papers</h2>
<ol style="list-style-type: decimal">
<li>Armstrong, J. Making reliable distributed systems in the presence of software errors. <a href="http://www.erlang.org/download/armstrong_thesis_2003.pdf" class="uri">http://www.erlang.org/download/armstrong_thesis_2003.pdf</a></li>
<li>Pike, L., Wegmann, N., Niller, S., &amp; Goodloe, A. (2012). Experience Report: a Do-It-Yourself High-Assurance Compiler. <a href="http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20120014570.pdf" class="uri">http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20120014570.pdf</a></li>
<li>Hickey, P. C., Pike, L., Elliott, T., Bielman, J., &amp; Launchbury, J. (2014) Building Embedded Systems with Embedded DSLs (Experience Report). <em>ACM 978-1-4503-2873-9/14/09.</em> <a href="https://github.com/GaloisInc/smaccmpilot-experiencereport/blob/master/embedded-experience.pdf?raw=true" class="uri">https://github.com/GaloisInc/smaccmpilot-experiencereport/blob/master/embedded-experience.pdf?raw=true</a></li>
</ol>
<h2 id="websites">Websites</h2>
<ol style="list-style-type: decimal">
<li>cocoOS <a href="http://www.cocoos.net/intro.html" class="uri">http://www.cocoos.net/intro.html</a></li>
<li>Cog: Python Success Stories <a href="https://www.python.org/about/success/cog/" class="uri">https://www.python.org/about/success/cog/</a></li>
<li>atom: A DSL for embedded hard realtime applications. <a href="http://hackage.haskell.org/package/atom" class="uri">http://hackage.haskell.org/package/atom</a> and <a href="https://github.com/tomahawkins/atom" class="uri">https://github.com/tomahawkins/atom</a></li>
<li>sbv: SMT Based Verification: Symbolic Haskell theorem prover using SMT solving. <a href="https://hackage.haskell.org/package/sbv" class="uri">https://hackage.haskell.org/package/sbv</a> and <a href="https://leventerkok.github.io/sbv/" class="uri">https://leventerkok.github.io/sbv/</a></li>
<li>copilot: A stream DSL for writing embedded C programs. <a href="http://hackage.haskell.org/package/copilot" class="uri">http://hackage.haskell.org/package/copilot</a> and <a href="https://leepike.github.io/Copilot/" class="uri">https://leepike.github.io/Copilot/</a></li>
<li>Ivory, an eDSL for safe systems programming. <a href="http://ivorylang.org/" class="uri">http://ivorylang.org/</a> and <a href="https://github.com/GaloisInc/ivory" class="uri">https://github.com/GaloisInc/ivory</a></li>
<li>Jhc Haskell Compiler. <a href="http://repetae.net/computer/jhc/" class="uri">http://repetae.net/computer/jhc/</a></li>
</ol>
]]></summary>
</entry>
<entry>
    <title>Haskelly Beginnings</title>
    <link href="http://haskellembedded.github.io//posts/2015-01-31-haskelly-beginnings.html" />
    <id>http://haskellembedded.github.io//posts/2015-01-31-haskelly-beginnings.html</id>
    <published>2015-01-31T00:00:00Z</published>
    <updated>2015-01-31T00:00:00Z</updated>
    <summary type="html"><![CDATA[<div class="info">
    Posted on January 31, 2015
    
        by Calvin Beck
    
</div>
<div class="info">
    
    Tags: 
    
</div>

<p>Greetings, all!</p>
<p>This is the start of a project to expand Haskell’s reach in the embedded world. At the moment we are simply exploring some of the available options, and we hope to expand on the documentation, and examples that are available. Here’s a list of some of the things which we are aware of? Got anymore? Let us know!</p>
<ul>
<li><a href="http://leepike.github.io/Copilot/">Copilot</a></li>
<li><a href="https://hackage.haskell.org/package/atom">Atom</a></li>
<li><a href="http://ivorylang.org/ivory-introduction.html">Ivory</a></li>
</ul>
<p>We’ll have a better idea of what’s going on with this soon. Right now this is essentially just a test of our super cool <a href="http://jaspervdj.be/hakyll/">Hakyll</a> blog using <a href="https://pages.github.com/">GitHub Pages</a></p>
<p>You can find the source for this blog <a href="https://github.com/HaskellEmbedded/HaskellEmbedded.github.io/tree/source">here</a>!</p>
]]></summary>
</entry>

</feed>