Rendering Logic Refactor

Table of Contents

• 1. Render Refactor
  • 1.1. Motivation
    • 1.1.1. Bugs
    • 1.1.2. Simplify Core
    • 1.1.3. Facilitate Future Development
  • 1.2. Approach

Render Refactor

This document proposes a refactor to converge's core engine to remove an explicit rendering phase, and to instead transform rendering into a query against a fact database that is generated through execution of the graph engine. This refactor will have significant effects on the internal architecture of the application, but should not break any existing APIs and should have only marginal effects on the execution of existing HCL files.

Motivation

Refactoring the core rendering code will allow us to address several bugs in one pass, along with simplifying the core execution engine and setting us up for a less painful implementation of proposed features.

Bugs

1. #493 (cannot use lookups in module params)

   This bug was the motivator for the proposed refactor; until we have a better mechanism for supporting rendering any fixes that support the use of lookup inside of a module will be hackish and require a lot of special-case code through the execution engine (see also 1.1.2).

   The proposed changes would eliminate any need for special handling of params defined in a module import statement.

2. #600 (module name template variable)

   The refactor would offer some minor benefits to the implementation of this feature; in particular tagging nodes with a specific module would happen during graph execution rather than at parse time, potentially reducing the need for special case handling of some scenarios.

3. #597 (add a way to determine if a resource will or has changed)

   The refactor would make execution queries a simple lookup in the fact database instead of requiring a full graph traversal. This would also minimize the number of special case branches required to deal with resources in switch branches or imported via module.

Simplify Core

The refactor will remove the initial rendering phase from the core execution engine. This will not only remove code, but will simplify the logical application flow, as graph execution will now simply be a fold over the loaded graph, with an execution context carrying information along the graph edges. We will eliminate the use of thunks and deferred evaluation completely, and should be able to supply better error information.

Facilitate Future Development

In addition to addressing bugs and simplifying the codebase, the refactor should help facilitate planned and proposed future development.

1. Resource-like HCL

   Supporting the ability to author resources with HCL using the proposed updated module syntax will require addressing #493. Supporting exported values from resource-like modules will also be easier when the modules may export values into the execution context without requiring graph traversal into child modules during lookups.

2. Type System

   Providing a fact database generated by the execution of nodes, especially a global context for variables, will facilitate the implementation of a type system. We should be able to perform runtime type analysis trivially and eventually perform some level of type inference at parse time by executing a type-check fold over the resources.

3. µKanren / Logic Refactor

   A fact database will be an early step toward refactoring some parts of the core engine to support logic programming of nodes.

Approach

We will begin by creating a simple data structure that will hold the results of ever phase of evaluation as well as node metadata. Graph traversal will fold over the graph and accumulate states, forking as necessary, and leaving us with n terminating states, where n is the number of leaf nodes in the graph. This may end up being represented monadically.