Loading
FretLink / @clementd

Config As Code?
Yup, but properly.
Have some Dhall

$ whoami

dev haskell, former CTO, I've worked with several langages, I used scala a lot (not anymore). Nowadays, haskell, rust, JS (a bit)
mise en relation chargeurs et transporteurs. suivi plus transparent, aider les petits transporteurs environnement technique historique: node mongo. nouveaux services : haskell / pg

Config is solved, we have yaml

if you agree with this, you won’t like the rest

Config is solved, we have toml*

toml solves issues with yaml,
but these are not the ones I’m interested in

Configuration is complicated and repetitive

especially with infra as code and software-defined everything

Config languages are simple*

Configuration languages are designed to be simple. no abstraction, no indirection (* yaml is an error)

The case for config as code

Webpack

webpack config file: module export

Webpack




const path = require('path');

module.exports = {
  entry: './src/index.js',
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: 'bundle.js'
  }
};

Webpack - pros

plain objects, you can use variables, ternaries, maybe functions

“it’s just JavaScript”

not a new language to learn (who knows the nginx config file format?)

simple abstraction

you can split your config in several parts,
use properly named variables
maybe use functions

imports

you can split your config file is several files
and use import to join them

Webpack - cons

“it’s just JavaScript”

I don’t want my config files to turn into angular 1.2
new HtmlWebpackPlugin <- what does it do?

npm install

now the config file has deps, so you need to npm install before
looking at what’s in it

side effects

when is webpack run?

side effects

does it modify other config files?

side effects

can I cache the webpack config?

side effects

does it depend on software that’s not in package.json?

😱 side effects 😱

does reading the config drop my database?

no caching

no automated security audit

just a black box

that may or may not have visible effects

Pretty printing a JSON does not trash your hard drive

config files should just describe standalone values (eg more or less the JSON AST)

Config is not code

config has a different lifecycle from code, and different requirements
boundary that’s important to respect
examples: xmonad, things that you have to recompile to configure

Desirable properties

not talking about syntax bikeshedding

Clear syntax

won’t bikeshed here, but the syntax has to be unambiguous
avoid yaml’s mistakes (whitespace sensitivity, ambiguous literals)

Clear semantics

it should reduce to more or less an AST equivalent to JSON
records, lists, scalars (bools, strings, numbers)

Imports

we should be able to split config files as we please
in a structured, and well defined way
(eg, we compose values, not concat files)

Some abstraction

at least variables, restricted functions are ok
(eg to build records)

No turing completeness

a turing-complete interpreter can be turned into anything
huge security risks (or just DoS)

No side* effects

Evaluating a config file should do nothing except evaluating it

Some side effects are ok, though

env vars are so useful, it’d be a shame to forego them

Dhall

made just for this (and some more!)

Clear syntax




{- Comments -}
{ firstKey = "string value"
, otherKey = 42
, nested = [ { otherValue = True } ]
, multiline = ''
              I can do
              as well''
}

not whitespace significant records are defined with { / = / , records are defined with [ / , multiline strings, naturals, booleans

Clear semantics




{- Comments -}
{ firstKey = "string value"
, otherKey = 42
, nested = [ { otherValue = True } ]
, multiline = ''
              I can do
              as well''
}

the semantics is: reduce the file to this AST. Nothing more

Imports

let otherFile = ./other.dhall
in otherFile
we should be able to split config files as we please
in a structured, and well defined way
(eg, we compose values, not concat files)

Some abstraction





{- Don't repeat yourself -}
let user = "bill"
in  { home       = "/home/${user}"
    , privateKey = "/home/${user}/id_ed25519"
    , publicKey  = "/home/${user}/id_ed25519.pub"
    }

variables and functions (what else?)

Some abstraction





let
  makeUser = \(user : Text) ->
    { home       = "/home/${user}"
    , privateKey = "/home/${user}/id_ed25519"
    , publicKey  = "/home/${user}/id_ed25519.pub"
    }
in makeUser "bill"

simple functions

😍 Types 😍

for complex configs (especially when there is no config checker available)
faster feedback loop
great tool to reason about your config files
(useful to fight complexity)

Scalars

let port    : Natural = 8080
let offset  : Integer = +12
let ratio   : Double  = 12.0
let string  : Text    = "just a string"
let boolean : Bool    = True
in …
all literals are unambiguous (take that, yaml)

Composite



let maybe : Optional Natural =
      Some 8080
let list : List Text =
      ["a", "string"]
let record : { key : Text } =
      { key = "value" }
in  …
list are homogeneous
records types are structural

Records

let User = { name : Text }
let Admin = User //\\ { level : Text }
let localAdmin: Admin =
      { name = "John Doe"
      , level = "local"
      }
in localAdmin
record types can be aliased and combined to avoid repetition

Union types

real config files are messy 

let Host =
      { address : Text
      , port : Natural
      }
let Socket =
      < LocalFile : Text
      | RemoteHost : Host
      >
in  Socket.RemoteHost
      { address = "10.0.0.1"
      , port = 8080
      }

you can get back heterogeneous lists with union types

No turing completeness

general recursion is not allowed in the language
recursive structures are not directly possible

State of the art type system

it’s not external checks, it’s a type system proven to be sound
all the interesting properties of dhall are thanks to the choice
of a good type system.

No side* effects

the semantics are: reduce this expression as much as possible. no side effects are possible to express, this way

Some side effects are ok




-- read a string (without the quotes)
{ currentHome = env:HOME as Text

-- read any dhall expression
, currentPort = env:PORT }

Reading env vars is fine

Some side effects are ok





let map = https://prelude.dhall-lang.org/List/map

in map Natural Bool Natural/even [ 2, 3, 5 ]

http imports. super easy to reuse code
stdlib distributed this way

Some side effects are ok






let map =
  https://prelude.dhall-lang.org/List/map
    sha256:c60cc328c8aab8253e7372ae973ab7fd7b37448dc5bd9602f8e17b52a950d57b

in map Natural Bool Natural/even [ 2, 3, 5 ]

you can freeze imports to avoid changes and use cache socket(AF_INET6, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 0 open(“/home/clementd/.cache/dhall/c60cc328c8aab8253e7372ae973ab7fd7b37448dc5bd9602f8e17b52a950d57b”, O_RDONLY|O_NOCTTY|O_NONBLOCK) = 0

Clear semantics

cat evenList.dhall \
  | dhall resolve \
  | dhall normalize
first we resolve all imports (http + env)
then we evaluate (β-reduce)

Clear semantics






let map =
  https://prelude.dhall-lang.org/List/map
    sha256:c60cc328c8aab8253e7372ae973ab7fd7b37448dc5bd9602f8e17b52a950d57b

in map Natural Bool Natural/even [ 2, 3, 5 ]

to go from this

Clear semantics

[ True, False, False ]
to this
for instance: commit the thing above, but the ops team can read
the generated config, or automated tools / linters can act on
the generated output.

Editor support

syntax highlighting in many editors, some of them have extra
support

dhall-lsp work in progress, so that should benefit everyone

live-preview of the config output

How to use it

if nobody reads the config file, it’s not very useful

from Haskell

config is type checked by dhall, then translated to haskell

Language bindings

complete implementation
except go, it uses JSON as an intermediate format, so while you
can use the whole language, you cannot get all the expressible
typse in go (but normalization is complete)

Language bindings (in progress)

work in progress As soon as the rust impl is done, it may be easier
to have bindings in more languages, like JavaScript

this is all good and nice, but either your lang doesn’t have bindings
or you don’t have power over the config format (eg kuberetes)

Normalize to json/yaml

cat config.dhall \
  | dhall-to-yaml \
  > config.yaml
you can normalize almost everything into json and yaml (except functions):
primitive types, records, lists, unions 

let
  makeUser = \(user : Text) -> \(groups: List Text) ->
    { name = user
    , metadata =
      { groups = groups
      , userType = "regular"
      }
    }
in { users =
       [ makeUser "bill" ["wheel"]
       , makeUser "joe" ["yolo"]
       , makeUser "roy-eastman-kodak" ["clarita"]
       ]
   }

users:
- name: bill
  metadata:
    groups:
    - wheel
    userType: regular
- name: joe
  metadata:
    groups:
    - yolo
    userType: regular
- name: roy-eastman-kodak
  metadata:
    groups:
    - clarita
    userType: regular

Types for common software

model types and helpers for kubernetes, ansible, all that. This allows quick
and inexpensive checks for big config files before actually running software

dhall-kubernetes

types, default values
then you get the ability to use imports and functions to modularize your config

dhall-to-text for custom formats

you can generate any config format (but you have to write the projection function)

Closing words

dhall is worth it just for the imports + env vars features
it’s a great equivalent to hocon for instance
Types and functions can come in handy later

Don’t go overboard

Even with its limitations, dhall is impressively expressive. You can implement a lot with it
don’t forget this is configuration

it’s still config, some redundancy is desirable (more than in regular programming)
even though people can normalize the conf and read it, it should stay easy to read and edit
having all this power is cool, as long as you don’t abuse it

Let’s keep it ops-friendly

try to put you in your ops team shoes
work with them to find the abstraction level everyone is comfortable in

Resources

Thanks!