Loading
FretLink / @clementd && @haitlah

Dev Envs: Use The Nix, Luke

$ whoami

dev haskell, former CTO, I've worked with several langages, I used scala a lot (not anymore). Nowadays, haskell, rust, JS (a bit)

$ whoami

Makes shipment simpler (logistics platform for shipment planning and tracking). Started with nodeJS / MongoDB, new services are using Haskell / PostgreSQL

Let’s build an AI

Clementd

… in python

Keras / tensorflow

“just install anaconda”

“yeah you can choose miniconda as well”

“but you can still use pip”

“you know what? just use docker”

“actually, run this VM image on AWS, it’ll be easier”

cancel skynet

Python dev environments

Python is one of the worst offenders: global packages, lots of system deps

Python solutions

virtualenv. Works ok for python deps, but what about system deps?

meh :/

What if I just don’t want to bother with that? Running scripts was supposed to be simple

#magic

Hussein
small demo, just nix-shell -p

Nix

Clémentd
functional package manager
cross platform (GNU/Linux, MacOS)

Nix store

clementd
Content adressable store (data indexed by hash)
no global mutable root
just like git

Deterministic builds

hussein
data structures, not scripts

Deterministic builds

hussein
No side effects => good for cache / reuse

$PATH & patchelf magic

hussein
a special environment is created with all the needed resources in the path
if needed, the binaries themselves are patched

No deJS / No problem

clementd
we started with python because it was the easiest target. Node, no problem, thanks to npm, right. Right?

“just use nvm”

clementd
multiple node & npm versions in parallel

System deps / compiled deps are a PITA

clementd
nice and easy… until you have C bindings either you have to install libs through your system package manager, or the JS package downloads pre-built versions for some OSs

Let’s have a lookie

hussein
node js demo shell.nix et mkShell

node2nix

hussein
what we’ve done: use nix for node, npm & system deps, and npm for js deps but we can also use nix for js deps as well

nix all the things?

hussein
autogen nix files for whole dep tree (node, haskell, python as well?) much more work, out of scope for this project but can provide interesting things, especially for CI in polyglot orgs

clementd

Rust is nice

clementd
rustup works well, cargo is nice to use

… system deps aren’t

clementd
C dependencies

oxydize

clementd
rust depo
+ direnv integration

Why not docker?

clementd

Docker cache is… well…

clementd
docker cache is not really good
takes disk space, hard to reuse

Dockerfiles are… well…

clementd
dockerfile builds are not deterministic
dockerfiles are linear

step 1: docker-compose up

step 2

clementd
dockerized dev environments tend to get really complex.

clementd
we just use docker for dependency management but we have to configure everything: reverse proxy, ports, volumes, etc…

clementd
just run your services and manage them as you want fewer constraints (docker provides some isolation, not practical on a dev machine)

Drop the databass

hussein
local dbs are annoying, system-wide config and all that

The One True DataBase

hussein
postgresql demo / postgis / shell hooks

There’s more to it

clementd
nix-shell for dev envs is only a small part of what you can do with nix

nix to docker

hussein
some people seem to think using docker in production is a good idea reuse is easier than with dockerfiles (forced linearity) works on a real dep graph, then linearizes it => better than manually written dockerfiles

nix-overlays

hussein
custom config, easy to share => local override of packages (flags, patches…)

nix-env

clementd
nix-shell is good for small distinct environments
nix-env allows you to have executables always in your path
it also provides rollback / history capabilites

nixos

clementd
GNU/Linux distrib, built around nix
nix as a package manager (of course)
+ deterministic config for everything
=> you can build a whole OS from a config file
=> you can rollback everything (software update, config change, etc)

conclusion

clementd
impressive effort, best model
still requires some polish
documentation is still lacking
please document your progression

Thanks!

Do you have questions?

(we're hiring)