• Development Notes
• Project Goal
• Short Note On Generative AI
• Supported Services
• Architecture
  • "Base" Terraform Layer
    • Flatcar Linux
    • Docker Compose
    • Terraform
  • Webapp
• Setup "Base" CLI Terraform For Deploying Individual Instance
  • Dependencies
  • Services
    • DigitalOcean
    • Cloudflare
    • Backblaze
  • Configuration
    • apps.config
    • production.tfvars
    • ssh keys
    • initializing the "off-site" Restic backups
  • Deploy
  • You're done!
• Setup Webapp
  • Dependencies
  • CHICKEN Scheme Libraries
  • html-widgets
  • Architecture
    • HTML Widgets
    • Styling

Development Notes

• Currently, only tested on Linux (Debian). Everything should theoretically also work on Mac but some commands may need updating and it has not been tested, but multi-platform support, except for Windows, was kept in mind during development.

Project Goal

To make deploying, managing, and updating self-hosted app instances easy.

Short Note On Generative AI

This project does NOT use any code or documentation generated by an LLM and no code or text generated by an LLM is acceptable as a contribution.

Supported Services

Currently, only a limited set of external services can be used for hosting, DNS, and backups. This can be easily extended later with modifications to the Terraform config. As of now, you will need to have accounts with these providers:

• DigitalOcean
• Cloudflare (and have DNS available for a domain)
• Backblaze B2

Architecture

The software stack is composed of a "base" command line interface that can be used to deploy and manage a single instance with a multi-instance, multi-user webapp that invokes the "base" as needed. The "base" can be run separately from the webapp. The webapp automatically generates the configs the "base" needs to run.

The Makefile at the root of this source tree is the point of interface for everything and all commands are run via make.

"Base" Terraform Layer

The project is designed so that if you want to just manage a single instance without the complexity of running a webapp you can easily do so. This is both so that individual users can take advantage of this but also so that when developing the Terraform and Docker Compose setup it can be done and tested without needing to deal with the web app as well.

The "base" layer is made up of the following: Flatcar Linux, Docker Compose, Terraform, and a Makefile with a set of BASH scripts.

Flatcar Linux

The deployed instance runs on Flatcar Linux. Flatcar is a "read only" Linux distribution designed to only run containers and nothing else. Flatcar is used because it provides a high-level of security and the OS itself auto-updates on a two-week schedule. Also, being "read only" it is much more difficult for an external attacker to attack and much harder for a user that does not know what they are doing to "mess up".

Docker Compose

Each individual supported web app (like NextCloud, Ghost, etc) runs via Docker and is configured via Docker Compose. (The docker compose files are all in the "all-apps" directory in this source tree).

The Flatcar Linux config contains a systemd unit (service file) that runs "docker compose". The Makefile copies all selected apps' docker compose files from all-apps/ to app/. The systemd unit runs all the docker compose files in the app/ directory. (The app/ directory is what actually gets copied to the Flatcar linux install, not the all-apps/ directory.)

The docker compose setup is specific and needs further documentation here (to cover things like the shared load-balancer network setup and how persistent storage is handled).

Terraform

Terraform is used to actually manage the deployed instances. Currently it is a static terraform config controlled only via terraform variables (see config/production.tfvars.tmpl). The terraform commands are run via the Makefile.

Webapp

The webapp is used both to provide a more "user-friendly" interface for setting up and managing instances as well as to provide a multi-user and multi-instance service. Internally, to manage an instance, the webapp generates the configs and invokes the same commands used when running the "base" CLI version by itself.

Setup "Base" CLI Terraform For Deploying Individual Instance

NOTE: some of this may be outdated. It has not been tested on its own outside of running via the webapp for a bit. It does work when run via the webapp and all of the services still need to be setup as detailed (the data can be input via the webapp instead of only via the config files).

Dependencies

• terraform
• docker
• bash
• GNU Make

Services

DigitalOcean

• Create a DigitalOcean account and sign in to it
• Click "API" on sidebar

• Click "generate a new token"

  • enter a name (can be anything, just for you to remember)
  • set expiration as you desire
  • set the "scope" to "Full Access"
  • save the generated token for placing in production.tfvars -> do_token

Cloudflare

• Create a CloudFlare account and sign into it
• Either register a new domain or if you already have a domain not being used you can continue with that

• On sidebar to to "Manage Account" -> "Account API Tokens"

  • Click "Create Token"
  • Under "templates" click "edit zone dns"
  • Under "Zone Resources", in the box labelled "Select…" select the domain you want to use
  • Click "continue to summary"
  • Click "create token"
  • Copy the token for use later on for the "cloudflare_api_token" in config/production.tfvars
• Click "Account Home" to go back to the top level

• Click on the domain you are using

  • This will show the "Overview"
  • Scroll down until you see the API heading and copy the "Zone ID" and "Account ID" These will be used later on in config/production.tfvars for cloudflare_zone_id and cloudflare_account_id

Backblaze

This is used automated for "off-site" backups / snapshots.

• Create a Backblaze B2 account and sign in to it

• Click "create a bucket"

  • Give it a unique name (recommended something like [my-domain-com]-app-backups) but replace my-domain-com with your domain
  • Files in bucket should be set to "Private"
  • Leave "Default Encryption" as "disabled" (restic will encrypt the data)
  • Leave "Object Lock" as disabled

• Click on "Lifecycle Settings" under the newly created bucket

  • Change to "Keep only the last version of the file"
  • Click "Update Bucket"
• Under the bucket details copy "Endpoint" for use later on in config/apps.config BACKBLAZE_BUCKET_URL

• Click "Application Keys"

  • Click "Add a new application key"
  • "Name" can be whatever you want to remember it is a key for the backups for your apps
  • Change "Allow access to buckets" to only the bucket you created in the previous step
  • Leave "Type of Access" set to "Read and Write"
  • Leave other options in their default values
  • Click "Create new key"
  • Copy/save the key for later use in config/apps.config BACKBLAZE_APPLICATION_KEY and the "keyID" for BACKBLAZE_KEY_ID

Configuration

apps.config

• cp config/apps.config.tmpl config/apps.config
• then edit config/apps.config and fill in all variables

production.tfvars

• cp config/production.tfvars.tmpl config/production.tfvars
• then edit config/production.tfvars and fill in all variables

ssh keys

• touch config/ssh-keys
• if you want to add your ssh key(s) for debugging then paste the pub ID in to the file

initializing the "off-site" Restic backups

• make restic-init

Deploy

• make apply

You're done!

It will take a few minutes to deploy the server, start it, and pull all the docker images. But after that you should be able to visit your site and the apps running on its subdomains!

Setup Webapp

Dependencies

• CHICKEN Scheme 5.3+
• docker
• GNU Make

The webapp is written in Lisp (CHICKEN Scheme) and connects to a PostgreSQL database. It also depends on being able to run some docker commands. It has only been tested on Linux. Running the commands on other platforms may need work. A Makefile command is provided for running the Postgres database via docker so Postgres is not a listed as a direct dependency.

CHICKEN Scheme Libraries

These will need to be installed via the chicken-install command.

postgresql sql-null srfi-1 srfi-13 srfi-18 srfi-158 srfi-194 openssl crypto-tools sxml-transforms schematra schematra-body-parser schematra-session uri-common http-client medea intarweb

html-widgets

This is a CHICKEN Scheme library that also needs to be installed but it is not available via the chicken-install repository as I wrote it for this project and I have not published it externally yet. You can get the project here: code.thintz.com/tjhintz/html-widgets

After downloading the project, you can install it by cd to the directory it is in and then running chicken-install.

Architecture

The webapp was designed to be very simple. Pages and handlers are based on the Schematra CHICKEN Scheme library (Chiccup is NOT used). Currently, there are no external dependencies other than docker, which is used to run postgres in development as well as generate ssh keys for users in production. There is no JavaScript or styling libraries. The webapp is built as HTML pages with forms.

The entry point for the webapp is src/nassella.scm.

HTML Widgets

The core of page markup is created with the html-widgets library which provides the define-widget function. This would be similar to a "component" in something like React. It allows defining properties of a widget (component) as well as a name and default values. It also handles style transforms, detailed in the next section.

Styling

All styling is done via style attributes on HTML elements. This is handled internally by the html-widgets library. The html-widgets library will extract all of the styles into a CSS stylesheet and replace the style attributes with a corresponding, auto-generated, CSS class name attribute. Nothing is needed to define or manage styles other than to set them directly via an HTML style attribute.

Shared styling values, like colors, are defined in the style-tokens global variable. It is a tree of style tokens (similar to Design Tokens). nassella.scm also includes some functions to make it easy for widgets to fetch style values. The most common being $ which allows getting a style token value based on a dotted symbol or a symbol list ($ 'color.primary.contrast) or ($ '(color primary contrast)