app/README.org

* Development Notes

- Currently, only tested on Linux (Debian). Everything should
  theoretically also work on Mac but some commands may need updating
  and it has not been tested, but multi-platform support, except for
  Windows, was kept in mind during development.

* Project Goal

To make deploying, managing, and updating self-hosted app instances
easy.

* Short Note On Generative AI

This project does NOT use any code or documentation generated by an
LLM and no code or text generated by an LLM is acceptable as a
contribution.

* Supported Services

Currently, only a limited set of external services can be used for
hosting, DNS, and backups. This can be easily extended later with
modifications to the Terraform config. As of now, you will need to
have accounts with these providers:

- DigitalOcean
- Cloudflare (and have DNS available for a domain)
- Backblaze B2

* Architecture

The software stack is composed of a "base" command line interface that
can be used to deploy and manage a single instance with a
multi-instance, multi-user webapp that invokes the "base" as
needed. The "base" can be run separately from the webapp. The webapp
automatically generates the configs the "base" needs to run.

The Makefile at the root of this source tree is the point of interface
for everything and all commands are run via make.

** "Base" Terraform Layer

The project is designed so that if you want to just manage a single
instance without the complexity of running a webapp you can easily do
so. This is both so that individual users can take advantage of this
but also so that when developing the Terraform and Docker Compose
setup it can be done and tested without needing to deal with the web
app as well.

The "base" layer is made up of the following: Flatcar Linux, Docker
Compose, Terraform, and a Makefile with a set of BASH scripts.

*** Flatcar Linux

The deployed instance runs on Flatcar Linux. Flatcar is a "read only"
Linux distribution designed to only run containers and nothing
else. Flatcar is used because it provides a high-level of security and
the OS itself auto-updates on a two-week schedule. Also, being "read
only" it is much more difficult for an external attacker to attack and
much harder for a user that does not know what they are doing to "mess
up".

*** Docker Compose

Each individual supported web app (like NextCloud, Ghost, etc) runs
via Docker and is configured via Docker Compose. (The docker compose
files are all in the "all-apps" directory in this source tree).

The Flatcar Linux config contains a systemd unit (service file) that
runs "docker compose". The Makefile copies all selected apps' docker
compose files from all-apps/ to app/. The systemd unit runs all the
docker compose files in the app/ directory. (The app/ directory is
what actually gets copied to the Flatcar linux install, not the
all-apps/ directory.)

The docker compose setup is specific and needs further documentation
here (to cover things like the shared load-balancer network setup and
how persistent storage is handled).

*** Terraform

Terraform is used to actually manage the deployed instances. Currently
it is a static terraform config controlled only via terraform
variables (see config/production.tfvars.tmpl). The terraform commands
are run via the Makefile.

** Webapp

The webapp is used both to provide a more "user-friendly" interface
for setting up and managing instances as well as to provide a
multi-user and multi-instance service. Internally, to manage an
instance, the webapp generates the configs and invokes the same
commands used when running the "base" CLI version by itself.

* Setup "Base" CLI Terraform For Deploying Individual Instance

NOTE: some of this may be outdated. It has not been tested on its own
outside of running via the webapp for a bit. It does work when run via
the webapp and all of the services still need to be setup as
detailed (the data can be input via the webapp instead of only via the
config files).

** Dependencies
- [[https://developer.hashicorp.com/terraform/install][terraform]]
- [[https://www.docker.com/][docker]]
- bash
- GNU Make

** Services

*** [[https://www.digitalocean.com/][DigitalOcean]]

- Create a DigitalOcean account and sign in to it

- [[https://cloud.digitalocean.com/account/api/tokens][Click "API" on sidebar]]

- Click "generate a new token"
  - enter a name (can be anything, just for you to remember)
  - set expiration as you desire
  - set the "scope" to "Full Access"
  - save the generated token for placing in production.tfvars -> do_token

*** [[https://www.cloudflare.com/][Cloudflare]]

- Create a CloudFlare account and sign into it

- Either register a new domain or if you already have a domain not being used you can continue with that

- On sidebar to to "Manage Account" -> "Account API Tokens"
  - Click "Create Token"
  - Under "templates" click "edit zone dns"
  - Under "Zone Resources", in the box labelled "Select..." select the domain you want to use
  - Click "continue to summary"
  - Click "create token"
  - Copy the token for use later on for the "cloudflare_api_token" in config/production.tfvars

- Click "Account Home" to go back to the top level

- Click on the domain you are using
  - This will show the "Overview"
  - Scroll down until you see the API heading and copy the "Zone ID" and "Account ID"

    These will be used later on in config/production.tfvars for cloudflare_zone_id and cloudflare_account_id

*** [[https://backblaze.com][Backblaze]]

This is used automated for "off-site" backups / snapshots.

- Create a Backblaze B2 account and sign in to it

- Click "create a bucket"
  - Give it a unique name (recommended something like [my-domain-com]-app-backups) but replace my-domain-com with your domain
  - Files in bucket should be set to "Private"
  - Leave "Default Encryption" as "disabled" (restic will encrypt the data)
  - Leave "Object Lock" as disabled

- Click on "Lifecycle Settings" under the newly created bucket
  - Change to "Keep only the last version of the file"
  - Click "Update Bucket"

- Under the bucket details copy "Endpoint" for use later on in config/apps.config BACKBLAZE_BUCKET_URL

- Click "Application Keys"
  - Click "Add a new application key"
  - "Name" can be whatever you want to remember it is a key for the backups for your apps
  - Change "Allow access to buckets" to only the bucket you created in the previous step
  - Leave "Type of Access" set to "Read and Write"
  - Leave other options in their default values
  - Click "Create new key"
  - Copy/save the key for later use in config/apps.config BACKBLAZE_APPLICATION_KEY and the "keyID" for BACKBLAZE_KEY_ID

** Configuration
*** apps.config

- ~cp config/apps.config.tmpl config/apps.config~

- then edit ~config/apps.config~ and fill in all variables

*** production.tfvars

- ~cp config/production.tfvars.tmpl config/production.tfvars~

- then edit ~config/production.tfvars~ and fill in all variables

*** ssh keys

- ~touch config/ssh-keys~

- if you want to add your ssh key(s) for debugging then paste the pub ID in to the file

*** initializing the "off-site" Restic backups

- ~make restic-init~

** Deploy

- ~make apply~

** You're done!

It will take a few minutes to deploy the server, start it, and pull
all the docker images. But after that you should be able to visit your
site and the apps running on its subdomains!

* Setup Webapp
** Dependencies

- [[https://code.call-cc.org/][CHICKEN Scheme 5.3+]]
- docker
- GNU Make

The webapp is written in Lisp (CHICKEN Scheme) and connects to a
PostgreSQL database. It also depends on being able to run some docker
commands. It has only been tested on Linux. Running the commands on
other platforms may need work. A Makefile command is provided for
running the Postgres database via docker so Postgres is not a listed
as a direct dependency.

** CHICKEN Scheme Libraries

These will need to be installed via the ~chicken-install~ command.

~postgresql sql-null srfi-1 srfi-13 srfi-18 srfi-158 srfi-194 openssl crypto-tools sxml-transforms schematra schematra-body-parser schematra-session uri-common http-client medea intarweb~

** html-widgets

This is a CHICKEN Scheme library that also needs to be installed but
it is not available via the ~chicken-install~ repository as I wrote it
for this project and I have not published it externally yet. You can
get the project here: [[https://code.thintz.com/tjhintz/html-widgets][https://code.thintz.com/tjhintz/html-widgets]]

After downloading the project, you can install it by ~cd~ to the
directory it is in and then running ~chicken-install~.