You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
194 lines
6.7 KiB
Org Mode
194 lines
6.7 KiB
Org Mode
* Development Notes
|
|
|
|
- Currently, only tested on Linux (Debian). Everything should
|
|
theoretically also work on Mac but some commands may need updating
|
|
and it has not been tested, but multi-platform support, except for
|
|
Windows, was kept in mind during development.
|
|
|
|
* Project Goal
|
|
|
|
To make deploying, managing, and updating self-hosted app instances
|
|
easy.
|
|
|
|
* Short Note On Generative AI
|
|
|
|
This project does NOT use any code or documentation generated by an
|
|
LLM and no code or text generated by an LLM is acceptable as a
|
|
contribution.
|
|
|
|
* Supported Services
|
|
|
|
Currently, only a limited set of external services can be used for
|
|
hosting, DNS, and backups. This can be easily extended later with
|
|
modifications to the Terraform config. As of now, you will need to
|
|
have accounts with these providers:
|
|
|
|
- DigitalOcean
|
|
- Cloudflare (and have DNS available for a domain)
|
|
- Backblaze B2
|
|
|
|
* Architecture
|
|
|
|
The software stack is composed of a "base" command line interface that
|
|
can be used to deploy and manage a single instance with a
|
|
multi-instance, multi-user webapp that invokes the "base" as
|
|
needed. The "base" can be run separately from the webapp. The webapp
|
|
automatically generates the configs the "base" needs to run.
|
|
|
|
** "Base" Terraform Layer
|
|
|
|
The project is designed so that if you want to just manage a single
|
|
instance without the complexity of running a webapp you can easily do
|
|
so. This is both so that individual users can take advantage of this
|
|
but also so that when developing the Terraform and Docker Compose
|
|
setup it can be done and tested without needing to deal with the web
|
|
app as well.
|
|
|
|
** Webapp
|
|
|
|
The webapp is used both to provide a more "user-friendly" interface
|
|
for setting up and managing instances as well as to provide a
|
|
multi-user and multi-instance service. Internally, to manage an
|
|
instance, the webapp generates the configs and invokes the same
|
|
commands used when running the "base" CLI version by itself.
|
|
|
|
* Setup "Base" CLI Terraform For Deploying Individual Instance
|
|
|
|
NOTE: some of this may be outdated. It has not been tested on its own
|
|
outside of running via the webapp for a bit. It does work when run via
|
|
the webapp and all of the services still need to be setup as
|
|
detailed (the data can be input via the webapp instead of only via the
|
|
config files).
|
|
|
|
** Dependencies
|
|
- [[https://developer.hashicorp.com/terraform/install][terraform]]
|
|
- [[https://www.docker.com/][docker]]
|
|
- bash
|
|
- GNU Make
|
|
|
|
** Services
|
|
|
|
*** [[https://www.digitalocean.com/][DigitalOcean]]
|
|
|
|
- Create a DigitalOcean account and sign in to it
|
|
|
|
- [[https://cloud.digitalocean.com/account/api/tokens][Click "API" on sidebar]]
|
|
|
|
- Click "generate a new token"
|
|
- enter a name (can be anything, just for you to remember)
|
|
- set expiration as you desire
|
|
- set the "scope" to "Full Access"
|
|
- save the generated token for placing in production.tfvars -> do_token
|
|
|
|
*** [[https://www.cloudflare.com/][Cloudflare]]
|
|
|
|
- Create a CloudFlare account and sign into it
|
|
|
|
- Either register a new domain or if you already have a domain not being used you can continue with that
|
|
|
|
- On sidebar to to "Manage Account" -> "Account API Tokens"
|
|
- Click "Create Token"
|
|
- Under "templates" click "edit zone dns"
|
|
- Under "Zone Resources", in the box labelled "Select..." select the domain you want to use
|
|
- Click "continue to summary"
|
|
- Click "create token"
|
|
- Copy the token for use later on for the "cloudflare_api_token" in config/production.tfvars
|
|
|
|
- Click "Account Home" to go back to the top level
|
|
|
|
- Click on the domain you are using
|
|
- This will show the "Overview"
|
|
- Scroll down until you see the API heading and copy the "Zone ID" and "Account ID"
|
|
|
|
These will be used later on in config/production.tfvars for cloudflare_zone_id and cloudflare_account_id
|
|
|
|
*** [[https://backblaze.com][Backblaze]]
|
|
|
|
This is used automated for "off-site" backups / snapshots.
|
|
|
|
- Create a Backblaze B2 account and sign in to it
|
|
|
|
- Click "create a bucket"
|
|
- Give it a unique name (recommended something like [my-domain-com]-app-backups) but replace my-domain-com with your domain
|
|
- Files in bucket should be set to "Private"
|
|
- Leave "Default Encryption" as "disabled" (restic will encrypt the data)
|
|
- Leave "Object Lock" as disabled
|
|
|
|
- Click on "Lifecycle Settings" under the newly created bucket
|
|
- Change to "Keep only the last version of the file"
|
|
- Click "Update Bucket"
|
|
|
|
- Under the bucket details copy "Endpoint" for use later on in config/apps.config BACKBLAZE_BUCKET_URL
|
|
|
|
- Click "Application Keys"
|
|
- Click "Add a new application key"
|
|
- "Name" can be whatever you want to remember it is a key for the backups for your apps
|
|
- Change "Allow access to buckets" to only the bucket you created in the previous step
|
|
- Leave "Type of Access" set to "Read and Write"
|
|
- Leave other options in their default values
|
|
- Click "Create new key"
|
|
- Copy/save the key for later use in config/apps.config BACKBLAZE_APPLICATION_KEY and the "keyID" for BACKBLAZE_KEY_ID
|
|
|
|
** Configuration
|
|
*** apps.config
|
|
|
|
- ~cp config/apps.config.tmpl config/apps.config~
|
|
|
|
- then edit ~config/apps.config~ and fill in all variables
|
|
|
|
*** production.tfvars
|
|
|
|
- ~cp config/production.tfvars.tmpl config/production.tfvars~
|
|
|
|
- then edit ~config/production.tfvars~ and fill in all variables
|
|
|
|
*** ssh keys
|
|
|
|
- ~touch config/ssh-keys~
|
|
|
|
- if you want to add your ssh key(s) for debugging then paste the pub ID in to the file
|
|
|
|
*** initializing the "off-site" Restic backups
|
|
|
|
- ~make restic-init~
|
|
|
|
** Deploy
|
|
|
|
- ~make apply~
|
|
|
|
** You're done!
|
|
|
|
It will take a few minutes to deploy the server, start it, and pull
|
|
all the docker images. But after that you should be able to visit your
|
|
site and the apps running on its subdomains!
|
|
|
|
* Setup Webapp
|
|
** Dependencies
|
|
|
|
- [[https://code.call-cc.org/][CHICKEN Scheme 5.3+]]
|
|
- docker
|
|
- GNU Make
|
|
|
|
The webapp is written in Lisp (CHICKEN Scheme) and connects to a
|
|
PostgreSQL database. It also depends on being able to run some docker
|
|
commands. It has only been tested on Linux. Running the commands on
|
|
other platforms may need work. A Makefile command is provided for
|
|
running the Postgres database via docker so Postgres is not a listed
|
|
as a direct dependency.
|
|
|
|
** CHICKEN Scheme Libraries
|
|
|
|
These will need to be installed via the ~chicken-install~ command.
|
|
~postgresql sql-null srfi-1 srfi-13 srfi-18 srfi-158 srfi-194 openssl crypto-tools sxml-transforms schematra schematra-body-parser schematra-session uri-common http-client medea intarweb~
|
|
|
|
** html-widgets
|
|
|
|
This is a CHICKEN Scheme library that also needs to be installed but
|
|
it is not available via the ~chicken-install~ repository as I wrote it
|
|
for this project and I have not published it externally yet. You can
|
|
get the project here: [[https://code.thintz.com/tjhintz/html-widgets][https://code.thintz.com/tjhintz/html-widgets]]
|
|
|
|
After downloading the project, you can install it by ~cd~ to the
|
|
directory it is in and then running ~chicken-install~.
|