# SKATO ANSIBLE

## Installation

## Essential Background

### Software Management

The standard use-case, which we call the container routing case, of this Ansible library is to run its playbooks in such a way that one host ends up acting as a router (via VPN service) to services on another host.

In the container routing case these services are defined by a repository declared in `${ANSIBLE_PROJECT_ROOT}/vars/source_code.yml`, via setting `source_code.repos.compose.name` and `source_code.repos.compose.rpath`, respectively, to the name of a version-controlled source repository and to a desired path within that repository (that path written such that it treats the repository path as root, i.e. as `/`).

As can then be noted, the `source_code` dictionary holds an attribute `repos` with itself attributes of fixed name representing various services or utilities, each of whom must have a `name` attribute defined that represents the name of the corresponding version-controlled source repository for that service or utility.

The point of this set-up is to allow the repository name at its endpoint to change without requiring manual changes in the code using or deploying that repository as consequence. One unfortunate result of this is that adding a service or utiltiy from one's git bare repositories means making manual corresponding changes in `pkgs` software object groups in relevant [host variable files](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#organizing-host-and-group-variables), in `${ANSIBLE_PROJECT_ROOT}/roles/init-server/handlers/{core,userspace}.yml` for calling installation or post-installation actions, and in `${ANSIBLE_PROJECT_ROOT}/roles/init-server/tasks/contingent/pkg/${SOFTWARE_NAME}.yml` for executing those actions. The convention is for [handler](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#host-variables) [notifiers](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_handlers.html#notifying-handlers), [handler](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#host-variables) [listeners](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_handlers.html#notifying-handlers) and software names to all be equivalent, though the software name can deviate from this relational pattern.

Importantly, the following top-level attributes of the `source_code` variable or dictionary are necessary if your version-controlled source repository source requires authentication:

- `host`
- `user`/`ssh_user`
- `http_password`/`ssh_password`

If you wish to be able to use both SSH and HTTP authentication methods for the version-controlled source repository or change the default preference for HTTP credential use, you may edit `${ANSIBLE_PROJECT_ROOT}/roles/init-server/templates/user/netrc.j2`. See [the CURL docuemntation for more information on the NetRC file format](https://everything.curl.dev/usingcurl/netrc.html).

The point of the `source_code` dictionary is that it allows plugging in and executing custom code from one's own source repositories.

All other software is simply handled through the aforementioned patttern of adding an entry to any of the following in a given [host variables file](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#organizing-host-and-group-variables):

- `pkgs.mngr.core` or `pkgs.mngr.userspace` (software installed by package manager--in this case, `apt` as most tasks assume a Linux Debian system at Ansible's managed node)
- `pkgs.script.core` or `pkgs.script.userspace` (since different installation shell scripts allow for different parameters, entries for installation shell scripts here are downloaded--for actual installation, the script has to be ran in a set of tasks specified in a YAML file under `${ANSIBLE_PROJECT_ROOT}/roles/init-server/tasks/contingent/pkg/`)
- `pkgs.archive.core` or `pkgs.archive.userspace` (similar as before, except things like path movement or specification, building/compilation, etc., are what take place via the set of tasks specified under the YAML file under `${ANSIBLE_PROJECT_ROOT}/roles/init-server/tasks/contingent/pkg/`)
- `pkgs.git_repos.core` or `pkgs.git_repos.userspace` (similar as before, fulfilling any arbitrary set of specified tasks for the pulled git source repository--you get the point)

Each entry item must have a `handler` attribute whose value will be used to [notify](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_handlers.html#notifying-handlers) a [handler](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#host-variables) [listener](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_handlers.html#notifying-handlers), whose `listen` attribute has that same value, that then includes tasks found in a YAML file under `${ANSIBLE_PROJECT_ROOT}/roles/init-server/tasks/contingent/pkg/` whose basename, conventionally, is the same as those `handler` and `listen` attribute values. All of which, again, also conventionally share the same value as the `name` attribute of the entry item.

The structure of entries within each `pkgs.${SOFTWARE_GROUP}.core`/`pkgs.${SOFTWARE_GROUP}.userspace` depends on `$SOFTWARE_GROUP` because it changes the semantics of installation. All entries are dictionary items.

> [!NOTE]
> A table will be added at a later date, covering the attributes of the dictionary items for each `$SOFTWARE_GROUP`. For now, the [host variable files](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#organizing-host-and-group-variables) for the staging hosts found in the [YAML inventory file](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html) can be used as reference.

### Configuration Management

#### System Configuration

The most important configuration is that needed for each host. This involves software installation, though [that is already covered in the above section](#software-management). Other important [host variables](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#host-variables) for any given [host variable file](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#organizing-host-and-group-variables) are:

variable name | type | default | semantics
--- | --- | --- | ---
`password` | `str` | auto-generated | the hashed password for the root user if the managed node had not spawned prior to playbook run
`ssh_keys` | `list` | `["ed25519@staging", "ecdsa@staging"]` | the basenames of the SSH key pairs for the root user
`admins` | `list` | `[{"username": "senpai", "services": None, "ssh_keys": ["ed25519@staging", "ecdsa@staging"], "password": ""} for d in admins]` | the list of Linux superusers, i.e. administrators, to exist
`token` | `str` | `None` | API key for cloud hosting account to spawn a VPS
`origin` | `str` | `us-east` | region or location of the VPS to determine timezones and locale; useful for Akamai Linode cloud hosting
`operating_system` | `str` | `None` | name or path to operating system for cloud hosted VPS or a VM / bare metal system

#### Software Configuration

Other configuration involves Ansible role templates or files, e.g. those found in `${ANSIBLE_PROJECT_ROOT}/roles/init-server/{templates,files}`. Generally, these Ansible role templates or files are sent over to the Ansible managed node by tasks provided by that same role, and referred to by tasks in the [playbooks](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_intro.html).

The convention for templates is to set some desired though non- software native "defaults" and leave some settings or values to variables, grouped by the software for which they are relevant into variable files in `${ANSIBLE_PROJECT_ROOT}/vars/`, or what we may call the [playbook variable files](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_variables.html#defining-variables-in-included-files-and-roles). These YAML variable files by convention have as their basename `${SOFTWARE_NAME}_settings` (with some exceptions), and must be listed (as paths relative to `$ANSIBLE_PROJECT_ROOT`) under `vars_files` on the [playbook](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_intro.html) that will be running the tasks/[handlers](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#host-variables), native to itself or taken from a role, that send the respective files or (after processing) templates over to the managed Ansible node(s). These kinds of files shall be called "settings files."

Inside each such settings file, the convention is for there to be a top-level variable (i.e., a single variable that takes a dictionary value) with a name of `$SOFTWARE_NAME`, whose value has attributes representing configuration options for the given software.

Some common, shared attributes for settings files are:

attribute name | type | default value | required? | semantics
--- | --- | --- | --- | ---
`containerized` | `bool` | `True` | only for services or software that is service-capable | determines whether the configuration is for a containerized or base system install
`mode` | `str` | "prod" | no | usefully allows for setting the mode for an individual piece of software rather than having it automatically agree with [the playbook mode](#essential-usage) or having it assume production mode

Other common ones, especially for services, are `port` (which can be an integer or string) and--for web servers--`scheme` (which can be either string "http" or "https", serving to toggle SSL/TLS DNS authentication for HTTP connections). These can exist at any nesting level under the top-level dictionary, based on the feature scope of the given software.

## Essential Usage

### Development Mode

For development purposes in the container routing case, either first manually set up two Debian virtual machines, or (**recommended**) change into the `${ANSIBLE_PROJECT_ROOT}/staging0.test/` directory then `${ANSIBLE_PROJECT_ROOT}/staging1.test/` directory, running `vagrant up` for each. Running the [playbooks](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_intro.html) in development mode also requires supplying "dev" as an argument for the extra `mode` parameter.

> [!WARNING]
> Vagrantfiles for neither staging host has yet been tested. Proceed by your own discretion.

If you intend to run the [playbooks](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_intro.html) in production, the argument for this extra parameter should be "prod". The additional extra parameter of `chosen_host` should be adjusted to reflect the name of the host enlisted in the [YAML inventory file](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html) for production purposes.

> [!IMPORTANT]
> The names of hosts should be equivalent to their intended final domain name for services on any given host to work correctly.

Double-check, of course, that the correct values are set for `chosen_user`. The first playbook you run for each host or IP should always have `chosen_user` as "root". Execute the below while in the `$ANSIBLE_PROJECT_ROOT` directory:

> [!TIP]
> The extra parameter `chosen_user` for the second `ansible-playbook` command of each pair of `ansible-playbook` commands shown below should take an argument whose value is the same as that of the `username` attribute of any of the items in the `admins` list in the [host variables file](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#organizing-host-and-group-variables) for the given host.

```bash
#!/bin/bash
set -euo pipefail

SSH_KEYS=()

sudo ansible-playbook -Kk -i hosts.yml --extra-vars "chosen_host=staging0.test" --extra-vars "chosen_user=root" --extra-vars "mode=dev" init\@vps.yml
$SSH_KEYS | xargs -I %k ssh-add %k # ADD NEEDED SSH KEY FILEPATHS
ansible-playbook -Kk -i hosts.yml --extra-vars "chosen_host=staging0.test" --extra-vars "chosen_user=senpai" --extra-vars "mode=dev" administrate\@vps.yml

sudo ansible-playbook -Kk -i hosts.yml --extra-vars "chosen_host=staging1.test" --extra-vars "chosen_user=root" --extra-vars "mode=dev" init\@homeserver.yml
# UNCOMMENT BELOW TO REASSIGN ENVIRONMENT VARIABLE FOR NEW ARRAY OF SSH KEYPAIR FILEPATHS
# SSH_KEYS=()
# UNCOMMENT BELOW TO ADD UPDATED SSH KEY PAIR FILEPATHS TO SSH_AGENT
# $SSH_KEYS | xargs -I %k ssh-add %k
ansible-playbook -Kk -i hosts.yml --extra-vars "chosen_host=staging1.test" --extra-vars "chosen_user=senpai" --extra-vars "mode=dev" administrate\@homeserver.yml
```

> [!WARNING]
> In the container routing case, development mode may not supply an accurate test of the intended networking result if the virtual machines for staging were manually created, but the [playbook](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_intro.html) should run successfully. This is because apppropriate networking must be set for the virtual machine itself for VPN client containers in one virtual machine to "speak" with the VPN server in another virtual machine. Th supplied Vagrantfiles are a WIP.

### Production Mode

Its important to note that the above shell script example for development mode would have to be modified in production mode for the container routing case such that every `ansible-playbook` call instance has its extra parameter `chosen_host` share the same value throughout. This is due to the combination of how SSL/TLS and domain names are set up in the [playbook](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_intro.html) and the requirements of the container routing case. However, `chosen_host`'s value being the same across all `ansible-playbook` commands does not mean that value should represent the same managed node or host IP throughout. Whether it does so is of course determined by whatever is in the [YAML inventory file](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html).

To drive all that home, in production we may instead need something like the following (again, executed while in the `$ANSIBLE_PROJECT_ROOT` directory):

```bash
#!/bin/bash
set -euo pipefail

SSH_KEYS=()
HOST_FQDN=web.site # a fully qualified domain name you own
CURRENT_IP= # whatever the current IP of the host $HOST_FQDN is 
UPDATED_IP= # the desired new IP for the host $HOST_FQDN

sudo ansible-playbook -Kk -i hosts.yml --extra-vars "chosen_host=${HOST_FQDN}" --extra-vars "chosen_user=root" --extra-vars "mode=prod" init\@vps.yml
$SSH_KEYS | xargs -I %k ssh-add %k # ADD NEEDED SSH KEY PAIR FILEPATHS TO SSH_AGENT
ansible-playbook -Kk -i hosts.yml --extra-vars "chosen_host=${HOST_FQDN}" --extra-vars "chosen_user=senpai" --extra-vars "mode=prod" administrate\@vps.yml

sed '/'"$HOST_FQDN"':{N;s/'"$HOST_FQDN"':\n {6}ansible_host: '"$CURRENT_IP"'/'"$HOST_FQDN"':\n      ansible_host: '"$NEW_IP"'/g}' hosts.yml

sudo ansible-playbook -Kk -i hosts.yml --extra-vars "chosen_host=${HOST_FQDN}" --extra-vars "chosen_user=root" --extra-vars "mode=prod" init\@homeserver.yml
# UNCOMMENT BELOW TO REASSIGN ENVIRONMENT VARIABLE FOR NEW ARRAY OF SSH KEYPAIR FILEPATHS
# SSH_KEYS=()
# UNCOMMENT BELOW TO ADD UPDATED SSH KEY PAIR FILEPATHS TO SSH_AGENT
# $SSH_KEYS | xargs -I %k ssh-add %k
ansible-playbook -Kk -i hosts.yml --extra-vars "chosen_host=${HOST_FQDN}" --extra-vars "chosen_user=senpai" --extra-vars "mode=prod" administrate\@homeserver.yml
```

### Custom Playbooks

As long as you are matching the appropriate set of [playbook variable files](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_variables.html#defining-variables-in-included-files-and-roles) or settings files with the [playbook](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_intro.html) according to its need of those variable definitions/declarations, mostly determined by [the intended targeted hosts' own variables](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#host-variables) together with role [handlers](https://docs.ansible.com/projects/ansible/latest/inventory_guide/intro_inventory.html#host-variables), and as long as you have appropriately set up software installations, (see [Software Management](#software-management)) creating new/custom [playbooks](https://docs.ansible.com/projects/ansible/latest/playbook_guide/playbooks_intro.html) should be relatively easy.

Depending on what you are trying to do with a new playbook, the best approach may be to just copy/duplicate the extant playbook files and then edit them, as this allows one to mitigate errors as well as have a reliable reference point for changes.

### CLI Tool

As an alternative, you can use the `skato-ansible` shell script in `$ANSIBLE_PROJECT_ROOT` as an abstraction of these `ansible-playbook` commands, though it is a WIP that currently only makes sense for development mode playbook executions and that I plan to replace with a `git` submodule pulling from a CLI app project in the long-term.