chrisfu/prole
1
0
Fork 0
mirror of https://github.com/dredx/prole.git synced 2026-09-23 11:03:59 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Enhance secret management and k8s infrastructure

Browse Source 
- Secret Management: Integrated AESGCM for temporary secret handling in install.py and enhanced OpenBao (Vault) support with namespace injection and additional secret paths (Grafana, Kerberos, TDE).
- Infrastructure & K8s:
    - Added Barman Object Store backup configuration (S3) to prole-db.yaml.
    - Updated Prometheus deployment with PVC and persistent configuration.
    - Updated k3s cluster/registry creation scripts.
    - Added etc/build-a-bao.sh for OpenBao setup.
- MSSQL Integration: Updated docker scripts and k8s deployments for Prole MSSQL database.
- Documentation: Added docs/PROLE-CFG-SECRETS.md explaining the new secret handling.
- General: Refined initialization scripts (init_authority.sh, init_openbao.sh, etc.) and updated the ncurses installer.
This commit is contained in:
chrisfu 2026-02-03 22:39:50 -08:00
parent edbc6c5385
commit bcc8f23a0d
31 changed files with 1083 additions and 123 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
docs/PROLE-CFG-SECRETS.md Normal file
View File

@ -0,0 +1,44 @@
# prole.cfg Secrets
This document describes how secrets are handled in `prole.cfg` and where they are stored in OpenBao.
## Temporary encrypted values
During installation, secrets are written to `prole.cfg` as temporary encrypted values so an interrupted run can resume:
- Format: `${PROLE_SECRET:v1:<nonce_b64>:<ciphertext_b64>}`
- Encryption: AES‑256‑GCM
- Key storage:
- macOS: login Keychain (service `prole-installer`)
- Other platforms: `~/.prole/secrets/installer.key` (0600)
These encrypted values are removed at Post‑Install by running `etc/build-a-bao.sh`.
## OpenBao placeholders
After Post‑Install, secrets in `prole.cfg` are replaced with OpenBao placeholders that point to a namespace‑scoped KV path:
- Format: `${OPENBAO:kv/prole/<namespace>/<leaf>#<key>}`
The namespace comes from `NAMESPACE` in `prole.cfg` and makes the file 1:1 with a single `prole-db` deployment.
## Secrets recorded in prole.cfg
The following keys are treated as secrets and stored in OpenBao:
| prole.cfg key | OpenBao KV path |
| --- | --- |
| `Inputs.init_password.db_password` | `kv/prole/<namespace>/db#password` |
| `Inputs.init_password.db_password_confirm` | `kv/prole/<namespace>/db#password` |
| `Global.DB_PASSWORD` | `kv/prole/<namespace>/db#password` |
| `Inputs.kerberos_config.password` | `kv/prole/<namespace>/kerberos#password` |
| `Kerberos Authentication.PASSWORD` | `kv/prole/<namespace>/kerberos#password` |
| `Monitoring.GRAFANA_ADMIN_PASSWORD` | `kv/prole/<namespace>/monitoring#grafana_admin_password` |
## Post‑Install step
Run the Build‑A‑Bao step on the Post‑Install screen (or `etc/build-a-bao.sh`) to:
1) Decrypt temporary secrets from `prole.cfg`
2) Write them to OpenBao under the namespace path
3) Replace `prole.cfg` secrets with OpenBao placeholders

244
etc/build-a-bao.sh Executable file
View File

@ -0,0 +1,244 @@
#!/usr/bin/env bash
set -euo pipefail
# build-a-bao.sh
# Purpose:
# - Decrypt temporary secrets stored in prole.cfg
# - Store them in OpenBao for the current namespace
# - Replace prole.cfg secrets with OpenBao placeholders
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# Load environment and config via prole_cfg.sh
# shellcheck disable=SC1090
source "$SCRIPT_DIR/prole_cfg.sh"
cfg_path=""
if [[ -n "${PROLE_CONF:-}" && -f "$PROLE_CONF/prole.cfg" ]]; then
cfg_path="$PROLE_CONF/prole.cfg"
elif [[ -n "${PROLE_HOME:-}" && -f "$PROLE_HOME/conf/prole.cfg" ]]; then
cfg_path="$PROLE_HOME/conf/prole.cfg"
elif [[ -f "$SCRIPT_DIR/../conf/prole.cfg" ]]; then
cfg_path="$SCRIPT_DIR/../conf/prole.cfg"
fi
if [[ -z "$cfg_path" || ! -f "$cfg_path" ]]; then
echo "ERROR: prole.cfg not found. Set PROLE_CONF or PROLE_HOME." >&2
exit 1
fi
tmp_env="$(mktemp)"
python3 - "$cfg_path" >"$tmp_env" <<'PY'
import base64
import configparser
import getpass
import os
import platform
import shlex
import subprocess
import sys
from pathlib import Path
PROLE_SECRET_PREFIX = "${PROLE_SECRET:"
PROLE_SECRET_SUFFIX = "}"
OPENBAO_PREFIX = "${OPENBAO:"
OPENBAO_SUFFIX = "}"
PROLE_SECRET_VERSION = "v1"
PROLE_SECRET_SERVICE = "prole-installer"
PROLE_SECRET_KEY_FILE = Path.home() / ".prole" / "secrets" / "installer.key"
def is_prole_secret(value: str | None) -> bool:
return bool(value) and value.startswith(PROLE_SECRET_PREFIX) and value.endswith(PROLE_SECRET_SUFFIX)
def is_openbao_ref(value: str | None) -> bool:
return bool(value) and value.startswith(OPENBAO_PREFIX) and value.endswith(OPENBAO_SUFFIX)
def get_keychain_key(service: str, account: str) -> bytes:
try:
res = subprocess.run(
["security", "find-generic-password", "-a", account, "-s", service, "-w"],
capture_output=True,
text=True,
)
if res.returncode == 0 and res.stdout.strip():
return base64.urlsafe_b64decode(res.stdout.strip().encode("utf-8"))
except Exception:
pass
key_b64 = base64.urlsafe_b64encode(os.urandom(32)).decode("utf-8")
try:
subprocess.run(
["security", "add-generic-password", "-a", account, "-s", service, "-w", key_b64, "-U"],
capture_output=True,
text=True,
)
except Exception:
pass
return base64.urlsafe_b64decode(key_b64.encode("utf-8"))
def get_file_key(path: Path) -> bytes:
path.parent.mkdir(parents=True, exist_ok=True)
if path.exists():
raw = path.read_text().strip()
try:
return base64.urlsafe_b64decode(raw.encode("utf-8"))
except Exception:
pass
key_b64 = base64.urlsafe_b64encode(os.urandom(32)).decode("utf-8")
path.write_text(key_b64)
try:
os.chmod(path, 0o600)
except Exception:
pass
return base64.urlsafe_b64decode(key_b64.encode("utf-8"))
def get_secret_key() -> bytes:
system = platform.system()
account = getpass.getuser() or "prole"
if system == "Darwin":
return get_keychain_key(PROLE_SECRET_SERVICE, account)
return get_file_key(PROLE_SECRET_KEY_FILE)
def decrypt_prole_secret(value: str) -> str:
if not is_prole_secret(value):
return value
inner = value[len(PROLE_SECRET_PREFIX):-len(PROLE_SECRET_SUFFIX)]
parts = inner.split(":")
if len(parts) != 3 or parts[0] != PROLE_SECRET_VERSION:
return ""
try:
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
nonce = base64.urlsafe_b64decode(parts[1].encode("utf-8"))
ciphertext = base64.urlsafe_b64decode(parts[2].encode("utf-8"))
key = get_secret_key()
aesgcm = AESGCM(key)
return aesgcm.decrypt(nonce, ciphertext, None).decode("utf-8")
except Exception:
return ""
cfg_path = Path(sys.argv[1]).expanduser()
cfg = configparser.ConfigParser(interpolation=None)
cfg.optionxform = str
cfg.read(cfg_path)
def get_val(section: str, key: str) -> str:
if cfg.has_option(section, key):
return cfg.get(section, key, fallback="").strip()
return ""
namespace = get_val("Global", "NAMESPACE") or get_val("Inputs", "env_setup.NAMESPACE") or get_val("Database Creation", "NAMESPACE") or "default"
db_user = get_val("Inputs", "init_password.db_username") or get_val("Global", "PROLE_DB_USER")
db_pass = get_val("Inputs", "init_password.db_password") or get_val("Global", "DB_PASSWORD")
krb_realm = get_val("Inputs", "kerberos_config.realm") or get_val("Kerberos Authentication", "REALM")
krb_kdc = get_val("Inputs", "kerberos_config.kdc") or get_val("Kerberos Authentication", "KDC")
krb_user = get_val("Inputs", "kerberos_config.user") or get_val("Kerberos Authentication", "USER")
krb_pass = get_val("Inputs", "kerberos_config.password") or get_val("Kerberos Authentication", "PASSWORD")
grafana_pass = get_val("Monitoring", "GRAFANA_ADMIN_PASSWORD")
at_rest = get_val("Inputs", "init_cluster.at_rest_encryption_enabled") or get_val("Optional Features", "AT_REST_ENCRYPTION_ENABLED")
if db_pass and not is_openbao_ref(db_pass):
db_pass = decrypt_prole_secret(db_pass)
else:
db_pass = ""
if krb_pass and not is_openbao_ref(krb_pass):
krb_pass = decrypt_prole_secret(krb_pass)
else:
krb_pass = ""
if grafana_pass and not is_openbao_ref(grafana_pass):
grafana_pass = decrypt_prole_secret(grafana_pass)
else:
grafana_pass = ""
def emit(key: str, value: str):
if value is None:
value = ""
print(f"export {key}={shlex.quote(str(value))}")
emit("NAMESPACE", namespace)
emit("PROLE_DB_USER", db_user)
emit("PROLE_DB_PASSWORD", db_pass)
emit("KRB5_REALM", krb_realm)
emit("KRB5_KDC", krb_kdc)
emit("KRB5_ADMIN", krb_kdc)
emit("KRB5_USER", krb_user)
emit("KRB5_PASSWORD", krb_pass)
emit("GRAFANA_ADMIN_PASSWORD", grafana_pass)
emit("AT_REST_ENCRYPTION_ENABLED", at_rest)
PY
# shellcheck disable=SC1090
source "$tmp_env"
rm -f "$tmp_env"
export PROLE_HOME="${PROLE_HOME:-$(cd "$SCRIPT_DIR/.." && pwd)}"
export PROLE_SERVICE="${PROLE_SERVICE:-$PROLE_HOME}"
export NAMESPACE="${NAMESPACE:-default}"
db_pass="${PROLE_DB_PASSWORD:-}"
if [[ -z "$db_pass" ]]; then
echo "[WARN] No database password found in prole.cfg; skipping db secret write."
fi
if [[ -n "$db_pass" ]]; then
printf '%s\n' "$db_pass" | "$SCRIPT_DIR/init_openbao.sh" initialize
else
"$SCRIPT_DIR/init_openbao.sh" initialize </dev/null
fi
python3 - "$cfg_path" <<'PY'
import configparser
import re
import sys
cfg_path = sys.argv[1]
cfg = configparser.ConfigParser(interpolation=None)
cfg.optionxform = str
cfg.read(cfg_path)
def get_val(section: str, key: str) -> str:
if cfg.has_option(section, key):
return cfg.get(section, key, fallback="").strip()
return ""
namespace = get_val("Global", "NAMESPACE") or get_val("Inputs", "env_setup.NAMESPACE") or get_val("Database Creation", "NAMESPACE") or "default"
def openbao_placeholder(ns: str, leaf: str, key: str) -> str:
return f"${{OPENBAO:kv/prole/{ns}/{leaf}#{key}}}"
secret_map = {
"init_password.db_password": openbao_placeholder(namespace, "db", "password"),
"init_password.db_password_confirm": openbao_placeholder(namespace, "db", "password"),
"kerberos_config.password": openbao_placeholder(namespace, "kerberos", "password"),
"DB_PASSWORD": openbao_placeholder(namespace, "db", "password"),
"PASSWORD": openbao_placeholder(namespace, "kerberos", "password"),
"GRAFANA_ADMIN_PASSWORD": openbao_placeholder(namespace, "monitoring", "grafana_admin_password"),
}
pattern = re.compile(r"^(?P<lead>\s*)(?P<key>[^=]+?)(?P<pre>\s*)=(?P<post>\s*).*$")
out_lines = []
with open(cfg_path, "r", encoding="utf-8") as f:
for line in f.read().splitlines():
stripped = line.strip()
if not stripped or stripped.startswith((';', '#')) or '=' not in line:
out_lines.append(line)
continue
m = pattern.match(line)
if not m:
out_lines.append(line)
continue
key = m.group("key").strip()
lead = m.group("lead")
pre = m.group("pre")
post = m.group("post")
if key in secret_map:
out_lines.append(f"{lead}{key}{pre}={post}{secret_map[key]}")
continue
if re.search(r"(?i)(?:^|[_.])(password|token|secret)$", key):
out_lines.append(f"{lead}{key}{pre}={post}${{OPENBAO:REDACTED}}")
continue
out_lines.append(line)
with open(cfg_path, "w", encoding="utf-8") as f:
f.write("\n".join(out_lines) + "\n")
print(f"[OK] Updated secrets in {cfg_path}")
PY

18
etc/init_authority.sh
View File

@ -20,6 +20,10 @@ ACTION=${1:-start}
NAMESPACE_AUTH="authority"
KDC_NAME="dog"
KDC_IMAGE="ubuntu:24.04"
BAO_NAMESPACE="${NAMESPACE:-default}"
BAO_PATH_PREFIX="prole/${BAO_NAMESPACE}"
BAO_PATH_ADMIN="${BAO_PATH_PREFIX}/admin"
BAO_PATH_AUTHORITY="${BAO_PATH_PREFIX}/authority"
log() { printf '%s\n' "$*"; }
err() { printf '%s\n' "$*" >&2; }
@ -83,27 +87,27 @@ deploy_dog() {
# Fetch secrets from OpenBao
local root_pass
root_pass=$(read_from_bao "prole/admin" "root_password")
root_pass=$(read_from_bao "$BAO_PATH_ADMIN" "root_password")
if [[ -z "$root_pass" || "$root_pass" == "null" ]]; then
root_pass=$(openssl rand -hex 16)
log "Generated new root password for dog server."
# We need to preserve existing data in prole/admin if we write to it
local admin_data
admin_data=$(get_openbao_url | xargs -I {} curl -sS -H "X-Vault-Token: $(get_openbao_token)" "{}/v1/kv/data/prole/admin" | jq -r ".data.data")
admin_data=$(get_openbao_url | xargs -I {} curl -sS -H "X-Vault-Token: $(get_openbao_token)" "{}/v1/kv/data/$BAO_PATH_ADMIN" | jq -r ".data.data")
if [[ "$admin_data" == "null" ]]; then
admin_data="{}"
fi
local new_admin_data
new_admin_data=$(echo "$admin_data" | jq -c ". + {\"root_password\":\"$root_pass\"}")
write_to_bao "prole/admin" "$new_admin_data"
write_to_bao "$BAO_PATH_ADMIN" "$new_admin_data"
fi
local ssh_pub
ssh_pub=$(read_from_bao "prole/admin" "admin_public_key_b64" | base64 -d 2>/dev/null || true)
ssh_pub=$(read_from_bao "$BAO_PATH_ADMIN" "admin_public_key_b64" | base64 -d 2>/dev/null || true)
local ansible_ssh_pub
# Try to read ansible key from OpenBao, fallback to local file if it exists
ansible_ssh_pub=$(read_from_bao "prole/admin" "ansible_public_key_b64" | base64 -d 2>/dev/null || true)
ansible_ssh_pub=$(read_from_bao "$BAO_PATH_ADMIN" "ansible_public_key_b64" | base64 -d 2>/dev/null || true)
if [[ -z "$ansible_ssh_pub" && -f "$HOME/.ssh/id_ed25519_ansible.pub" ]]; then
ansible_ssh_pub=$(cat "$HOME/.ssh/id_ed25519_ansible.pub")
fi
@ -198,8 +202,8 @@ configure_authority() {
kubectl exec -n "$NAMESPACE_AUTH" "$pod_name" -- bash -c "echo 'Setting up AD...'"
log "Saving administrator password to OpenBao ..."
write_to_bao "prole/authority" "{\"administrator_password\":\"$admin_pass\"}"
log "Administrator password saved to kv/prole/authority/administrator_password"
write_to_bao "$BAO_PATH_AUTHORITY" "{\"administrator_password\":\"$admin_pass\"}"
log "Administrator password saved to kv/$BAO_PATH_AUTHORITY/administrator_password"
local kdc_ip
kdc_ip=$(kubectl get svc -n "$NAMESPACE_AUTH" "$KDC_NAME" -o jsonpath='{.spec.clusterIP}' 2>/dev/null || echo "dog.${NAMESPACE_AUTH}.svc.cluster.local")

18
etc/init_cloudnative_pg.sh
View File

@ -56,6 +56,10 @@ ADMIN_PUB_ED25519="$SECRETS_DIR/admin_ed25519.pub"
ADMIN_PRIV_GENERIC="$SECRETS_DIR/admin.key"
ADMIN_PUB_GENERIC="$SECRETS_DIR/admin.pub"
OPENBAO_TOKEN_FILE="$SECRETS_DIR/openbao-root-token"
BAO_NAMESPACE="${NAMESPACE:-default}"
BAO_PATH_PREFIX="prole/${BAO_NAMESPACE}"
BAO_PATH_ADMIN="${BAO_PATH_PREFIX}/admin"
BAO_PATH_DB="${BAO_PATH_PREFIX}/db"
ensure_tools() {
for t in kubectl curl openssl base64 jq; do
@ -213,11 +217,11 @@ fetch_admin_keys_and_db_pass_from_bao_or_local() {
fi
url=$(bao_service_url)
if [[ -n "$token" ]]; then
echo "Attempting to read admin key pair from OpenBao kv/prole/admin ..."
if curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/prole/admin" | jq -e '.data.data' >/dev/null 2>&1; then
echo "Attempting to read admin key pair from OpenBao kv/$BAO_PATH_ADMIN ..."
if curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/$BAO_PATH_ADMIN" | jq -e '.data.data' >/dev/null 2>&1; then
local priv_b64 pub_b64
priv_b64=$(curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/prole/admin" | jq -r '.data.data.admin_private_key_b64')
pub_b64=$(curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/prole/admin" | jq -r '.data.data.admin_public_key_b64')
priv_b64=$(curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/$BAO_PATH_ADMIN" | jq -r '.data.data.admin_private_key_b64')
pub_b64=$(curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/$BAO_PATH_ADMIN" | jq -r '.data.data.admin_public_key_b64')
# Use a temporary file to determine where to save based on existing legacy or generic preference
local target_priv="$ADMIN_PRIV_GENERIC"
local target_pub="$ADMIN_PUB_GENERIC"
@ -232,10 +236,10 @@ fetch_admin_keys_and_db_pass_from_bao_or_local() {
cp "$target_pub" "$ADMIN_PUB_ED25519" 2>/dev/null || true
fi
echo "Attempting to read database password from OpenBao kv/prole/db ..."
if curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/prole/db" | jq -e '.data.data' >/dev/null 2>&1; then
echo "Attempting to read database password from OpenBao kv/$BAO_PATH_DB ..."
if curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/$BAO_PATH_DB" | jq -e '.data.data' >/dev/null 2>&1; then
local db_pass
db_pass=$(curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/prole/db" | jq -r '.data.data.password')
db_pass=$(curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/$BAO_PATH_DB" | jq -r '.data.data.password')
if [[ -n "$db_pass" ]]; then
echo "Updating database user secret 'prole-db-user' from OpenBao ..."
kubectl create secret generic prole-db-user -n "$NAMESPACE" \

46
etc/init_kerberos.sh
View File

@ -44,7 +44,7 @@ log() { printf '%s\n' "$*"; }
err() { printf '%s\n' "$*" >&2; }
ensure_tools() {
for t in kubectl; do
for t in kubectl curl jq; do
command -v "$t" >/dev/null || { err "Missing required tool: $t"; exit 1; }
done
}
@ -70,6 +70,48 @@ ensure_namespace() {
fi
}
openbao_url() {
if [[ -n "${PROLE_OPENBAO_URL:-}" ]]; then
echo "$PROLE_OPENBAO_URL"
elif curl -sS "http://127.0.0.1:18200/v1/sys/health" >/dev/null 2>&1; then
echo "http://127.0.0.1:18200"
else
echo "http://openbao.${NAMESPACE}.svc.cluster.local:8200"
fi
}
openbao_token() {
if [[ -f "$PROLE_SERVICE/secrets/openbao-root-token" ]]; then
cat "$PROLE_SERVICE/secrets/openbao-root-token"
else
echo "${OPENBAO_ROOT_TOKEN:-}"
fi
}
fetch_openbao_secret() {
local path="$1"
local key="$2"
local token url
token=$(openbao_token)
url=$(openbao_url)
if [[ -z "$token" ]]; then
echo ""
return 0
fi
curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/$path" | jq -r ".data.data.\"$key\"" || echo ""
}
resolve_krb5_password() {
if [[ -z "${KRB5_PASSWORD}" || "${KRB5_PASSWORD}" == '${OPENBAO:'* || "${KRB5_PASSWORD}" == '${PROLE_SECRET:'* ]]; then
local path="prole/${NAMESPACE:-default}/kerberos"
local fetched
fetched=$(fetch_openbao_secret "$path" "password")
if [[ -n "$fetched" && "$fetched" != "null" ]]; then
KRB5_PASSWORD="$fetched"
fi
fi
}
resolve_krb5_defaults() {
# If 'dog' KDC is available in 'authority' namespace, use it as default
if kubectl get pod -n "authority" -l app=dog >/dev/null 2>&1; then
@ -406,6 +448,7 @@ initialize() {
ensure_tools
ensure_namespace
resolve_krb5_defaults
resolve_krb5_password
# Support both names for the toggle from prole.cfg
local enabled="${KERBEROS_ENABLED:-${ENABLED:-false}}"
@ -437,6 +480,7 @@ run_test() {
ensure_tools
ensure_namespace
resolve_krb5_defaults
resolve_krb5_password
if [[ -z "${KRB5_REALM}" || -z "${KRB5_KDC}" || -z "${KRB5_USER}" || -z "${KRB5_PASSWORD}" ]]; then
err "ERROR: Missing Kerberos configuration. Ensure KRB5_REALM, KRB5_KDC, KRB5_USER, KRB5_PASSWORD are set."

45
etc/init_kerberos_test.sh
View File

@ -40,7 +40,7 @@ KRB5_TEST_HOST_NETWORK=${KRB5_TEST_HOST_NETWORK:-0}
KRB5_TEST_DNS_POLICY=${KRB5_TEST_DNS_POLICY:-}
ensure_tools() {
for t in kubectl; do
for t in kubectl curl jq; do
command -v "$t" >/dev/null || { echo "Missing required tool: $t" >&2; exit 1; }
done
}
@ -52,6 +52,48 @@ ensure_namespace() {
fi
}
openbao_url() {
if [[ -n "${PROLE_OPENBAO_URL:-}" ]]; then
echo "$PROLE_OPENBAO_URL"
elif curl -sS "http://127.0.0.1:18200/v1/sys/health" >/dev/null 2>&1; then
echo "http://127.0.0.1:18200"
else
echo "http://openbao.${NAMESPACE}.svc.cluster.local:8200"
fi
}
openbao_token() {
if [[ -f "$PROLE_SERVICE/secrets/openbao-root-token" ]]; then
cat "$PROLE_SERVICE/secrets/openbao-root-token"
else
echo "${OPENBAO_ROOT_TOKEN:-}"
fi
}
fetch_openbao_secret() {
local path="$1"
local key="$2"
local token url
token=$(openbao_token)
url=$(openbao_url)
if [[ -z "$token" ]]; then
echo ""
return 0
fi
curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/$path" | jq -r ".data.data.\"$key\"" || echo ""
}
resolve_krb5_password() {
if [[ -z "${KRB5_PASSWORD}" || "${KRB5_PASSWORD}" == '${OPENBAO:'* || "${KRB5_PASSWORD}" == '${PROLE_SECRET:'* ]]; then
local path="prole/${NAMESPACE:-default}/kerberos"
local fetched
fetched=$(fetch_openbao_secret "$path" "password")
if [[ -n "$fetched" && "$fetched" != "null" ]]; then
KRB5_PASSWORD="$fetched"
fi
fi
}
detect_image() {
if [[ -n "$KRB5_TEST_IMAGE" ]]; then
echo "$KRB5_TEST_IMAGE"
@ -192,6 +234,7 @@ join_realm() {
run_test() {
ensure_tools
ensure_namespace
resolve_krb5_password
# Support both names for the toggle from prole.cfg
local enabled="${KERBEROS_ENABLED:-${ENABLED:-false}}"

72
etc/init_openbao.sh
View File

@ -47,6 +47,14 @@ mkdir -p "$SECRETS_DIR"
admin_key_priv="$SECRETS_DIR/admin_ed25519.key"
admin_key_pub="$SECRETS_DIR/admin_ed25519.pub"
root_token_file="$SECRETS_DIR/openbao-root-token"
BAO_NAMESPACE="${NAMESPACE:-default}"
BAO_PATH_PREFIX="prole/${BAO_NAMESPACE}"
BAO_PATH_ADMIN="${BAO_PATH_PREFIX}/admin"
BAO_PATH_USER="${BAO_PATH_PREFIX}/user"
BAO_PATH_DB="${BAO_PATH_PREFIX}/db"
BAO_PATH_TDE="${BAO_PATH_PREFIX}/tde"
BAO_PATH_KERBEROS="${BAO_PATH_PREFIX}/kerberos"
BAO_PATH_MONITORING="${BAO_PATH_PREFIX}/monitoring"
ensure_tools() {
for t in curl openssl base64 jq; do
@ -288,16 +296,16 @@ init_openbao_kv_and_store_admin_key() {
# Check if admin key already exists and matches
local existing_data
existing_data=$(curl -sS -H "X-Vault-Token: $token" "$svc/v1/kv/data/prole/admin" 2>/dev/null || true)
existing_data=$(curl -sS -H "X-Vault-Token: $token" "$svc/v1/kv/data/$BAO_PATH_ADMIN" 2>/dev/null || true)
if [[ -n "$existing_data" ]]; then
local ex_priv ex_pub
ex_priv=$(echo "$existing_data" | grep -o '"admin_private_key_b64":"[^"]*' | cut -d'"' -f4 || true)
ex_pub=$(echo "$existing_data" | grep -o '"admin_public_key_b64":"[^"]*' | cut -d'"' -f4 || true)
if [[ "$ex_priv" == "$priv" && "$ex_pub" == "$pub" ]]; then
echo "Admin key pair in OpenBao kv/prole/admin is already up to date."
echo "Admin key pair in OpenBao kv/$BAO_PATH_ADMIN is already up to date."
else
echo "Writing admin key pair to kv/prole/admin ..."
echo "Writing admin key pair to kv/$BAO_PATH_ADMIN ..."
# Preserve other fields (like root_password or ansible_public_key_b64)
local current_json
current_json=$(echo "$existing_data" | jq -c ".data.data")
@ -305,32 +313,32 @@ init_openbao_kv_and_store_admin_key() {
new_json=$(echo "$current_json" | jq -c ". + {\"admin_private_key_b64\":\"$priv\",\"admin_public_key_b64\":\"$pub\"}")
curl -sS -H "X-Vault-Token: $token" -H 'Content-Type: application/json' \
-X POST "$svc/v1/kv/data/prole/admin" \
-X POST "$svc/v1/kv/data/$BAO_PATH_ADMIN" \
-d "{\"data\":$new_json}" >/dev/null
echo "Stored admin key pair in OpenBao kv/prole/admin."
echo "Stored admin key pair in OpenBao kv/$BAO_PATH_ADMIN."
fi
else
echo "Writing admin key pair to kv/prole/admin ..."
echo "Writing admin key pair to kv/$BAO_PATH_ADMIN ..."
curl -sS -H "X-Vault-Token: $token" -H 'Content-Type: application/json' \
-X POST "$svc/v1/kv/data/prole/admin" \
-X POST "$svc/v1/kv/data/$BAO_PATH_ADMIN" \
-d "{\"data\":{\"admin_private_key_b64\":\"$priv\",\"admin_public_key_b64\":\"$pub\"}}" >/dev/null
echo "Stored admin key pair in OpenBao kv/prole/admin."
echo "Stored admin key pair in OpenBao kv/$BAO_PATH_ADMIN."
fi
# Store Ansible public key if it exists
local ansible_pub_path="$HOME/.ssh/id_ed25519_ansible.pub"
if [[ -f "$ansible_pub_path" ]]; then
echo "Storing Ansible public key in OpenBao kv/prole/admin ..."
echo "Storing Ansible public key in OpenBao kv/$BAO_PATH_ADMIN ..."
local a_pub
a_pub=$(base64 <"$ansible_pub_path" | tr -d '\n')
local current_admin_data
current_admin_data=$(curl -sS -H "X-Vault-Token: $token" "$svc/v1/kv/data/prole/admin" | jq -c ".data.data" 2>/dev/null || echo "{}")
current_admin_data=$(curl -sS -H "X-Vault-Token: $token" "$svc/v1/kv/data/$BAO_PATH_ADMIN" | jq -c ".data.data" 2>/dev/null || echo "{}")
local updated_admin_data
updated_admin_data=$(echo "$current_admin_data" | jq -c ". + {\"ansible_public_key_b64\":\"$a_pub\"}")
curl -sS -H "X-Vault-Token: $token" -H 'Content-Type: application/json' \
-X POST "$svc/v1/kv/data/prole/admin" \
-X POST "$svc/v1/kv/data/$BAO_PATH_ADMIN" \
-d "{\"data\":$updated_admin_data}" >/dev/null
echo "Ansible public key stored."
fi
@ -344,20 +352,20 @@ init_openbao_kv_and_store_admin_key() {
u_pub=$(base64 <"$user_key_pub" | tr -d '\n')
local username=${PROLE_DB_USER:-"prole"}
echo "Writing user SSH keys for '$username' to kv/prole/user ..."
echo "Writing user SSH keys for '$username' to kv/$BAO_PATH_USER ..."
curl -sS -H "X-Vault-Token: $token" -H 'Content-Type: application/json' \
-X POST "$svc/v1/kv/data/prole/user" \
-X POST "$svc/v1/kv/data/$BAO_PATH_USER" \
-d "{\"data\":{\"username\":\"$username\",\"private_key_b64\":\"$u_priv\",\"public_key_b64\":\"$u_pub\"}}" >/dev/null
echo "Stored user SSH keys in OpenBao kv/prole/user."
echo "Stored user SSH keys in OpenBao kv/$BAO_PATH_USER."
fi
if [[ -n "$db_pass" ]]; then
local username=${PROLE_DB_USER:-"prole"}
echo "Writing database user password for '$username' to kv/prole/db ..."
echo "Writing database user password for '$username' to kv/$BAO_PATH_DB ..."
curl -sS -H "X-Vault-Token: $token" -H 'Content-Type: application/json' \
-X POST "$svc/v1/kv/data/prole/db" \
-X POST "$svc/v1/kv/data/$BAO_PATH_DB" \
-d "{\"data\":{\"username\":\"$username\",\"password\":\"$db_pass\"}}" >/dev/null
echo "Stored database password in OpenBao kv/prole/db."
echo "Stored database password in OpenBao kv/$BAO_PATH_DB."
fi
# Store TDE keypair when at-rest encryption is enabled
@ -381,30 +389,30 @@ init_openbao_kv_and_store_admin_key() {
tde_pub_b64=$(base64 <"$tde_pub" | tr -d '\n')
local tde_existing
tde_existing=$(curl -sS -H "X-Vault-Token: $token" "$svc/v1/kv/data/prole/tde" 2>/dev/null || true)
tde_existing=$(curl -sS -H "X-Vault-Token: $token" "$svc/v1/kv/data/$BAO_PATH_TDE" 2>/dev/null || true)
if [[ -n "$tde_existing" ]]; then
local ex_priv ex_pub
ex_priv=$(echo "$tde_existing" | grep -o '"tde_private_key_b64":"[^"]*' | cut -d'"' -f4 || true)
ex_pub=$(echo "$tde_existing" | grep -o '"tde_public_key_b64":"[^"]*' | cut -d'"' -f4 || true)
if [[ "$ex_priv" == "$tde_priv_b64" && "$ex_pub" == "$tde_pub_b64" ]]; then
echo "TDE keypair in OpenBao kv/prole/tde is already up to date."
echo "TDE keypair in OpenBao kv/$BAO_PATH_TDE is already up to date."
else
echo "Writing TDE keypair to kv/prole/tde ..."
echo "Writing TDE keypair to kv/$BAO_PATH_TDE ..."
local tde_json
tde_json=$(echo "$tde_existing" | jq -c ".data.data")
local new_tde_json
new_tde_json=$(echo "$tde_json" | jq -c ". + {\"tde_private_key_b64\":\"$tde_priv_b64\",\"tde_public_key_b64\":\"$tde_pub_b64\"}")
curl -sS -H "X-Vault-Token: $token" -H 'Content-Type: application/json' \
-X POST "$svc/v1/kv/data/prole/tde" \
-X POST "$svc/v1/kv/data/$BAO_PATH_TDE" \
-d "{\"data\":$new_tde_json}" >/dev/null
echo "Stored TDE keypair in OpenBao kv/prole/tde."
echo "Stored TDE keypair in OpenBao kv/$BAO_PATH_TDE."
fi
else
echo "Writing TDE keypair to kv/prole/tde ..."
echo "Writing TDE keypair to kv/$BAO_PATH_TDE ..."
curl -sS -H "X-Vault-Token: $token" -H 'Content-Type: application/json' \
-X POST "$svc/v1/kv/data/prole/tde" \
-X POST "$svc/v1/kv/data/$BAO_PATH_TDE" \
-d "{\"data\":{\"tde_private_key_b64\":\"$tde_priv_b64\",\"tde_public_key_b64\":\"$tde_pub_b64\"}}" >/dev/null
echo "Stored TDE keypair in OpenBao kv/prole/tde."
echo "Stored TDE keypair in OpenBao kv/$BAO_PATH_TDE."
fi
;;
esac
@ -412,11 +420,19 @@ init_openbao_kv_and_store_admin_key() {
local krb_user
krb_user="${KRB5_USER:-${KRB5_USERNAME:-}}"
if [[ -n "${KRB5_PASSWORD:-}" && -n "${KRB5_REALM:-}" && -n "$krb_user" ]]; then
echo "Writing Kerberos credentials for '$krb_user@${KRB5_REALM}' to kv/prole/kerberos ..."
echo "Writing Kerberos credentials for '$krb_user@${KRB5_REALM}' to kv/$BAO_PATH_KERBEROS ..."
curl -sS -H "X-Vault-Token: $token" -H 'Content-Type: application/json' \
-X POST "$svc/v1/kv/data/prole/kerberos" \
-X POST "$svc/v1/kv/data/$BAO_PATH_KERBEROS" \
-d "{\"data\":{\"username\":\"$krb_user\",\"realm\":\"$KRB5_REALM\",\"kdc\":\"${KRB5_KDC:-}\",\"password\":\"$KRB5_PASSWORD\"}}" >/dev/null
echo "Stored Kerberos credentials in OpenBao kv/prole/kerberos."
echo "Stored Kerberos credentials in OpenBao kv/$BAO_PATH_KERBEROS."
fi
if [[ -n "${GRAFANA_ADMIN_PASSWORD:-}" ]]; then
echo "Writing Grafana admin password to kv/$BAO_PATH_MONITORING ..."
curl -sS -H "X-Vault-Token: $token" -H 'Content-Type: application/json' \
-X POST "$svc/v1/kv/data/$BAO_PATH_MONITORING" \
-d "{\"data\":{\"grafana_admin_password\":\"$GRAFANA_ADMIN_PASSWORD\"}}" >/dev/null
echo "Stored Grafana admin password in OpenBao kv/$BAO_PATH_MONITORING."
fi
}

BIN
img/.DS_Store vendored
View File

Binary file not shown.

473
install.py
View File

@ -24,10 +24,12 @@ import tempfile
import getpass
import uuid
import re
import base64
from datetime import datetime
import urllib.request
import urllib.parse
import urllib.error
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
# Refactor: import shared helpers from the root-level installer package
from installer import config as inst_config
@ -85,6 +87,125 @@ DEFAULT_ACTION_FLAGS = {
'build.run_build': False,
}
# Secret handling (temporary encrypted values in prole.cfg)
PROLE_SECRET_PREFIX = "${PROLE_SECRET:"
PROLE_SECRET_SUFFIX = "}"
OPENBAO_PREFIX = "${OPENBAO:"
OPENBAO_SUFFIX = "}"
PROLE_SECRET_VERSION = "v1"
PROLE_SECRET_SERVICE = "prole-installer"
PROLE_SECRET_KEY_FILE = Path.home() / ".prole" / "secrets" / "installer.key"
# Map config keys to OpenBao paths (namespace injected at runtime)
SECRET_KEY_SPECS = {
("Inputs", "init_password.db_password"): ("db", "password"),
("Inputs", "init_password.db_password_confirm"): ("db", "password"),
("Inputs", "kerberos_config.password"): ("kerberos", "password"),
("Global", "DB_PASSWORD"): ("db", "password"),
("Kerberos Authentication", "PASSWORD"): ("kerberos", "password"),
("Monitoring", "GRAFANA_ADMIN_PASSWORD"): ("monitoring", "grafana_admin_password"),
}
def _is_prole_secret(value: str | None) -> bool:
return bool(value) and value.startswith(PROLE_SECRET_PREFIX) and value.endswith(PROLE_SECRET_SUFFIX)
def _is_openbao_ref(value: str | None) -> bool:
return bool(value) and value.startswith(OPENBAO_PREFIX) and value.endswith(OPENBAO_SUFFIX)
def _get_secret_key_file() -> Path:
return PROLE_SECRET_KEY_FILE
def _get_keychain_key(service: str, account: str) -> bytes:
try:
res = subprocess.run(
["security", "find-generic-password", "-a", account, "-s", service, "-w"],
capture_output=True,
text=True
)
if res.returncode == 0 and res.stdout.strip():
return base64.urlsafe_b64decode(res.stdout.strip().encode("utf-8"))
except Exception:
pass
key_b64 = base64.urlsafe_b64encode(os.urandom(32)).decode("utf-8")
try:
subprocess.run(
["security", "add-generic-password", "-a", account, "-s", service, "-w", key_b64, "-U"],
capture_output=True,
text=True
)
except Exception:
pass
return base64.urlsafe_b64decode(key_b64.encode("utf-8"))
def _get_file_key(path: Path) -> bytes:
path.parent.mkdir(parents=True, exist_ok=True)
if path.exists():
raw = path.read_text().strip()
try:
return base64.urlsafe_b64decode(raw.encode("utf-8"))
except Exception:
pass
key_b64 = base64.urlsafe_b64encode(os.urandom(32)).decode("utf-8")
path.write_text(key_b64)
try:
os.chmod(path, 0o600)
except Exception:
pass
return base64.urlsafe_b64decode(key_b64.encode("utf-8"))
def _get_secret_key() -> bytes:
system = platform.system()
account = getpass.getuser() or "prole"
if system == "Darwin":
return _get_keychain_key(PROLE_SECRET_SERVICE, account)
return _get_file_key(_get_secret_key_file())
def _encrypt_prole_secret(plaintext: str) -> str:
if plaintext is None:
return ""
if _is_prole_secret(plaintext):
return plaintext
key = _get_secret_key()
aesgcm = AESGCM(key)
nonce = os.urandom(12)
ciphertext = aesgcm.encrypt(nonce, plaintext.encode("utf-8"), None)
nonce_b64 = base64.urlsafe_b64encode(nonce).decode("utf-8")
ct_b64 = base64.urlsafe_b64encode(ciphertext).decode("utf-8")
return f"{PROLE_SECRET_PREFIX}{PROLE_SECRET_VERSION}:{nonce_b64}:{ct_b64}{PROLE_SECRET_SUFFIX}"
def _decrypt_prole_secret(value: str) -> str:
if not _is_prole_secret(value):
return value
inner = value[len(PROLE_SECRET_PREFIX):-len(PROLE_SECRET_SUFFIX)]
parts = inner.split(":")
if len(parts) != 3 or parts[0] != PROLE_SECRET_VERSION:
return value
try:
nonce = base64.urlsafe_b64decode(parts[1].encode("utf-8"))
ciphertext = base64.urlsafe_b64decode(parts[2].encode("utf-8"))
key = _get_secret_key()
aesgcm = AESGCM(key)
return aesgcm.decrypt(nonce, ciphertext, None).decode("utf-8")
except Exception:
return value
def _openbao_placeholder(namespace: str, leaf: str, key: str | None = None) -> str:
ns = (namespace or "").strip() or "default"
path = f"kv/prole/{ns}/{leaf}"
if key:
return f"{OPENBAO_PREFIX}{path}#{key}{OPENBAO_SUFFIX}"
return f"{OPENBAO_PREFIX}{path}{OPENBAO_SUFFIX}"
def _bool_str(val: bool) -> str:
return 'true' if bool(val) else 'false'
@ -796,6 +917,8 @@ class ProleInstaller:
'Prole DB Deploy': {},
'Install': {}
}
self._cfg_secret_cache = {}
self._secrets_finalized = False
# Try to load existing docker_import_dir from prole.cfg
try:
@ -874,6 +997,12 @@ class ProleInstaller:
except Exception:
pass
# Load any encrypted secrets from existing prole.cfg to allow resume
try:
self._load_secret_cache_from_cfg()
except Exception:
pass
# Create pages
self.page_frames = {}
self.verify_mode = tk.BooleanVar(value=False)
@ -2207,8 +2336,11 @@ class ProleInstaller:
# Database creation
_set('init_password.db_namespace', ns_val)
_set('init_password.db_username', _get_var(self.db_username, ''))
_set('init_password.db_password', _get_var(self.db_password, ''))
_set('init_password.db_password_confirm', _get_var(self.db_password_confirm, ''))
db_pw = _get_var(self.db_password, '')
db_pw_confirm = _get_var(self.db_password_confirm, '') or db_pw
db_pw_cfg = self._secret_cfg_value('Inputs', 'init_password.db_password', db_pw, 'db', 'password')
_set('init_password.db_password', db_pw_cfg)
_set('init_password.db_password_confirm', db_pw_cfg or db_pw_confirm)
_set('init_password.db_host_port', _get_var(self.db_host_port, '5432'))
_set_bool('init_password.generate_ssh_key', _action('init_password.generate_ssh_key', DEFAULT_ACTION_FLAGS.get('init_password.generate_ssh_key', True)))
@ -2227,7 +2359,8 @@ class ProleInstaller:
_set('kerberos_config.realm', _get_var(self.kerberos_realm, ''))
_set('kerberos_config.kdc', _get_var(self.kerberos_kdc, ''))
_set('kerberos_config.user', _get_var(self.kerberos_user, ''))
_set('kerberos_config.password', _get_var(self.kerberos_password, ''))
krb_pw = _get_var(self.kerberos_password, '')
_set('kerberos_config.password', self._secret_cfg_value('Inputs', 'kerberos_config.password', krb_pw, 'kerberos', 'password'))
_set_bool('kerberos_config.test_connection', _action('kerberos_config.test_connection', DEFAULT_ACTION_FLAGS.get('kerberos_config.test_connection', False)))
_set_bool('kerberos_config.init_authority', _action('kerberos_config.init_authority', DEFAULT_ACTION_FLAGS.get('kerberos_config.init_authority', False)))
@ -2296,6 +2429,116 @@ class ProleInstaller:
tk.Button(btn_frame, text="Show Logs (lsof)", command=show_lsof, bg='#F5F5DC', relief='flat', padx=10).pack(side='left', padx=10)
tk.Button(btn_frame, text="OK", command=dialog.destroy, bg='#007aff', fg='white', relief='flat', padx=20).pack(side='left', padx=10)
def _secret_namespace(self) -> str:
ns = ''
try:
ns = (self.db_namespace.get() or '').strip()
except Exception:
ns = ''
if not ns:
ns = os.environ.get('NAMESPACE', '') or 'default'
return ns
def _secret_cfg_value(self, section: str, key: str, plaintext: str, leaf: str, bao_key: str) -> str:
cache_key = (section, key)
cached = self._cfg_secret_cache.get(cache_key, '')
if self._secrets_finalized:
value = _openbao_placeholder(self._secret_namespace(), leaf, bao_key)
self._cfg_secret_cache[cache_key] = value
return value
if plaintext:
try:
value = _encrypt_prole_secret(plaintext)
self._cfg_secret_cache[cache_key] = value
return value
except Exception:
return cached or ''
if cached:
return cached
return ''
def _load_secret_cache_from_cfg(self):
try:
if self._cfg_path_override is not None:
cfg_path = self._cfg_path_override
if cfg_path.is_dir():
cfg_path = cfg_path / 'prole.cfg'
else:
conf_dir = self._resolve_prole_conf_dir()
cfg_path = conf_dir / 'prole.cfg'
except Exception:
return
if not cfg_path.exists():
return
cfg = configparser.ConfigParser(interpolation=None)
cfg.optionxform = str
cfg.read(cfg_path)
for (section, key), _spec in SECRET_KEY_SPECS.items():
if cfg.has_option(section, key):
val = cfg.get(section, key, fallback="").strip()
if val:
self._cfg_secret_cache[(section, key)] = val
if _is_openbao_ref(val):
self._secrets_finalized = True
def _resolve_cached(pairs: list[tuple[str, str]]) -> str:
for pair in pairs:
val = self._cfg_secret_cache.get(pair, '')
if val:
return val
return ''
db_val = _resolve_cached([
("Inputs", "init_password.db_password"),
("Global", "DB_PASSWORD"),
])
if db_val and _is_prole_secret(db_val):
db_plain = _decrypt_prole_secret(db_val)
elif db_val and not _is_openbao_ref(db_val):
db_plain = db_val
else:
db_plain = ''
if db_plain:
try:
self.db_password.set(db_plain)
self.db_password_confirm.set(db_plain)
except Exception:
pass
krb_val = _resolve_cached([
("Inputs", "kerberos_config.password"),
("Kerberos Authentication", "PASSWORD"),
])
if krb_val and _is_prole_secret(krb_val):
krb_plain = _decrypt_prole_secret(krb_val)
elif krb_val and not _is_openbao_ref(krb_val):
krb_plain = krb_val
else:
krb_plain = ''
if krb_plain:
try:
self.kerberos_password.set(krb_plain)
except Exception:
pass
def _sanitize_sections_for_cfg(self, sections: dict) -> dict:
sanitized = {k: dict(v) for k, v in sections.items()}
# Kerberos password
if "Kerberos Authentication" in sanitized:
val = sanitized["Kerberos Authentication"].get("PASSWORD", "")
if val or ("Kerberos Authentication", "PASSWORD") in self._cfg_secret_cache:
sanitized["Kerberos Authentication"]["PASSWORD"] = self._secret_cfg_value(
"Kerberos Authentication", "PASSWORD", val, "kerberos", "password"
)
# Monitoring (Grafana)
if "Monitoring" in sanitized:
val = sanitized["Monitoring"].get("GRAFANA_ADMIN_PASSWORD", "")
if val or ("Monitoring", "GRAFANA_ADMIN_PASSWORD") in self._cfg_secret_cache:
sanitized["Monitoring"]["GRAFANA_ADMIN_PASSWORD"] = self._secret_cfg_value(
"Monitoring", "GRAFANA_ADMIN_PASSWORD", val, "monitoring", "grafana_admin_password"
)
return sanitized
def _save_prole_cfg(self):
"""Generates prole.cfg; master configuration file containing all values used in install process."""
try:
@ -2332,13 +2575,16 @@ class ProleInstaller:
globals_to_save = {
'PROLE_HOME': os.environ.get('PROLE_HOME', ''),
'PROLE_DB_USER': self.db_username.get(),
'DB_PASSWORD': self.db_password.get(),
'DB_PASSWORD': self._secret_cfg_value('Global', 'DB_PASSWORD', self.db_password.get(), 'db', 'password'),
'CLUSTER_ENV': self.cluster_env.get(),
'NAMESPACE': (self.db_namespace.get() or '').strip(),
'DB_HOST_PORT': (self.db_host_port.get() or '5432').strip()
}
# Add any other values already in self.prole_cfg_data['Global']
globals_to_save.update(self.prole_cfg_data.get('Global', {}))
globals_to_save['DB_PASSWORD'] = self._secret_cfg_value(
'Global', 'DB_PASSWORD', self.db_password.get(), 'db', 'password'
)
# Inputs section (replayable UI inputs)
try:
@ -2348,10 +2594,11 @@ class ProleInstaller:
sections = {k: self.prole_cfg_data.get(k, {}) for k in [
'Welcome', 'Dependencies', 'Network', 'System Environment',
'Kerberos Authentication', 'Optional Features', 'Database Creation',
'Monitoring', 'Kerberos Authentication', 'Optional Features', 'Database Creation',
'Initialize Cluster', 'Docker Build', 'Initialization Scripts',
'Prole DB Deploy', 'Install'
]}
sections = self._sanitize_sections_for_cfg(sections)
cfg_text = _render_prole_cfg(inputs, globals_to_save, sections)
cfg_path.write_text(cfg_text)
@ -2570,7 +2817,7 @@ class ProleInstaller:
# Ensure PYTHONUNBUFFERED=1 is set in case prole-agent is/uses Python
env = os.environ.copy()
env['PYTHONUNBUFFERED'] = '1'
process = subprocess.Popen([str(scan_binary), "-t", "10", "-s"],
process = subprocess.Popen([str(scan_binary), "-t", "10"],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
bufsize=0,
@ -3686,8 +3933,13 @@ class ProleInstaller:
self.k3s_services_status["openbao"].set("Checking...")
# Connection details for remote k3s (same as in ncurses version)
k3s_server = "https://pi.prole.org:6443"
k3s_token = "K10af6fadc27a4cc8b859a10fc69259bc90378c54ecb5f895cbe7c54c6954faa7e0::server:7ad0aa18511842387814d4fb4bf6461f"
k3s_server = os.environ.get("PROLE_K3S_SERVER", "https://pi.prole.org:6443")
k3s_token = os.environ.get("PROLE_K3S_TOKEN", "")
if not k3s_token:
self.k3s_services_status["registry"].set("Missing token")
self.k3s_services_status["openbao"].set("Missing token")
return
base_cmd = [
"kubectl",
@ -6621,6 +6873,45 @@ echo "-------------------------------------------------------------------";
threading.Thread(target=worker, daemon=True).start()
def run_build_a_bao(self):
def worker():
self.safe_after(lambda: self._build_a_bao_button.configure(state='disabled') if hasattr(self, '_build_a_bao_button') and self._build_a_bao_button.winfo_exists() else None)
self.safe_after(lambda: self.bg_canvas.itemconfig(self._build_a_bao_status_label, text="Saving secrets to OpenBao...", fill='blue') if hasattr(self, '_build_a_bao_status_label') and self.bg_canvas.winfo_exists() else None)
script_name = "build-a-bao.sh"
if hasattr(self, 'install_consoles') and script_name in self.install_consoles:
self.safe_after(lambda: self.install_tabs.select(self.install_consoles[script_name].master) if hasattr(self, 'install_tabs') and self.install_tabs.winfo_exists() else None)
env = os.environ.copy()
env["PROLE_HOME"] = str(PROJECT_ROOT)
env["PROLE_SERVICE"] = str(PROJECT_ROOT)
env["NAMESPACE"] = (self.db_namespace.get() or '').strip()
console = getattr(self, 'install_consoles', {}).get(script_name)
if console:
console.clear()
console.write(f"Running {script_name} ...\n")
def _on_line(line):
if console:
console.write(line)
rc = self.controller.run_script(
script_name,
env=env,
on_line=_on_line
)
if rc == 0:
self._secrets_finalized = True
self.safe_after(lambda: self.bg_canvas.itemconfig(self._build_a_bao_status_label, text="Secrets saved to OpenBao.", fill='#34c759') if hasattr(self, '_build_a_bao_status_label') and self.bg_canvas.winfo_exists() else None)
else:
self.safe_after(lambda: self.bg_canvas.itemconfig(self._build_a_bao_status_label, text=f"Failed with code {rc}", fill='#ff3b30') if hasattr(self, '_build_a_bao_status_label') and self.bg_canvas.winfo_exists() else None)
self.safe_after(lambda: self._build_a_bao_button.configure(state='normal') if hasattr(self, '_build_a_bao_button') and self._build_a_bao_button.winfo_exists() else None)
threading.Thread(target=worker, daemon=True).start()
def _collect_install_logs(self) -> list[Path]:
logs_dir = self._resolve_prole_logs_dir()
build_logs = []
@ -6739,18 +7030,31 @@ echo "-------------------------------------------------------------------";
# Add final_deployment.sh tab
_add_tab('final_deployment.sh', script_name='final_deployment.sh')
# Add build-a-bao.sh tab
_add_tab('build-a-bao.sh', script_name='build-a-bao.sh')
# Use tk.Button
# Build-A-Bao Button
self._build_a_bao_button = tk.Button(self.bg_canvas, text='Build-A-Bao', command=self.run_build_a_bao,
bg='#F5F5DC', fg='black', activebackground='#E5E5D5',
highlightbackground='#F5F5DC', highlightthickness=0,
relief='flat', font=('SF Pro Text', 11), padx=16, pady=8)
bao_btn_window = self.bg_canvas.create_window(48, 760, window=self._build_a_bao_button, anchor='nw', width=180)
self._canvas_items.append(bao_btn_window)
self._overlay_widgets.append(self._build_a_bao_button)
# Save Deployment Button
self._save_deployment_button = tk.Button(self.bg_canvas, text='Save Deployment', command=self.run_final_deployment,
bg='#F5F5DC', fg='black', activebackground='#E5E5D5',
highlightbackground='#F5F5DC', highlightthickness=0,
relief='flat', font=('SF Pro Text', 11), padx=16, pady=8)
btn_window = self.bg_canvas.create_window(48, 760, window=self._save_deployment_button, anchor='nw', width=180)
btn_window = self.bg_canvas.create_window(240, 760, window=self._save_deployment_button, anchor='nw', width=180)
self._canvas_items.append(btn_window)
self._overlay_widgets.append(self._save_deployment_button)
# Status Label
self._save_deployment_status_label = ui.canvas_text(self, 240, 772, "", fill='black', font=('SF Pro Text', 12))
# Status Labels
self._build_a_bao_status_label = ui.canvas_text(self, 48, 802, "", fill='black', font=('SF Pro Text', 12))
self._canvas_items.append(self._build_a_bao_status_label)
self._save_deployment_status_label = ui.canvas_text(self, 240, 802, "", fill='black', font=('SF Pro Text', 12))
self._canvas_items.append(self._save_deployment_status_label)
def _render_build_summary_page(self):
@ -7924,7 +8228,6 @@ esac
if krb_img:
images.add(krb_img)
images.discard('supabase/supabase:latest')
return sorted(images)
def _prepull_images_to_registry(self, include_supabase: bool, include_kerberos_proxy: bool, log=None) -> bool:
@ -8350,6 +8653,8 @@ class ProleSilentInstaller:
'Prole DB Deploy': {},
'Install': {}
}
self._cfg_secret_cache = {}
self._secrets_finalized = False
self._db_built_success = False
self._scripts_success = False
self._cnpg_success = False
@ -8380,6 +8685,13 @@ class ProleSilentInstaller:
if not self.cfg_path.exists():
raise FileNotFoundError(f"prole.cfg not found at {self.cfg_path}")
cfg.read(self.cfg_path)
for (section, key), _spec in SECRET_KEY_SPECS.items():
if cfg.has_option(section, key):
val = cfg.get(section, key, fallback="").strip()
if val:
self._cfg_secret_cache[(section, key)] = val
if _is_openbao_ref(val):
self._secrets_finalized = True
self._auto_input_keys = set()
auto_kdc = None
auto_enabled = None
@ -8404,6 +8716,12 @@ class ProleSilentInstaller:
if cfg.has_section('Inputs'):
inputs.update({k: v for k, v in cfg.items('Inputs')})
mark_auto(inputs)
# Resolve encrypted/anchored secrets for runtime use
for key in ('init_password.db_password', 'init_password.db_password_confirm', 'kerberos_config.password'):
if key in inputs and inputs[key]:
inputs[key] = self._resolve_secret_value(inputs[key])
if inputs.get('init_password.db_password') and not inputs.get('init_password.db_password_confirm'):
inputs['init_password.db_password_confirm'] = inputs['init_password.db_password']
return inputs
# Legacy fallback mapping
legacy = {}
@ -8472,6 +8790,11 @@ class ProleSilentInstaller:
if 'DB_USER' in sec:
legacy['init_password.db_username'] = sec.get('DB_USER', '')
mark_auto(legacy)
for key in ('init_password.db_password', 'init_password.db_password_confirm', 'kerberos_config.password'):
if key in legacy and legacy[key]:
legacy[key] = self._resolve_secret_value(legacy[key])
if legacy.get('init_password.db_password') and not legacy.get('init_password.db_password_confirm'):
legacy['init_password.db_password_confirm'] = legacy['init_password.db_password']
return legacy
def _default_inputs(self) -> dict:
@ -8544,6 +8867,86 @@ class ProleSilentInstaller:
def _get_input_bool(self, key: str, default: bool = False) -> bool:
return _parse_bool(self._get_input(key, None), default=default)
def _secret_namespace(self) -> str:
ns = (self._get_input('init_password.db_namespace', '') or '').strip()
if not ns:
ns = (self._get_input('env_setup.NAMESPACE', '') or '').strip()
return ns or 'default'
def _secret_cfg_value(self, section: str, key: str, plaintext: str, leaf: str, bao_key: str) -> str:
cache_key = (section, key)
cached = self._cfg_secret_cache.get(cache_key, '')
if self._secrets_finalized:
value = _openbao_placeholder(self._secret_namespace(), leaf, bao_key)
self._cfg_secret_cache[cache_key] = value
return value
if plaintext:
try:
value = _encrypt_prole_secret(plaintext)
self._cfg_secret_cache[cache_key] = value
return value
except Exception:
return cached or ''
if cached:
return cached
return ''
def _resolve_openbao_ref(self, value: str) -> str:
if not _is_openbao_ref(value):
return value
inner = value[len(OPENBAO_PREFIX):-len(OPENBAO_SUFFIX)]
path, key = (inner.split('#', 1) + [""])[:2]
if not path or not key:
return ""
mount = "kv"
secret_path = path
if "/" in path:
maybe_mount, rest = path.split("/", 1)
if maybe_mount:
mount = maybe_mount
secret_path = rest
token = os.environ.get("OPENBAO_ROOT_TOKEN", "")
if not token:
prole_service = os.environ.get("PROLE_SERVICE")
if prole_service:
token_path = Path(prole_service) / "secrets" / "openbao-root-token"
if token_path.exists():
token = token_path.read_text().strip()
if not token:
return value
url = (os.environ.get("PROLE_OPENBAO_URL") or "http://127.0.0.1:18200").rstrip("/")
try:
req = urllib.request.Request(f"{url}/v1/{mount}/data/{secret_path}")
req.add_header("X-Vault-Token", token)
with urllib.request.urlopen(req, timeout=4) as resp:
payload = json.loads(resp.read().decode("utf-8"))
return payload.get("data", {}).get("data", {}).get(key, "") or ""
except Exception:
return value
def _resolve_secret_value(self, value: str) -> str:
if _is_prole_secret(value):
return _decrypt_prole_secret(value)
if _is_openbao_ref(value):
return self._resolve_openbao_ref(value)
return value
def _sanitize_sections_for_cfg(self, sections: dict) -> dict:
sanitized = {k: dict(v) for k, v in sections.items()}
if "Kerberos Authentication" in sanitized:
val = sanitized["Kerberos Authentication"].get("PASSWORD", "")
if val or ("Kerberos Authentication", "PASSWORD") in self._cfg_secret_cache:
sanitized["Kerberos Authentication"]["PASSWORD"] = self._secret_cfg_value(
"Kerberos Authentication", "PASSWORD", val, "kerberos", "password"
)
if "Monitoring" in sanitized:
val = sanitized["Monitoring"].get("GRAFANA_ADMIN_PASSWORD", "")
if val or ("Monitoring", "GRAFANA_ADMIN_PASSWORD") in self._cfg_secret_cache:
sanitized["Monitoring"]["GRAFANA_ADMIN_PASSWORD"] = self._secret_cfg_value(
"Monitoring", "GRAFANA_ADMIN_PASSWORD", val, "monitoring", "grafana_admin_password"
)
return sanitized
# ---------------- Namespace helpers ----------------
def _get_local_owner(self) -> str:
try:
@ -8862,7 +9265,6 @@ class ProleSilentInstaller:
if krb_img:
images.add(krb_img)
images.discard('supabase/supabase:latest')
return sorted(images)
def _prepull_images_to_registry(self, include_supabase: bool, include_kerberos_proxy: bool) -> bool:
@ -9002,24 +9404,39 @@ class ProleSilentInstaller:
self.inputs['kerberos_config.enabled'] = _bool_str(True)
def _write_cfg(self):
inputs = dict(self.inputs)
db_pw = self._get_input('init_password.db_password', '')
db_pw_cfg = self._secret_cfg_value('Inputs', 'init_password.db_password', db_pw, 'db', 'password')
if db_pw_cfg:
inputs['init_password.db_password'] = db_pw_cfg
inputs['init_password.db_password_confirm'] = db_pw_cfg
elif db_pw:
inputs['init_password.db_password'] = db_pw
inputs['init_password.db_password_confirm'] = self._get_input('init_password.db_password_confirm', '') or db_pw
krb_pw = self._get_input('kerberos_config.password', '')
if krb_pw:
inputs['kerberos_config.password'] = self._secret_cfg_value('Inputs', 'kerberos_config.password', krb_pw, 'kerberos', 'password')
globals_to_save = {
'PROLE_HOME': self._get_input('env_setup.PROLE_HOME', ''),
'PROLE_DB_USER': self._get_input('init_password.db_username', ''),
'DB_PASSWORD': self._get_input('init_password.db_password', ''),
'DB_PASSWORD': self._secret_cfg_value('Global', 'DB_PASSWORD', db_pw, 'db', 'password'),
'CLUSTER_ENV': self._get_input('init_cluster.cluster_env', ''),
'NAMESPACE': (self._get_input('init_password.db_namespace', '') or '').strip(),
'DB_HOST_PORT': (self._get_input('init_password.db_host_port', '5432') or '5432').strip(),
'DOCKER_IMPORT_DIR': self.docker_import_dir or ''
}
globals_to_save.update(self.prole_cfg_data.get('Global', {}))
globals_to_save['DB_PASSWORD'] = self._secret_cfg_value('Global', 'DB_PASSWORD', db_pw, 'db', 'password')
sections = {k: self.prole_cfg_data.get(k, {}) for k in [
'Welcome', 'Dependencies', 'Network', 'System Environment',
'Kerberos Authentication', 'Optional Features', 'Database Creation',
'Monitoring', 'Kerberos Authentication', 'Optional Features', 'Database Creation',
'Initialize Cluster', 'Docker Build', 'Initialization Scripts',
'Prole DB Deploy', 'Install'
]}
cfg_text = _render_prole_cfg(self.inputs, globals_to_save, sections)
sections = self._sanitize_sections_for_cfg(sections)
cfg_text = _render_prole_cfg(inputs, globals_to_save, sections)
self.cfg_path.parent.mkdir(parents=True, exist_ok=True)
self.cfg_path.write_text(cfg_text)
self.log(f"[CONFIG] Wrote {self.cfg_path}")
@ -9112,7 +9529,7 @@ class ProleSilentInstaller:
except Exception:
continue
rc = self._run_cmd([str(scan_binary), "-t", "10", "-s"], cwd=str(scan_dir), on_stdout=_handle_stdout)
rc = self._run_cmd([str(scan_binary), "-t", "10"], cwd=str(scan_dir), on_stdout=_handle_stdout)
if rc == 0:
self.log("[OK] Scan complete.")
else:
@ -9346,7 +9763,7 @@ class ProleSilentInstaller:
self.log(f"[OK] Cluster ready: {cluster_name}")
def _step_kerberos(self) -> None:
enabled = self._get_input_bool('kerberos_config.enabled', self.kerberos_enabled.get())
enabled = self._get_input_bool('kerberos_config.enabled', False)
if not enabled:
self.log("[SKIP] Kerberos disabled.")
return
@ -9503,7 +9920,7 @@ class ProleSilentInstaller:
self.err(f"[ERROR] Rollout failed (code {rc2})")
def _step_supabase(self) -> None:
if not self._get_input_bool('supabase_config.run_deploy', self.supabase_enabled.get()):
if not self._get_input_bool('supabase_config.run_deploy', False):
self.log("[SKIP] Supabase deploy disabled.")
self.prole_cfg_data['Supabase'] = {'STATUS': 'Skipped'}
return
@ -9536,6 +9953,19 @@ class ProleSilentInstaller:
self.prole_cfg_data['Supabase'] = {'STATUS': 'Attempted'}
self.err(f"[ERROR] Supabase deploy failed (code {rc})")
def _finalize_secrets(self) -> None:
self.log("==> OpenBao: finalize secrets")
env = os.environ.copy()
env["PROLE_HOME"] = str(self.project_root)
env["PROLE_SERVICE"] = str(self.project_root)
env["NAMESPACE"] = (self._get_input('init_password.db_namespace', '') or '').strip()
rc = self._run_script("build-a-bao.sh", env=env)
if rc == 0:
self._secrets_finalized = True
self.log("[OK] Secrets saved to OpenBao and prole.cfg updated.")
else:
self.err(f"[WARN] build-a-bao.sh failed (code {rc}).")
def run(self) -> int:
_configure_unbuffered_io()
self.log(f"[CONFIG] Using {self.cfg_path}")
@ -9580,6 +10010,9 @@ class ProleSilentInstaller:
self._step_cnpg_deploy()
self._write_cfg()
self._finalize_secrets()
self._write_cfg()
self.prole_cfg_data['Install']['STATUS'] = 'Finished'
self._write_cfg()
self.log("[DONE] Silent install completed.")

8
installer/ncurses_installer.py
View File

@ -53,8 +53,8 @@ class ProleNcursesInstaller:
self.prod_artifacts_path.set_value(str(self.project_root / "data" / "staging"))
# Connection details for remote k3s
self.k3s_server = "https://pi.prole.org:6443"
self.k3s_token = "K10af6fadc27a4cc8b859a10fc69259bc90378c54ecb5f895cbe7c54c6954faa7e0::server:7ad0aa18511842387814d4fb4bf6461f"
self.k3s_server = os.environ.get("PROLE_K3S_SERVER", "https://pi.prole.org:6443")
self.k3s_token = os.environ.get("PROLE_K3S_TOKEN", "")
try:
self.namespace_owner = getpass.getuser()
@ -322,6 +322,10 @@ class ProleNcursesInstaller:
"""Verify registry:2 and openbao on remote k3s cluster."""
def _verify():
self.k3s_services_status = {"registry": "Checking...", "openbao": "Checking..."}
if not self.k3s_token:
self.k3s_services_status = {"registry": "Missing token", "openbao": "Missing token"}
return
base_cmd = [
"kubectl",

72
k3s/create_k3d_prole_data_cluster.sh
View File

@ -1,11 +1,71 @@
#!/bin/bash
set -x
export PROLE_HOME=$HOME/dev/prole
PROLE_PASSWD=`cat $PROLE_HOME/prole-db/.prole_user_password | base64 -d`
HOST_IP=`ifconfig en0 | grep 'inet ' | awk '{print $2}'`
#!/usr/bin/env bash
set -o pipefail
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck disable=SC1090
source "$SCRIPT_DIR/../etc/prole_cfg.sh"
set -euo pipefail
PROLE_HOME="${PROLE_HOME:-$(cd "$SCRIPT_DIR/.." && pwd)}"
export PROLE_HOME
PROLE_SERVICE="${PROLE_SERVICE:-$PROLE_HOME/etc}"
export PROLE_SERVICE
openbao_url() {
if [[ -n "${PROLE_OPENBAO_URL:-}" ]]; then
echo "$PROLE_OPENBAO_URL"
return 0
fi
echo "http://127.0.0.1:18200"
}
openbao_token() {
if [[ -f "$PROLE_SERVICE/secrets/openbao-root-token" ]]; then
cat "$PROLE_SERVICE/secrets/openbao-root-token"
else
echo "${OPENBAO_ROOT_TOKEN:-}"
fi
}
fetch_openbao_secret() {
local path="$1"
local key="$2"
local token url
token=$(openbao_token)
url=$(openbao_url)
if [[ -z "$token" ]]; then
echo ""
return 0
fi
curl -sS -H "X-Vault-Token: $token" "$url/v1/kv/data/$path" | jq -r ".data.data.\"$key\"" || echo ""
}
PROLE_PASSWD="${PROLE_PASSWD:-}"
if [[ -z "$PROLE_PASSWD" || "$PROLE_PASSWD" == '${OPENBAO:'* || "$PROLE_PASSWD" == '${PROLE_SECRET:'* ]]; then
command -v curl >/dev/null 2>&1 || { echo "ERROR: curl is required to read OpenBao secrets." >&2; exit 1; }
command -v jq >/dev/null 2>&1 || { echo "ERROR: jq is required to read OpenBao secrets." >&2; exit 1; }
path="prole/${NAMESPACE:-default}/db"
fetched=$(fetch_openbao_secret "$path" "password")
if [[ -n "$fetched" && "$fetched" != "null" ]]; then
PROLE_PASSWD="$fetched"
fi
fi
if [[ -z "$PROLE_PASSWD" ]]; then
echo "ERROR: PROLE_PASSWD is required. Store it in OpenBao (kv/prole/<namespace>/db#password) or export it." >&2
exit 1
fi
HOST_IP="${HOST_IP:-}"
if [[ -z "$HOST_IP" ]]; then
HOST_IP=$(ifconfig en0 2>/dev/null | grep 'inet ' | awk '{print $2}' | head -n1 || true)
fi
if [[ -z "$HOST_IP" ]]; then
echo "ERROR: Unable to detect HOST_IP. Set HOST_IP to the host interface address." >&2
exit 1
fi
k3d cluster create prole-data-cluster \
--agents 2 \
--registry-use k3d-prole-data-registry:5000 \
--registry-config ${PROLE_HOME}/k3s/registries.yaml \
--k3s-arg "--datastore-endpoint=mysql://prole:$PROLE_PASSWD\@tcp($HOST_IP:3306)/k3s@server:*"

5
k3s/create_k3d_prole_data_registry.sh
View File

@ -1,5 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash
set -euo pipefail
export PROLE_HOME=$HOME/dev/prole
PROLE_PASSWD=`cat $PROLE_HOME/prole-db/.prole_user_password | base64 -d`
k3d registry create prole-data-registry --port 5000 --timestamps --verbose

1
k8s/prole/kustomization.yaml
View File

@ -14,6 +14,7 @@ resources:
- openbao-service.yaml
- prometheus-configmap.yaml
- prometheus-deployment.yaml
- prometheus-pvc.yaml
- prometheus-service.yaml
- grafana-configmap.yaml
- grafana-deployment.yaml

17
k8s/prole/prole-db.yaml
View File

@ -62,6 +62,23 @@ spec:
walStorage:
size: 1Gi
backup:
barmanObjectStore:
destinationPath: s3://prole-db-backups/
endpointURL: http://garage.default.svc.cluster.local:3900
s3Credentials:
accessKeyId:
name: prole-db-barman-s3
key: ACCESS_KEY_ID
secretAccessKey:
name: prole-db-barman-s3
key: SECRET_ACCESS_KEY
wal:
compression: gzip
data:
compression: gzip
retentionPolicy: 30d
monitoring:
enablePodMonitor: true

26
k8s/prole/prometheus-configmap.yaml
View File

@ -23,6 +23,26 @@ data:
target_label: __address__
regex: (.*?):\d+;(\d+)
replacement: $1:$2
- job_name: 'cnpg-exporter'
static_configs:
- targets: ['prole-db-001.default.svc.cluster.local:9187']
- job_name: 'cnpg-metrics'
kubernetes_sd_configs:
- role: pod
relabel_configs:
- action: keep
source_labels: [__meta_kubernetes_pod_label_cnpg_io_cluster]
regex: .+
- action: keep
source_labels: [__meta_kubernetes_pod_phase]
regex: Running
- action: replace
source_labels: [__meta_kubernetes_pod_ip]
target_label: __address__
replacement: $1:9187
- action: replace
source_labels: [__meta_kubernetes_namespace]
target_label: namespace
- action: replace
source_labels: [__meta_kubernetes_pod_name]
target_label: pod
- action: replace
source_labels: [__meta_kubernetes_pod_label_cnpg_io_cluster]
target_label: cluster

3
k8s/prole/prometheus-deployment.yaml
View File

@ -34,4 +34,5 @@ spec:
configMap:
name: prometheus-config
- name: data
emptyDir: {}
persistentVolumeClaim:
claimName: prometheus-data

10
k8s/prole/prometheus-pvc.yaml Normal file
View File

@ -0,0 +1,10 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: prometheus-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi

2
prole-db/.prole_user_password
View File

@ -1 +1 @@
RnIwYjB6emcwYmwhbiEK
${OPENBAO:kv/prole/<namespace>/db#password}

4
prole-db/cr_prole_mssql_fdw.sql
View File

@ -6,7 +6,7 @@ CREATE SERVER prole_mssql_db
OPTIONS (servername '127.0.0.1', port '1433', database 'prole', tds_version '7.4');
CREATE USER MAPPING FOR postgres SERVER prole_mssql_db OPTIONS (username 'sa', password 'QGFYaC3!pqWy');
CREATE USER MAPPING FOR postgres SERVER prole_mssql_db OPTIONS (username 'sa', password '${OPENBAO:kv/prole/<namespace>/mssql#sa_password}');
IMPORT FOREIGN SCHEMA dbo FROM SERVER prole_mssql_db INTO public OPTIONS (import_default 'true');
@ -36,4 +36,4 @@ CREATE FOREIGN TABLE prole_mssql_db_wells (
SystemEntryDate TIMESTAMP
)
SERVER prole_mssql_db
OPTIONS (schema_name 'dbo', table_name 'wells', row_estimate_method 'showplan_all');
OPTIONS (schema_name 'dbo', table_name 'wells', row_estimate_method 'showplan_all');

27
prole-db/supabase.md
View File

@ -4,14 +4,9 @@ This document explains how Supabase is deployed alongside the Prole CloudNative-
## What is deployed
Supabase is deployed in the same Kubernetes namespace as the Prole CNPG cluster using the following resources:
Supabase can be deployed alongside the Prole CNPG cluster. Currently, the base `supabase/supabase` image is unavailable on Docker Hub, so the default deployment manifests have been removed.
- Supabase Deployment: `k8s/prole/supabase-deployment.yaml`
- Supabase Service (HTTPS): `k8s/prole/supabase-service.yaml`
- Supabase ConfigMap: `k8s/prole/supabase-configmap.yaml`
- Postgres Service (CNPG primary): `k8s/prole/prole-db-postgres-service.yaml`
The Supabase Deployment uses environment variables from the ConfigMap and the `prole-db-user` Kubernetes Secret to connect to Postgres.
If you wish to deploy Supabase, you can use the `./supabase/deploy.sh` script which uses the official Supabase repository and its Docker Compose setup to generate Kubernetes manifests.
## Postgres connectivity
@ -29,10 +24,7 @@ The database credentials come from the existing secret:
Supabase is exposed via a ClusterIP service on port 443. The deployment assumes the Supabase container listens on HTTPS at port 443.
If your Supabase image expects a different port, update both:
- `k8s/prole/supabase-deployment.yaml` (container port)
- `k8s/prole/supabase-service.yaml` (service port/targetPort)
If your Supabase image expects a different port, update your custom manifests.
## Optional feature toggle
@ -65,15 +57,12 @@ kubectl get svc prole-db-postgres
5) Check Supabase logs if needed:
```bash
kubectl logs deploy/supabase
kubectl logs -n supabase -l app=supabase
```
## Updating the Postgres target
If you need Supabase to connect to a different CNPG cluster name or namespace, update the following values:
- `k8s/prole/supabase-configmap.yaml` (`POSTGRES_HOST`, `POSTGRES_PORT`, `POSTGRES_DB`)
- `k8s/prole/prole-db-postgres-service.yaml` (service name and selectors)
If you need Supabase to connect to a different CNPG cluster name or namespace, update your configuration.
## Troubleshooting
@ -88,9 +77,7 @@ If you need Supabase to connect to a different CNPG cluster name or namespace, u
## Files referenced
- `k8s/prole/supabase-configmap.yaml`
- `k8s/prole/supabase-deployment.yaml`
- `k8s/prole/supabase-service.yaml`
- `k8s/prole/prole-db-postgres-service.yaml`
- `prole-db-postgres-service.yaml`
- `k8s/prole/kustomization.yaml`
- `install.py` (optional feature toggle persistence)
- `./supabase/deploy.sh`

6
prole-mssql-db/docker-root-mssql-server.sh
View File

@ -1,6 +1,8 @@
# export MSSQL_SA_PASSWORD=$(openssl rand -base64 32)
# echo "$MSSQL_SA_PASSWORD" > mssql-password.txt
export MSSQL_SA_PASSWORD=$(cat mssql-password.txt)
if [[ -z "${MSSQL_SA_PASSWORD:-}" ]]; then
echo "ERROR: MSSQL_SA_PASSWORD is required. Store it in OpenBao (kv/prole/<namespace>/mssql#sa_password) or export it." >&2
exit 1
fi
DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run --rm -it \
--hostname=prole-mssql-db001 -p 1433:1433 \

6
prole-mssql-db/docker-root-prole-mssql-db.sh
View File

@ -1,6 +1,8 @@
# export MSSQL_SA_PASSWORD=$(openssl rand -base64 32)
# echo "$MSSQL_SA_PASSWORD" > mssql-password.txt
export MSSQL_SA_PASSWORD=$(cat mssql-password.txt)
if [[ -z "${MSSQL_SA_PASSWORD:-}" ]]; then
echo "ERROR: MSSQL_SA_PASSWORD is required. Store it in OpenBao (kv/prole/<namespace>/mssql#sa_password) or export it." >&2
exit 1
fi
DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run --rm -it \
--hostname=prole-mssql-db001 -p 1433:1433 \

7
prole-mssql-db/docker-run-mssql-server.sh
View File

@ -1,7 +1,12 @@
# softwareupdate --install-rosetta
# docker pull mcr.microsoft.com/mssql/server:2022-latest
docker run -e "ACCEPT_EULA=Y" -e "MSSQL_SA_PASSWORD=QGFYaC3!pqWy" \
if [[ -z "${MSSQL_SA_PASSWORD:-}" ]]; then
echo "ERROR: MSSQL_SA_PASSWORD is required. Store it in OpenBao (kv/prole/<namespace>/mssql#sa_password) or export it." >&2
exit 1
fi
docker run -e "ACCEPT_EULA=Y" -e "MSSQL_SA_PASSWORD=$MSSQL_SA_PASSWORD" \
-p 1433:1433 --name mssql-server --platform linux/amd64 \
-v /Users/chrisfu/dev/prole/mssql/data:/var/opt/mssql/data \
-d mcr.microsoft.com/mssql/server:2022-latest

6
prole-mssql-db/docker-run-prole-mssql-db.sh
View File

@ -1,6 +1,8 @@
# export MSSQL_SA_PASSWORD=$(openssl rand -base64 32)
# echo "$MSSQL_SA_PASSWORD" > mssql-password.txt
export MSSQL_SA_PASSWORD=$(cat mssql-password.txt)
if [[ -z "${MSSQL_SA_PASSWORD:-}" ]]; then
echo "ERROR: MSSQL_SA_PASSWORD is required. Store it in OpenBao (kv/prole/<namespace>/mssql#sa_password) or export it." >&2
exit 1
fi
DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run --rm -it \
--hostname=prole-mssql-db-1 \

2
prole-mssql-db/prole-mssql-db-deployment.yaml
View File

@ -24,7 +24,7 @@ spec:
- name: ACCEPT_EULA
value: 'Y'
- name: MSSQL_SA_PASSWORD
value: 'QGFYaC3!pqWy'
value: '${OPENBAO:kv/prole/<namespace>/mssql#sa_password}'
volumes:
- name: prole-db-mssql-data
persistentVolumeClaim:

2
prole-mssql-db/prole-mssql-db-nfs-deployment.yaml
View File

@ -24,7 +24,7 @@ spec:
- name: ACCEPT_EULA
value: 'Y'
- name: MSSQL_SA_PASSWORD
value: 'QGFYaC3!pqWy'
value: '${OPENBAO:kv/prole/<namespace>/mssql#sa_password}'
volumes:
- name: prole-db-mssql-data
persistentVolumeClaim:

BIN
prole-tools-app/.DS_Store vendored
View File

Binary file not shown.

4
prole/tests/test_init_authority.sh
View File

@ -26,8 +26,8 @@ mock_tool() {
# LOG THE CALL
echo "Mocked $1 called with \$@" >> "$TMP_DIR/mock_calls.log"
case "\$1" in
*v1/kv/data/prole/admin)
echo '{"data":{"data":{"root_password":"root", "admin_public_key_b64":"LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF6S3Z4CkNvS0tLS0tRU5EIFBVQkxJQyBLRVktLS0tLQo="}}}'
*v1/kv/data/prole/test-namespace/admin)
echo '{"data":{"data":{"root_password":"REDACTED", "admin_public_key_b64":"LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF6S3Z4CkNvS0tLS0tRU5EIFBVQkxJQyBLRVktLS0tLQo="}}}'
;;
*)
if [[ "$1" == "base64" ]]; then

5
scripts/docker-root-prole-db.sh
View File

@ -2,7 +2,10 @@
PROLE_HOME=~/dev/prole
export POSTGRES_PASSWORD=$(cat $PROLE_HOME/postgres-password.txt)
if [[ -z "${POSTGRES_PASSWORD:-}" ]]; then
echo "ERROR: POSTGRES_PASSWORD is required. Store it in OpenBao (kv/prole/<namespace>/db#password) or export it." >&2
exit 1
fi
PG_VERSION=$(cat "$PROLE_HOME/conf/postgresql/.version" 2>/dev/null | tr -d '[:space:]')
RELEASE=$(cat "$PROLE_HOME/prole-db/.version" 2>/dev/null | tr -d '[:space:]')

6
scripts/docker-run-prole-db.sh
View File

@ -1,7 +1,9 @@
set -eux
# export POSTGRES_PASSWORD=$(openssl rand -base64 32)
# echo "$POSTGRES_PASSWORD" > postgres-password.txt
export POSTGRES_PASSWORD=$(cat postgres-password.txt)
if [[ -z "${POSTGRES_PASSWORD:-}" ]]; then
echo "ERROR: POSTGRES_PASSWORD is required. Store it in OpenBao (kv/prole/<namespace>/db#password) or export it." >&2
exit 1
fi
PROLE_HOME=~/dev/prole
PG_VERSION=$(cat "$PROLE_HOME/conf/postgresql/.version" 2>/dev/null | tr -d '[:space:]')

27
tests/silent_install_test.sh
View File

@ -194,6 +194,7 @@ python3 - "$CFG_PATH" "$ROOT_DIR/prole/conf/prole-db/prole.cfg" <<'PY'
import re
import sys
from pathlib import Path
import configparser
source = Path(sys.argv[1]).expanduser()
target = Path(sys.argv[2]).expanduser()
@ -201,13 +202,25 @@ target.parent.mkdir(parents=True, exist_ok=True)
text = source.read_text()
cfg = configparser.ConfigParser(interpolation=None)
cfg.optionxform = str
cfg.read(source)
namespace = (
cfg.get("Global", "NAMESPACE", fallback="")
or cfg.get("Inputs", "env_setup.NAMESPACE", fallback="")
or "default"
).strip() or "default"
def bao(leaf: str, key: str) -> str:
return f"${{OPENBAO:kv/prole/{namespace}/{leaf}#{key}}}"
secret_map = {
"init_password.db_password": "${OPENBAO:prole/db_password}",
"init_password.db_password_confirm": "${OPENBAO:prole/db_password}",
"kerberos_config.password": "${OPENBAO:kerberos/password}",
"DB_PASSWORD": "${OPENBAO:prole/db_password}",
"PASSWORD": "${OPENBAO:kerberos/password}",
"GRAFANA_ADMIN_PASSWORD": "${OPENBAO:monitoring/grafana_admin_password}",
"init_password.db_password": bao("db", "password"),
"init_password.db_password_confirm": bao("db", "password"),
"kerberos_config.password": bao("kerberos", "password"),
"DB_PASSWORD": bao("db", "password"),
"PASSWORD": bao("kerberos", "password"),
"GRAFANA_ADMIN_PASSWORD": bao("monitoring", "grafana_admin_password"),
}
pattern = re.compile(r"^(?P<lead>\s*)(?P<key>[^=]+?)(?P<pre>\s*)=(?P<post>\s*).*$")
@ -230,7 +243,7 @@ for line in text.splitlines():
value = secret_map[key]
out_lines.append(f"{lead}{key}{pre}={post}{value}")
continue
if re.search(r"(?i)(password|token|secret)", key):
if re.search(r"(?i)(?:^|[_.])(password|token|secret)$", key):
out_lines.append(f"{lead}{key}{pre}={post}${{OPENBAO:REDACTED}}")
continue
out_lines.append(line)