mirror of
https://github.com/dredx/prole.git
synced 2026-09-24 11:34:31 +00:00
Update Prole-DB and improve Supabase integration
- Bumped Prole-DB image version to 17.7-053 in scripts, Dockerfile, and manifests. - Replaced `prole-scan` with `prole-agent` throughout scripts and tests. - Refined Kubernetes setup for Supabase to use namespace 'supabase'. - Introduced conversion of Supabase Docker Compose to Kubernetes manifests with `kompose`. - Added support for Kerberos toggle via environment variables in `init_kerberos.sh`. - Improved error handling and logging in scripts for better maintainability.
This commit is contained in:
chrisfu
parent
9c1f58caf4
commit
fff18fdbe4
@ -14,7 +14,7 @@ The Prole Installer package includes several embedded resources that must be acc
|
||||
- **proleIconblueprint.png** (1.6 MB) - Blueprint icon variant
|
||||
|
||||
### 2. Binary Executables
|
||||
- **prole-net/prole-scan** (6.8 MB) - Network scanner
|
||||
- **prole-net/prole-agent** (6.8 MB) - Network scanner
|
||||
- Universal binary (x86_64 + arm64)
|
||||
- Used by network scan screen
|
||||
- Detects Kerberos, Active Directory, etc.
|
||||
@ -54,7 +54,7 @@ if bg_path.exists():
|
||||
|
||||
**Binary Execution:**
|
||||
```python
|
||||
scan_binary = get_resource_path("prole-net/prole-scan")
|
||||
scan_binary = get_resource_path("prole-net/prole-agent")
|
||||
if scan_binary.exists():
|
||||
process = subprocess.Popen([str(scan_binary)], ...)
|
||||
```
|
||||
@ -86,7 +86,7 @@ datas = [
|
||||
**Binaries:**
|
||||
```python
|
||||
binaries = [
|
||||
('prole-net/prole-scan', 'prole-net'),
|
||||
('prole-net/prole-agent', 'prole-net'),
|
||||
]
|
||||
```
|
||||
|
||||
@ -98,7 +98,7 @@ Total embedded resources: ~30-35 MB
|
||||
|
||||
Breakdown:
|
||||
- Images: ~11 MB
|
||||
- prole-scan: 6.8 MB
|
||||
- prole-agent: 6.8 MB
|
||||
- Prole Tools.app: ~12 MB
|
||||
- Other resources: ~5-10 MB
|
||||
|
||||
@ -108,10 +108,10 @@ Final installer bundle: ~50-100 MB (includes Python runtime)
|
||||
|
||||
### Network Scan Screen
|
||||
|
||||
The network scan screen uses `prole-scan` to detect services:
|
||||
The network scan screen uses `prole-agent` to detect services:
|
||||
|
||||
```python
|
||||
scan_binary = get_resource_path("prole-net/prole-scan")
|
||||
scan_binary = get_resource_path("prole-net/prole-agent")
|
||||
process = subprocess.Popen([str(scan_binary)],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
@ -140,7 +140,7 @@ python3 test_embedded_resources.sh
|
||||
|
||||
This tests:
|
||||
- ✓ All image files exist
|
||||
- ✓ prole-scan binary exists and is executable
|
||||
- ✓ prole-agent binary exists and is executable
|
||||
- ✓ Prole Tools.app bundle exists
|
||||
- ✓ Spec file includes all resources
|
||||
- ✓ Resource sizes
|
||||
@ -163,18 +163,18 @@ ls -la "/tmp/_MEI*/prole-app/dist/Prole Tools.app"
|
||||
|
||||
### Binary Not Found Error
|
||||
|
||||
**Error:** `Scan binary not found at /var/folders/.../prole-net/prole-scan`
|
||||
**Error:** `Scan binary not found at /var/folders/.../prole-net/prole-agent`
|
||||
|
||||
**Cause:** Binary not included in package or path not using `get_resource_path()`
|
||||
|
||||
**Solution:**
|
||||
1. Verify spec includes binary: `grep prole-scan installer.spec`
|
||||
2. Check code uses `get_resource_path()`: `grep "get_resource_path.*prole-scan" install.py`
|
||||
1. Verify spec includes binary: `grep prole-agent installer.spec`
|
||||
2. Check code uses `get_resource_path()`: `grep "get_resource_path.*prole-agent" install.py`
|
||||
3. Rebuild: `make clean && make package`
|
||||
|
||||
### Binary Not Executable
|
||||
|
||||
**Error:** `Permission denied` when running prole-scan
|
||||
**Error:** `Permission denied` when running prole-agent
|
||||
|
||||
**Cause:** Binary permissions not preserved in package
|
||||
|
||||
@ -182,10 +182,10 @@ ls -la "/tmp/_MEI*/prole-app/dist/Prole Tools.app"
|
||||
Ensure binary is in `binaries` list (not `datas`) in spec file:
|
||||
```python
|
||||
binaries = [
|
||||
('prole-net/prole-scan', 'prole-net'), # Correct - preserves +x
|
||||
('prole-net/prole-agent', 'prole-net'), # Correct - preserves +x
|
||||
]
|
||||
# NOT in datas:
|
||||
# datas = [('prole-net/prole-scan', 'prole-net')] # Wrong - loses +x
|
||||
# datas = [('prole-net/prole-agent', 'prole-net')] # Wrong - loses +x
|
||||
```
|
||||
|
||||
### App Bundle Not Found
|
||||
@ -217,7 +217,7 @@ binaries = [
|
||||
3. **Exclude debug symbols:**
|
||||
Strip binaries before packaging:
|
||||
```bash
|
||||
strip prole-net/prole-scan
|
||||
strip prole-net/prole-agent
|
||||
```
|
||||
|
||||
## Build Process
|
||||
@ -250,7 +250,7 @@ When the packaged installer runs:
|
||||
Embedded binaries should be code signed:
|
||||
|
||||
```bash
|
||||
codesign --force --sign "Developer ID Application: Your Name" prole-net/prole-scan
|
||||
codesign --force --sign "Developer ID Application: Your Name" prole-net/prole-agent
|
||||
```
|
||||
|
||||
Then build the installer - the signed binary will be included.
|
||||
@ -260,8 +260,8 @@ Then build the installer - the signed binary will be included.
|
||||
Users can verify embedded binaries:
|
||||
|
||||
```bash
|
||||
# Check signature of prole-scan after extraction
|
||||
codesign --verify --verbose /tmp/_MEI*/prole-net/prole-scan
|
||||
# Check signature of prole-agent after extraction
|
||||
codesign --verify --verbose /tmp/_MEI*/prole-net/prole-agent
|
||||
|
||||
# Check installer bundle signature
|
||||
codesign --verify --verbose "dist/Prole Installer.app"
|
||||
|
||||
@ -50,7 +50,7 @@ subprocess.Popen(['docker', 'build', '-t', 'prole-db:TAG', '.'], cwd=build_dir)
|
||||
|
||||
**Purpose:** Writable working directory for network scan operations
|
||||
|
||||
**Why:** The `prole-scan` binary may need to write output files, cache data, or store temporary results. Running from a read-only directory causes failures.
|
||||
**Why:** The `prole-agent` binary may need to write output files, cache data, or store temporary results. Running from a read-only directory causes failures.
|
||||
|
||||
**Usage:**
|
||||
```python
|
||||
@ -65,7 +65,7 @@ subprocess.Popen([str(scan_binary)], cwd=str(scan_dir))
|
||||
**Contents:**
|
||||
- Network scan results (temporary)
|
||||
- Ollama API interaction cache
|
||||
- Any intermediate files created by prole-scan
|
||||
- Any intermediate files created by prole-agent
|
||||
|
||||
**Size:** Varies, typically < 1 MB
|
||||
|
||||
|
||||
@ -97,7 +97,7 @@ A Python script (`scripts/generate_spec.py`) creates the PyInstaller specificati
|
||||
- `prole-app/dist/Prole Tools.app` - Pre-built Prole Tools application bundle (entire .app)
|
||||
|
||||
**Included Binaries:**
|
||||
- `prole-net/prole-scan` - Network scanner binary (universal: x86_64 + arm64, 6.8 MB)
|
||||
- `prole-net/prole-agent` - Network scanner binary (universal: x86_64 + arm64, 6.8 MB)
|
||||
|
||||
**Resource Path Resolution:**
|
||||
The installer uses a `get_resource_path()` helper function that automatically resolves paths correctly whether running from source or from a PyInstaller bundle:
|
||||
|
||||
@ -103,7 +103,7 @@ ensure_prole_stack_resources() {
|
||||
dir="$SCRIPT_DIR/../k8s/prole"
|
||||
for file in "$dir"/*.yaml; do
|
||||
case "$(basename "$file")" in
|
||||
prole-db.yaml|kustomization.yaml)
|
||||
prole-db.yaml|kustomization.yaml|supabase-*.yaml)
|
||||
continue
|
||||
;;
|
||||
esac
|
||||
@ -115,7 +115,7 @@ ensure_prole_stack_resources() {
|
||||
dir="$SCRIPT_DIR/../k8s/prole"
|
||||
for file in "$dir"/*.yaml; do
|
||||
case "$(basename "$file")" in
|
||||
kustomization.yaml)
|
||||
kustomization.yaml|supabase-*.yaml)
|
||||
continue
|
||||
;;
|
||||
esac
|
||||
@ -147,21 +147,46 @@ wait_for_cnpg_pods() {
|
||||
local start_time
|
||||
start_time=$(date +%s)
|
||||
|
||||
while true; do
|
||||
if kubectl -n "$NAMESPACE" get cluster "$CNPG_CLUSTER_NAME" >/dev/null 2>&1; then
|
||||
if kubectl -n "$NAMESPACE" get pods -l "cnpg.io/cluster=$CNPG_CLUSTER_NAME" --no-headers 2>/dev/null | grep -q .; then
|
||||
if kubectl -n "$NAMESPACE" wait --for=condition=Ready pod -l "cnpg.io/cluster=$CNPG_CLUSTER_NAME" --timeout=60s >/dev/null 2>&1; then
|
||||
return 0
|
||||
local target_pods="${CNPG_TARGET_PODS:-}"
|
||||
if [[ -z "$target_pods" ]]; then
|
||||
target_pods=$(kubectl -n "$NAMESPACE" get cluster "$CNPG_CLUSTER_NAME" -o jsonpath='{.spec.instances}' 2>/dev/null || true)
|
||||
fi
|
||||
if [[ -z "$target_pods" ]]; then
|
||||
target_pods=3
|
||||
fi
|
||||
|
||||
echo "Waiting for $target_pods CNPG pods for cluster '$CNPG_CLUSTER_NAME' in namespace '$NAMESPACE' to be Running..."
|
||||
while true; do
|
||||
local pods
|
||||
pods=$(kubectl -n "$NAMESPACE" get pods -l "cnpg.io/cluster=$CNPG_CLUSTER_NAME" --no-headers 2>/dev/null || true)
|
||||
|
||||
if [[ -n "$pods" ]]; then
|
||||
# Check for Error or CrashLoopBackOff
|
||||
if echo "$pods" | grep -E "Error|CrashLoopBackOff" >/dev/null; then
|
||||
echo "ERROR: Some CNPG pods are in Error or CrashLoopBackOff state:" >&2
|
||||
echo "$pods" | grep -E "Error|CrashLoopBackOff" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Count Running pods by name (exclude initdb)
|
||||
local running_pods
|
||||
running_pods=$(kubectl -n "$NAMESPACE" get pods --no-headers 2>/dev/null \
|
||||
| grep "^${CNPG_CLUSTER_NAME}-" \
|
||||
| grep -v "initdb" \
|
||||
| grep "Running" \
|
||||
| wc -l | xargs)
|
||||
|
||||
if [[ "$running_pods" -ge "$target_pods" ]]; then
|
||||
echo "All $running_pods/$target_pods pods are Running."
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
if (( $(date +%s) - start_time > timeout )); then
|
||||
echo "ERROR: Timed out waiting for CNPG pods for cluster '$CNPG_CLUSTER_NAME' in namespace '$NAMESPACE'." >&2
|
||||
echo "ERROR: Timed out waiting for $target_pods CNPG pods for cluster '$CNPG_CLUSTER_NAME' in namespace '$NAMESPACE'." >&2
|
||||
return 1
|
||||
fi
|
||||
echo "Waiting for CNPG pods for cluster '$CNPG_CLUSTER_NAME'..."
|
||||
sleep 3
|
||||
sleep 5
|
||||
done
|
||||
}
|
||||
|
||||
@ -299,6 +324,9 @@ initialize() {
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Support both names for the toggle from prole.cfg
|
||||
local kerberos_enabled="${KERBEROS_ENABLED:-${ENABLED:-false}}"
|
||||
if [[ "$kerberos_enabled" == "true" || "$kerberos_enabled" == "True" || "$kerberos_enabled" == "1" ]]; then
|
||||
if [[ -x "$SCRIPT_DIR/init_kerberos.sh" ]]; then
|
||||
echo "Configuring Kerberos for CNPG pods ..."
|
||||
if ! "$SCRIPT_DIR/init_kerberos.sh" initialize; then
|
||||
@ -307,6 +335,9 @@ initialize() {
|
||||
else
|
||||
echo "WARN: init_kerberos.sh not found; skipping Kerberos configuration."
|
||||
fi
|
||||
else
|
||||
echo "Kerberos is disabled; skipping Kerberos configuration."
|
||||
fi
|
||||
|
||||
# Ensure port-forward is running for Postgres (local access)
|
||||
echo "Ensuring port-forward for Postgres is active ..."
|
||||
|
||||
@ -215,13 +215,28 @@ init_layout() {
|
||||
assign_count=$((assign_count + 1))
|
||||
done
|
||||
|
||||
# Get current version for apply
|
||||
local version
|
||||
version=$(echo "$layout" | grep "Version:" | awk '{print $2}' || echo "0")
|
||||
if [[ -z "$version" ]]; then version=0; fi
|
||||
local next_version=$((version + 1))
|
||||
echo "Applying Garage layout (version $next_version) ..."
|
||||
kubectl exec -n "$NAMESPACE" "$pod" -- /garage layout apply --version "$next_version" || true
|
||||
# Re-fetch layout and apply staged changes using the current layout version
|
||||
local layout_after version apply_ok
|
||||
layout_after=$(kubectl exec -n "$NAMESPACE" "$pod" -- /garage layout show 2>/dev/null || true)
|
||||
version=$(echo "$layout_after" | awk -F: '/[Vv]ersion/ {gsub(/ /,"",$2); print $2; exit}' || true)
|
||||
if echo "$layout_after" | grep -qi "staged"; then
|
||||
if [[ -n "$version" ]]; then
|
||||
echo "Applying Garage layout (version $version) ..."
|
||||
if kubectl exec -n "$NAMESPACE" "$pod" -- /garage layout apply --version "$version"; then
|
||||
apply_ok=1
|
||||
else
|
||||
apply_ok=0
|
||||
fi
|
||||
else
|
||||
apply_ok=0
|
||||
fi
|
||||
if [[ "${apply_ok:-0}" -eq 0 ]]; then
|
||||
echo "WARN: layout apply with explicit version failed; retrying without version..."
|
||||
kubectl exec -n "$NAMESPACE" "$pod" -- /garage layout apply || true
|
||||
fi
|
||||
else
|
||||
echo "No staged layout changes detected; skipping layout apply."
|
||||
fi
|
||||
else
|
||||
echo "Garage layout already assigned for node $short_id."
|
||||
fi
|
||||
|
||||
@ -57,6 +57,15 @@ ensure_namespace() {
|
||||
}
|
||||
|
||||
resolve_krb5_defaults() {
|
||||
# If 'dog' KDC is available in 'authority' namespace, use it as default
|
||||
if kubectl get pod -n "authority" -l app=dog >/dev/null 2>&1; then
|
||||
KRB5_KDC="dog.authority.svc.cluster.local"
|
||||
KRB5_ADMIN="dog.authority.svc.cluster.local"
|
||||
if [[ -z "${KRB5_REALM}" ]]; then
|
||||
KRB5_REALM="PROLE.ORG"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ -z "${KRB5_REALM}" && -n "${REALM:-}" ]]; then
|
||||
KRB5_REALM="$REALM"
|
||||
fi
|
||||
@ -184,16 +193,29 @@ patch_cnpg_cluster_for_auth() {
|
||||
# Best-effort patch; schema may vary with CNPG version.
|
||||
# We remove krb_srvname as it is unrecognized in some PG 17 builds.
|
||||
# We revert certificates to default to avoid operator TLS issues.
|
||||
if ! kubectl -n "$NAMESPACE" patch cluster "$CNPG_CLUSTER_NAME" --type merge -p "{
|
||||
\"spec\": {
|
||||
\"postgresql\": {
|
||||
\"parameters\": {\"krb_srvname\": null},
|
||||
\"pg_hba\": [
|
||||
|
||||
local kerberos_enabled="${KERBEROS_ENABLED:-${ENABLED:-false}}"
|
||||
local pg_hba
|
||||
if [[ "$kerberos_enabled" == "true" || "$kerberos_enabled" == "True" || "$kerberos_enabled" == "1" ]]; then
|
||||
pg_hba="[
|
||||
\"local all postgres trust\",
|
||||
\"host all postgres all scram-sha-256\",
|
||||
\"host all all all gss include_realm=1 krb_realm=$KRB5_REALM\",
|
||||
\"host all all all scram-sha-256\"
|
||||
]
|
||||
]"
|
||||
else
|
||||
pg_hba="[
|
||||
\"local all postgres trust\",
|
||||
\"host all postgres all scram-sha-256\",
|
||||
\"host all all all scram-sha-256\"
|
||||
]"
|
||||
fi
|
||||
|
||||
if ! kubectl -n "$NAMESPACE" patch cluster "$CNPG_CLUSTER_NAME" --type merge -p "{
|
||||
\"spec\": {
|
||||
\"postgresql\": {
|
||||
\"parameters\": {\"krb_srvname\": null},
|
||||
\"pg_hba\": $pg_hba
|
||||
},
|
||||
\"certificates\": {
|
||||
\"serverCASecret\": null,
|
||||
@ -323,6 +345,15 @@ initialize() {
|
||||
ensure_namespace
|
||||
resolve_krb5_defaults
|
||||
|
||||
# Support both names for the toggle from prole.cfg
|
||||
local enabled="${KERBEROS_ENABLED:-${ENABLED:-false}}"
|
||||
if [[ "$enabled" == "false" || "$enabled" == "0" || "$enabled" == "False" ]]; then
|
||||
log "Kerberos is disabled (KERBEROS_ENABLED=$enabled). Skipping initialization."
|
||||
# Cleanup any existing Kerberos config if it was previously enabled
|
||||
kubectl -n "$NAMESPACE" delete configmap prole-krb5-conf --ignore-not-found
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [[ -z "${KRB5_REALM}" || -z "${KRB5_KDC}" ]]; then
|
||||
err "ERROR: KRB5_REALM and KRB5_KDC must be set for Kerberos initialization."
|
||||
err " Set KRB5_REALM/KRB5_KDC or REALM/DOMAIN in env or config."
|
||||
|
||||
@ -81,7 +81,7 @@ detect_image() {
|
||||
if [[ -n "$release_file" && -f "$release_file" ]]; then
|
||||
release=$(cat "$release_file" | tr -d '[:space:]')
|
||||
else
|
||||
release="41"
|
||||
release="43"
|
||||
fi
|
||||
|
||||
if [[ "$release" =~ ^[0-9]+$ ]]; then
|
||||
@ -142,6 +142,13 @@ run_test() {
|
||||
ensure_tools
|
||||
ensure_namespace
|
||||
|
||||
# Support both names for the toggle from prole.cfg
|
||||
local enabled="${KERBEROS_ENABLED:-${ENABLED:-false}}"
|
||||
if [[ "$enabled" == "false" || "$enabled" == "0" || "$enabled" == "False" ]]; then
|
||||
echo "Kerberos is disabled (KERBEROS_ENABLED=$enabled). Skipping test."
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [[ -z "$KRB5_REALM" || -z "$KRB5_USER" || -z "$KRB5_PASSWORD" || -z "$KRB5_KDC" ]]; then
|
||||
echo "ERROR: Missing Kerberos configuration. Ensure KRB5_REALM, KRB5_KDC, KRB5_USER, KRB5_PASSWORD are set." >&2
|
||||
exit 1
|
||||
|
||||
@ -68,7 +68,7 @@ get_latest_image() {
|
||||
if [[ -f "$release_file" ]]; then
|
||||
release=$(cat "$release_file" | tr -d '[:space:]')
|
||||
else
|
||||
release="41"
|
||||
release="43"
|
||||
fi
|
||||
|
||||
if [[ "$release" =~ ^[0-9]+$ ]]; then
|
||||
@ -78,6 +78,24 @@ get_latest_image() {
|
||||
echo "prole-db:${pg_version}-${release}"
|
||||
}
|
||||
|
||||
sync_manifest_image() {
|
||||
local image="$1"
|
||||
local manifest_dir
|
||||
manifest_dir=$(dirname "$CNPG_MANIFEST")
|
||||
local files=("$CNPG_MANIFEST")
|
||||
if [[ -f "$manifest_dir/prole-db-recovery.yaml.tpl" ]]; then
|
||||
files+=("$manifest_dir/prole-db-recovery.yaml.tpl")
|
||||
fi
|
||||
local f tmp
|
||||
for f in "${files[@]}"; do
|
||||
if [[ -f "$f" ]] && grep -qE '^[[:space:]]*imageName:' "$f"; then
|
||||
tmp=$(mktemp)
|
||||
sed -E "s|^([[:space:]]*imageName:).*|\\1 ${image}|" "$f" > "$tmp"
|
||||
mv "$tmp" "$f"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
start() {
|
||||
ensure_tools
|
||||
ensure_namespace
|
||||
@ -116,6 +134,7 @@ start() {
|
||||
# Compare latest image with deployed
|
||||
local latest_image
|
||||
latest_image=$(get_latest_image)
|
||||
sync_manifest_image "$latest_image"
|
||||
|
||||
local current_image
|
||||
current_image=$(kubectl get cluster "$CNPG_CLUSTER_NAME" -n "$NAMESPACE" -o jsonpath='{.spec.imageName}' 2>/dev/null || echo "")
|
||||
@ -190,6 +209,7 @@ deploy() {
|
||||
else
|
||||
image="prole-db:$VERSION"
|
||||
fi
|
||||
sync_manifest_image "$image"
|
||||
|
||||
echo "Deploying $image to cluster $CNPG_CLUSTER_NAME..."
|
||||
|
||||
|
||||
@ -4,9 +4,10 @@ set -euo pipefail
|
||||
|
||||
# init_supabase.sh
|
||||
# Purpose:
|
||||
# - Deploy Supabase in Kubernetes and connect it to the CNPG Postgres cluster.
|
||||
# - Deploy Supabase stack in Kubernetes using the official supabase/docker compose
|
||||
# - Rewire Supabase to use the CNPG Postgres service
|
||||
# - Disable image pulls (imagePullPolicy: Never) for now
|
||||
|
||||
# Initialize SCRIPT_DIR
|
||||
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||
|
||||
# Load environment and config via prole_cfg.sh
|
||||
@ -16,103 +17,389 @@ source "$SCRIPT_DIR/prole_cfg.sh"
|
||||
ACTION=${1:-}
|
||||
|
||||
CNPG_CLUSTER_NAME=${CNPG_CLUSTER_NAME:-prole-db}
|
||||
|
||||
# Support both PROLE_HOME/k8s and sibling k8s directory
|
||||
if [[ -d "$SCRIPT_DIR/../k8s/prole" ]]; then
|
||||
SUPABASE_MANIFEST_DIR="$SCRIPT_DIR/../k8s/prole"
|
||||
elif [[ -n "${PROLE_HOME:-}" && -d "$PROLE_HOME/k8s/prole" ]]; then
|
||||
SUPABASE_MANIFEST_DIR="$PROLE_HOME/k8s/prole"
|
||||
else
|
||||
SUPABASE_MANIFEST_DIR="$SCRIPT_DIR/../k8s/prole"
|
||||
fi
|
||||
|
||||
SUPABASE_FILES=(
|
||||
"$SUPABASE_MANIFEST_DIR/supabase-configmap.yaml"
|
||||
"$SUPABASE_MANIFEST_DIR/supabase-deployment.yaml"
|
||||
"$SUPABASE_MANIFEST_DIR/supabase-service.yaml"
|
||||
)
|
||||
SUPABASE_HOME=${SUPABASE_HOME:-}
|
||||
SUPABASE_USE_DEV_COMPOSE=${SUPABASE_USE_DEV_COMPOSE:-0}
|
||||
SUPABASE_IMAGE_PULL_POLICY=${SUPABASE_IMAGE_PULL_POLICY:-Never}
|
||||
SUPABASE_K8S_DIR=${SUPABASE_K8S_DIR:-${PROLE_HOME:-$SCRIPT_DIR/..}/build/supabase-k8s}
|
||||
SUPABASE_POSTGRES_HOST=${SUPABASE_POSTGRES_HOST:-db}
|
||||
SUPABASE_POSTGRES_DB=${SUPABASE_POSTGRES_DB:-postgres}
|
||||
SUPABASE_POSTGRES_PORT=${SUPABASE_POSTGRES_PORT:-5432}
|
||||
SUPABASE_APPLY_DB_MIGRATIONS=${SUPABASE_APPLY_DB_MIGRATIONS:-1}
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: $0 [start|stop|restart|status]
|
||||
|
||||
Actions:
|
||||
start Apply Supabase manifests
|
||||
stop Delete Supabase manifests
|
||||
start Convert Supabase docker-compose to K8s and apply in namespace
|
||||
stop Delete Supabase K8s resources from namespace
|
||||
restart Re-apply Supabase manifests
|
||||
status Show Supabase resources
|
||||
|
||||
Env overrides:
|
||||
SUPABASE_HOME, SUPABASE_USE_DEV_COMPOSE, SUPABASE_K8S_DIR
|
||||
SUPABASE_IMAGE_PULL_POLICY (default: Never)
|
||||
SUPABASE_POSTGRES_HOST (default: db)
|
||||
SUPABASE_POSTGRES_DB (default: postgres)
|
||||
SUPABASE_POSTGRES_PORT (default: 5432)
|
||||
SUPABASE_APPLY_DB_MIGRATIONS (default: 1)
|
||||
EOF
|
||||
exit 1
|
||||
}
|
||||
|
||||
log() { printf '%s\n' "$*"; }
|
||||
err() { printf '%s\n' "$*" >&2; }
|
||||
|
||||
ensure_tools() {
|
||||
for t in kubectl; do
|
||||
command -v "$t" >/dev/null || { echo "Missing required tool: $t" >&2; exit 1; }
|
||||
for t in kubectl kompose python3 sed awk base64; do
|
||||
command -v "$t" >/dev/null || { err "Missing required tool: $t"; exit 1; }
|
||||
done
|
||||
}
|
||||
|
||||
ensure_namespace() {
|
||||
if ! kubectl get namespace "$NAMESPACE" >/dev/null 2>&1; then
|
||||
echo "Creating namespace '$NAMESPACE' ..."
|
||||
kubectl create namespace "$NAMESPACE" >/dev/null 2>&1 || true
|
||||
if ! kubectl get namespace "supabase" >/dev/null 2>&1; then
|
||||
err "ERROR: namespace 'supabase' not found. Supabase must be deployed in its own namespace."
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
ensure_prereqs() {
|
||||
ensure_namespace
|
||||
if ! kubectl get cluster "$CNPG_CLUSTER_NAME" -n "$NAMESPACE" >/dev/null 2>&1; then
|
||||
echo "ERROR: CNPG cluster '$CNPG_CLUSTER_NAME' not found in namespace '$NAMESPACE'." >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! kubectl get secret prole-db-user -n "$NAMESPACE" >/dev/null 2>&1; then
|
||||
echo "ERROR: Secret 'prole-db-user' not found in namespace '$NAMESPACE'." >&2
|
||||
# We still want to ensure prole-db-superuser secret is in the CURRENT namespace (where prole-db is)
|
||||
# but supabase itself will be in 'supabase' namespace.
|
||||
# The issue description says: "connect supabase postgres network ports to our new namespace"
|
||||
# This implies we might need to create services in the CURRENT namespace that point to supabase.
|
||||
if ! kubectl get secret prole-db-superuser -n "$NAMESPACE" >/dev/null 2>&1; then
|
||||
err "ERROR: Secret 'prole-db-superuser' not found in namespace '$NAMESPACE'."
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
apply_manifests() {
|
||||
for f in "${SUPABASE_FILES[@]}"; do
|
||||
if [[ -f "$f" ]]; then
|
||||
kubectl apply -n "$NAMESPACE" -f "$f"
|
||||
detect_supabase_home() {
|
||||
if [[ -n "$SUPABASE_HOME" && -d "$SUPABASE_HOME" ]]; then
|
||||
echo "$SUPABASE_HOME"
|
||||
return 0
|
||||
fi
|
||||
if [[ -d "$HOME/prole/supabase" ]]; then
|
||||
echo "$HOME/prole/supabase"
|
||||
return 0
|
||||
fi
|
||||
if [[ -d "$HOME/dev/supabase" ]]; then
|
||||
echo "$HOME/dev/supabase"
|
||||
return 0
|
||||
fi
|
||||
if [[ -n "${PROLE_HOME:-}" && -d "$PROLE_HOME/../supabase" ]]; then
|
||||
echo "$PROLE_HOME/../supabase"
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
get_db_password() {
|
||||
kubectl -n "$NAMESPACE" get secret prole-db-superuser -o jsonpath='{.data.password}' 2>/dev/null | base64 -d
|
||||
}
|
||||
|
||||
set_env_kv() {
|
||||
local file="$1" key="$2" value="$3"
|
||||
# Ensure value is quoted if it contains spaces and is not already quoted
|
||||
if [[ "$value" == *" "* && ! "$value" =~ ^\".*\"$ && ! "$value" =~ ^\'.*\'$ ]]; then
|
||||
value="\"$value\""
|
||||
fi
|
||||
if grep -q "^${key}=" "$file" 2>/dev/null; then
|
||||
sed -i.bak "s|^${key}=.*|${key}=${value}|" "$file"
|
||||
else
|
||||
echo "ERROR: Missing manifest: $f" >&2
|
||||
printf "%s=%s\n" "$key" "$value" >> "$file"
|
||||
fi
|
||||
}
|
||||
|
||||
build_env_file() {
|
||||
local supa_home="$1"
|
||||
local docker_dir="$supa_home/docker"
|
||||
local env_base="$docker_dir/.env"
|
||||
local env_example="$docker_dir/.env.example"
|
||||
local env_out="$SUPABASE_K8S_DIR/.env"
|
||||
local db_pass
|
||||
db_pass=$(get_db_password || true)
|
||||
|
||||
mkdir -p "$SUPABASE_K8S_DIR"
|
||||
|
||||
if [[ -f "$env_base" ]]; then
|
||||
cp "$env_base" "$env_out"
|
||||
elif [[ -f "$env_example" ]]; then
|
||||
cp "$env_example" "$env_out"
|
||||
else
|
||||
err "ERROR: Supabase .env or .env.example not found in $docker_dir"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# If we used .env.example, generate required secrets
|
||||
if [[ ! -f "$env_base" ]]; then
|
||||
if [[ -x "$docker_dir/utils/generate-keys.sh" ]]; then
|
||||
log "Generating Supabase secrets from utils/generate-keys.sh ..."
|
||||
local gen_out
|
||||
gen_out=$(bash "$docker_dir/utils/generate-keys.sh" </dev/null || true)
|
||||
while IFS= read -r line; do
|
||||
[[ "$line" =~ ^[A-Z0-9_]+= ]] || continue
|
||||
local k="${line%%=*}"
|
||||
local v="${line#*=}"
|
||||
set_env_kv "$env_out" "$k" "$v"
|
||||
done <<< "$gen_out"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ -n "$db_pass" ]]; then
|
||||
set_env_kv "$env_out" "POSTGRES_PASSWORD" "$db_pass"
|
||||
fi
|
||||
set_env_kv "$env_out" "POSTGRES_HOST" "$SUPABASE_POSTGRES_HOST"
|
||||
set_env_kv "$env_out" "POSTGRES_DB" "$SUPABASE_POSTGRES_DB"
|
||||
set_env_kv "$env_out" "POSTGRES_PORT" "$SUPABASE_POSTGRES_PORT"
|
||||
}
|
||||
|
||||
convert_compose_to_k8s() {
|
||||
local supa_home="$1"
|
||||
local docker_dir="$supa_home/docker"
|
||||
local compose_file="$docker_dir/docker-compose.yml"
|
||||
local dev_compose="$docker_dir/dev/docker-compose.dev.yml"
|
||||
|
||||
if [[ ! -f "$compose_file" ]]; then
|
||||
err "ERROR: docker-compose.yml not found at $compose_file"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
rm -rf "$SUPABASE_K8S_DIR"
|
||||
mkdir -p "$SUPABASE_K8S_DIR"
|
||||
build_env_file "$supa_home"
|
||||
|
||||
# Export .env for kompose interpolation
|
||||
set -a
|
||||
# Fix unquoted values with spaces in the .env file before sourcing
|
||||
# Use python for robust .env parsing and quoting
|
||||
python3 - <<PY
|
||||
import pathlib
|
||||
import re
|
||||
|
||||
env_file = pathlib.Path("$SUPABASE_K8S_DIR/.env")
|
||||
if env_file.exists():
|
||||
content = env_file.read_text()
|
||||
new_lines = []
|
||||
for line in content.splitlines():
|
||||
# Match KEY=VALUE where VALUE has spaces and is not quoted
|
||||
m = re.match(r'^([A-Z0-9_]+)=([^"\'].* .*)$', line)
|
||||
if m:
|
||||
key, val = m.groups()
|
||||
new_lines.append(f'{key}="{val}"')
|
||||
else:
|
||||
new_lines.append(line)
|
||||
env_file.write_text("\n".join(new_lines) + "\n")
|
||||
PY
|
||||
|
||||
# shellcheck disable=SC1090
|
||||
source "$SUPABASE_K8S_DIR/.env"
|
||||
set +a
|
||||
|
||||
local files=("-f" "$compose_file")
|
||||
if [[ "$SUPABASE_USE_DEV_COMPOSE" == "1" && -f "$dev_compose" ]]; then
|
||||
files+=("-f" "$dev_compose")
|
||||
fi
|
||||
|
||||
log "Converting Supabase docker-compose to Kubernetes manifests for namespace 'supabase' ..."
|
||||
kompose "${files[@]}" convert -n "supabase" -o "$SUPABASE_K8S_DIR" --volumes=configMap --suppress-warnings
|
||||
|
||||
# Remove the built-in Supabase DB workloads (we use CNPG instead)
|
||||
rm -f "$SUPABASE_K8S_DIR"/db-*.yaml "$SUPABASE_K8S_DIR"/db*.yaml 2>/dev/null || true
|
||||
|
||||
# Enforce imagePullPolicy
|
||||
python3 - <<PY
|
||||
import pathlib
|
||||
import re
|
||||
|
||||
k8s_dir = pathlib.Path("$SUPABASE_K8S_DIR")
|
||||
policy = "$SUPABASE_IMAGE_PULL_POLICY"
|
||||
|
||||
def patch_file(path: pathlib.Path):
|
||||
lines = path.read_text().splitlines()
|
||||
out = []
|
||||
for line in lines:
|
||||
out.append(line)
|
||||
m = re.match(r'^(\s*)image:\s*.+', line)
|
||||
if m:
|
||||
indent = m.group(1)
|
||||
out.append(f"{indent}imagePullPolicy: {policy}")
|
||||
path.write_text("\n".join(out) + "\n")
|
||||
|
||||
# Sanitize container names (must not contain dots)
|
||||
def sanitize_container_names(path: pathlib.Path):
|
||||
content = path.read_text()
|
||||
# Find container names that contain dots and replace dots with hyphens
|
||||
# Look for "name: some.name" within container blocks
|
||||
def repl(match):
|
||||
indent = match.group(1)
|
||||
name = match.group(2)
|
||||
sanitized = name.replace('.', '-')
|
||||
return f"{indent}name: {sanitized}"
|
||||
|
||||
# Match name: value where value has dots, but only if it looks like a container name
|
||||
# We can be more specific by looking for lines that start with - name: or just name: inside containers
|
||||
# Updated regex to be more comprehensive for container names in various K8s objects
|
||||
# 1. Matches '- name: realtime-dev.supabase-realtime'
|
||||
new_content = re.sub(r'(^\s+-\s+name:\s+)([a-zA-Z0-9.-]+)', repl, content, flags=re.MULTILINE)
|
||||
# 2. Matches 'name: realtime-dev.supabase-realtime' (if not already matched by 1)
|
||||
new_content = re.sub(r'(^\s+name:\s+)([a-zA-Z0-9.-]+\.[a-zA-Z0-9.-]+)', repl, new_content, flags=re.MULTILINE)
|
||||
|
||||
# 3. Final safety check: if we are in a containers: block, dots are NOT allowed in names.
|
||||
# This is a bit more aggressive but safer.
|
||||
lines = new_content.splitlines()
|
||||
final_lines = []
|
||||
in_containers = False
|
||||
for line in lines:
|
||||
if 'containers:' in line:
|
||||
in_containers = True
|
||||
|
||||
# If we are in containers block, look for - name: and sanitize it
|
||||
if in_containers:
|
||||
# Use a specific match for - name: to avoid mangling other things
|
||||
if re.match(r'^\s+-\s+name:\s+', line):
|
||||
line = re.sub(r'(\s+-\s+name:\s+)([a-zA-Z0-9.-]+)', repl, line)
|
||||
# If we hit another top level key (no indent), we might be out of spec/containers
|
||||
elif line and not line.startswith(' '):
|
||||
in_containers = False
|
||||
|
||||
final_lines.append(line)
|
||||
|
||||
new_content = "\n".join(final_lines) + "\n"
|
||||
|
||||
if content != new_content:
|
||||
path.write_text(new_content)
|
||||
|
||||
for path in k8s_dir.glob("*.yaml"):
|
||||
patch_file(path)
|
||||
sanitize_container_names(path)
|
||||
PY
|
||||
}
|
||||
|
||||
apply_k8s_resources() {
|
||||
log "Applying Supabase resources to namespace 'supabase' ..."
|
||||
kubectl apply -n "supabase" -f "$SUPABASE_K8S_DIR"
|
||||
|
||||
# Create an alias service 'supabase-db' in the CURRENT namespace that points to Supabase Postgres in 'supabase' namespace
|
||||
# This allows prole-db (in current namespace) to connect to Supabase
|
||||
cat <<EOF | kubectl apply -n "$NAMESPACE" -f -
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: supabase-db
|
||||
labels:
|
||||
app: supabase-db-federated
|
||||
spec:
|
||||
type: ExternalName
|
||||
externalName: db.supabase.svc.cluster.local
|
||||
EOF
|
||||
|
||||
# Also create an alias 'db' in 'supabase' namespace pointing to prole-db in CURRENT namespace
|
||||
# This allows Supabase components to use prole-db as their primary DB
|
||||
cat <<EOF | kubectl apply -n "supabase" -f -
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: db
|
||||
labels:
|
||||
app: prole-db-alias
|
||||
spec:
|
||||
type: ExternalName
|
||||
externalName: prole-db-rw.${NAMESPACE}.svc.cluster.local
|
||||
EOF
|
||||
}
|
||||
|
||||
apply_db_migrations() {
|
||||
if [[ "$SUPABASE_APPLY_DB_MIGRATIONS" != "1" ]]; then
|
||||
log "SUPABASE_APPLY_DB_MIGRATIONS=0; skipping Supabase DB migrations."
|
||||
return
|
||||
fi
|
||||
|
||||
local supa_home="$1"
|
||||
local sql_dir="$supa_home/docker/volumes/db"
|
||||
if [[ ! -d "$sql_dir" ]]; then
|
||||
err "WARN: Supabase SQL directory not found: $sql_dir"
|
||||
return
|
||||
fi
|
||||
|
||||
local primary
|
||||
primary=$(kubectl -n "$NAMESPACE" get cluster "$CNPG_CLUSTER_NAME" -o jsonpath='{.status.currentPrimary}' 2>/dev/null || true)
|
||||
if [[ -z "$primary" ]]; then
|
||||
primary=$(kubectl -n "$NAMESPACE" get pods -l "cnpg.io/cluster=$CNPG_CLUSTER_NAME" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null || true)
|
||||
fi
|
||||
if [[ -z "$primary" ]]; then
|
||||
err "WARN: Unable to locate primary CNPG pod; skipping migrations."
|
||||
return
|
||||
fi
|
||||
|
||||
local db_pass
|
||||
db_pass=$(get_db_password || true)
|
||||
if [[ -z "$db_pass" ]]; then
|
||||
err "WARN: Unable to read prole-db-superuser password; skipping migrations."
|
||||
return
|
||||
fi
|
||||
|
||||
log "Applying Supabase SQL migrations to CNPG (${SUPABASE_POSTGRES_DB}) ..."
|
||||
local files=(
|
||||
"$sql_dir/_supabase.sql"
|
||||
"$sql_dir/webhooks.sql"
|
||||
"$sql_dir/realtime.sql"
|
||||
"$sql_dir/logs.sql"
|
||||
"$sql_dir/pooler.sql"
|
||||
"$sql_dir/jwt.sql"
|
||||
"$sql_dir/roles.sql"
|
||||
)
|
||||
|
||||
for f in "${files[@]}"; do
|
||||
if [[ -f "$f" ]]; then
|
||||
log "Applying $(basename "$f") ..."
|
||||
kubectl -n "$NAMESPACE" exec -i "$primary" -c postgres -- \
|
||||
env PGPASSWORD="$db_pass" psql -U postgres -d "$SUPABASE_POSTGRES_DB" -v ON_ERROR_STOP=1 -f - < "$f" || true
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
delete_manifests() {
|
||||
for f in "${SUPABASE_FILES[@]}"; do
|
||||
if [[ -f "$f" ]]; then
|
||||
kubectl delete -n "$NAMESPACE" -f "$f" --ignore-not-found
|
||||
delete_k8s_resources() {
|
||||
if [[ -d "$SUPABASE_K8S_DIR" ]]; then
|
||||
kubectl delete -n "supabase" -f "$SUPABASE_K8S_DIR" --ignore-not-found || true
|
||||
fi
|
||||
done
|
||||
kubectl delete -n "$NAMESPACE" svc/supabase-db --ignore-not-found || true
|
||||
kubectl delete -n "supabase" svc/db --ignore-not-found || true
|
||||
}
|
||||
|
||||
status() {
|
||||
ensure_tools
|
||||
echo "Supabase status in namespace '$NAMESPACE':"
|
||||
kubectl get deploy supabase -n "$NAMESPACE" || true
|
||||
kubectl get svc supabase -n "$NAMESPACE" || true
|
||||
log "Supabase status in namespace 'supabase':"
|
||||
kubectl get deploy,svc -n "supabase" | grep -E "supabase|kong|auth|rest|realtime|storage|meta|analytics|vector|imgproxy|functions|edge|pooler|db" || true
|
||||
log "Federated services in namespace '$NAMESPACE':"
|
||||
kubectl get svc -n "$NAMESPACE" | grep supabase-db || true
|
||||
}
|
||||
|
||||
case "$ACTION" in
|
||||
start)
|
||||
ensure_tools
|
||||
ensure_prereqs
|
||||
echo "Applying Supabase manifests in namespace '$NAMESPACE'..."
|
||||
apply_manifests
|
||||
SUPABASE_HOME="$(detect_supabase_home)" || { err "ERROR: Supabase repo not found. Set SUPABASE_HOME or symlink ~/prole/supabase."; exit 1; }
|
||||
log "Using Supabase repo: $SUPABASE_HOME"
|
||||
log "Using namespace: $NAMESPACE"
|
||||
convert_compose_to_k8s "$SUPABASE_HOME"
|
||||
apply_k8s_resources
|
||||
apply_db_migrations "$SUPABASE_HOME"
|
||||
;;
|
||||
stop)
|
||||
ensure_tools
|
||||
echo "Deleting Supabase manifests from namespace '$NAMESPACE'..."
|
||||
delete_manifests
|
||||
ensure_namespace
|
||||
delete_k8s_resources
|
||||
;;
|
||||
restart)
|
||||
ensure_tools
|
||||
ensure_prereqs
|
||||
echo "Re-applying Supabase manifests in namespace '$NAMESPACE'..."
|
||||
apply_manifests
|
||||
SUPABASE_HOME="$(detect_supabase_home)" || { err "ERROR: Supabase repo not found. Set SUPABASE_HOME or symlink ~/prole/supabase."; exit 1; }
|
||||
convert_compose_to_k8s "$SUPABASE_HOME"
|
||||
delete_k8s_resources
|
||||
apply_k8s_resources
|
||||
apply_db_migrations "$SUPABASE_HOME"
|
||||
;;
|
||||
status)
|
||||
status
|
||||
|
||||
527
install.py
527
install.py
@ -105,6 +105,330 @@ def _expand_path(val: str | None) -> str:
|
||||
return os.path.expandvars(os.path.expanduser(str(val)))
|
||||
|
||||
|
||||
def _resolve_supabase_home(project_root: Path) -> Path | None:
|
||||
env_home = (os.environ.get('SUPABASE_HOME') or '').strip()
|
||||
if env_home:
|
||||
try:
|
||||
candidate = Path(_expand_path(env_home))
|
||||
if candidate.is_dir():
|
||||
return candidate
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
candidates = [
|
||||
Path.home() / 'prole' / 'supabase',
|
||||
Path.home() / 'dev' / 'supabase',
|
||||
project_root / 'supabase',
|
||||
]
|
||||
prole_home = (os.environ.get('PROLE_HOME') or '').strip()
|
||||
if prole_home:
|
||||
try:
|
||||
candidates.append(Path(_expand_path(prole_home)).parent / 'supabase')
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
for candidate in candidates:
|
||||
try:
|
||||
if candidate.is_dir():
|
||||
return candidate
|
||||
except Exception:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _collect_images_from_files(paths: list[Path]) -> set[str]:
|
||||
images = set()
|
||||
for path in paths:
|
||||
if not path or not path.exists():
|
||||
continue
|
||||
try:
|
||||
for line in path.read_text().splitlines():
|
||||
m = re.match(r'^\s*image:\s*([^\s#]+)', line)
|
||||
if not m:
|
||||
continue
|
||||
img = m.group(1).strip().strip('"').strip("'")
|
||||
if not img:
|
||||
continue
|
||||
img = os.path.expandvars(img)
|
||||
if '${' in img or '}' in img or '$' in img:
|
||||
continue
|
||||
images.add(img)
|
||||
except Exception:
|
||||
continue
|
||||
return images
|
||||
|
||||
|
||||
def _clean_yaml_value(raw: str) -> str:
|
||||
if raw is None:
|
||||
return ''
|
||||
val = raw.strip()
|
||||
if '#' in val:
|
||||
val = val.split('#', 1)[0].strip()
|
||||
if len(val) >= 2 and ((val[0] == val[-1]) and val.startswith(("'", '"'))):
|
||||
val = val[1:-1]
|
||||
return val.strip()
|
||||
|
||||
|
||||
def _parse_yaml_scalar_values(path: Path, keys: set[str]) -> dict:
|
||||
data = {}
|
||||
if not path.exists():
|
||||
return data
|
||||
try:
|
||||
for raw in path.read_text().splitlines():
|
||||
line = raw.strip()
|
||||
if not line or line.startswith('#') or line == '---':
|
||||
continue
|
||||
if line.startswith('- '):
|
||||
continue
|
||||
if ':' not in line:
|
||||
continue
|
||||
key, val = line.split(':', 1)
|
||||
key = key.strip()
|
||||
if key not in keys:
|
||||
continue
|
||||
clean_val = _clean_yaml_value(val)
|
||||
if clean_val:
|
||||
data[key] = clean_val
|
||||
except Exception:
|
||||
pass
|
||||
return data
|
||||
|
||||
|
||||
def _parse_internal_a_records(path: Path) -> tuple[str, dict, dict]:
|
||||
domain = ''
|
||||
ip_map: dict[str, str] = {}
|
||||
fqdn_map: dict[str, str] = {}
|
||||
if not path.exists():
|
||||
return domain, ip_map, fqdn_map
|
||||
try:
|
||||
lines = path.read_text().splitlines()
|
||||
except Exception:
|
||||
return domain, ip_map, fqdn_map
|
||||
in_block = False
|
||||
current: dict[str, str] = {}
|
||||
records = []
|
||||
for raw in lines:
|
||||
if not in_block:
|
||||
s = raw.strip()
|
||||
if s.startswith('prole_domain:'):
|
||||
domain = _clean_yaml_value(s.split(':', 1)[1])
|
||||
if s.startswith('prole_internal_a_records:'):
|
||||
in_block = True
|
||||
continue
|
||||
if raw and not raw.startswith((' ', '\t')):
|
||||
break
|
||||
s = raw.strip()
|
||||
if not s:
|
||||
continue
|
||||
if s.startswith('- '):
|
||||
if current:
|
||||
records.append(current)
|
||||
current = {}
|
||||
s = s[2:].strip()
|
||||
if ':' in s:
|
||||
key, val = s.split(':', 1)
|
||||
key = key.strip()
|
||||
if key in ('fqdn', 'ipv4'):
|
||||
current[key] = _clean_yaml_value(val)
|
||||
if current:
|
||||
records.append(current)
|
||||
for rec in records:
|
||||
fqdn = rec.get('fqdn')
|
||||
ip = rec.get('ipv4')
|
||||
if not fqdn or not ip:
|
||||
continue
|
||||
fqdn_map[fqdn] = ip
|
||||
ip_map[fqdn] = ip
|
||||
short = fqdn.split('.')[0]
|
||||
if short and short not in ip_map:
|
||||
ip_map[short] = ip
|
||||
return domain, ip_map, fqdn_map
|
||||
|
||||
|
||||
def _parse_ansible_inventory_hosts(path: Path) -> dict:
|
||||
groups: dict[str, list[str]] = {}
|
||||
if not path.exists():
|
||||
return groups
|
||||
try:
|
||||
current_group = None
|
||||
for raw in path.read_text().splitlines():
|
||||
line = raw.strip()
|
||||
if not line or line.startswith(('#', ';')):
|
||||
continue
|
||||
if line.startswith('[') and line.endswith(']'):
|
||||
current_group = line[1:-1].strip()
|
||||
groups.setdefault(current_group, [])
|
||||
continue
|
||||
if current_group is None:
|
||||
continue
|
||||
host = line.split()[0]
|
||||
if host and host not in groups[current_group]:
|
||||
groups[current_group].append(host)
|
||||
except Exception:
|
||||
pass
|
||||
return groups
|
||||
|
||||
|
||||
def _resolve_host_ip(host: str, domain: str, ip_map: dict) -> str:
|
||||
if not host:
|
||||
return ''
|
||||
if host in ip_map:
|
||||
return ip_map[host]
|
||||
if domain:
|
||||
if '.' not in host:
|
||||
fqdn = f"{host}.{domain}"
|
||||
if fqdn in ip_map:
|
||||
return ip_map[fqdn]
|
||||
else:
|
||||
short = host.split('.')[0]
|
||||
if short in ip_map:
|
||||
return ip_map[short]
|
||||
return ''
|
||||
|
||||
|
||||
def _detect_ansible_topology(project_root: Path) -> dict:
|
||||
prole_home = (os.environ.get('PROLE_HOME') or '').strip()
|
||||
base_candidates = []
|
||||
if prole_home:
|
||||
base_candidates.append(Path(_expand_path(prole_home)))
|
||||
base_candidates.append(project_root)
|
||||
infra_path = None
|
||||
for base in base_candidates:
|
||||
try:
|
||||
candidate = base / 'infrastructure'
|
||||
if candidate.is_dir():
|
||||
infra_path = candidate
|
||||
break
|
||||
except Exception:
|
||||
continue
|
||||
if infra_path is None:
|
||||
return {}
|
||||
inventory_path = infra_path / 'inventory'
|
||||
if not inventory_path.is_dir():
|
||||
return {}
|
||||
groups = _parse_ansible_inventory_hosts(inventory_path / 'hosts.ini')
|
||||
vars_vals = _parse_yaml_scalar_values(
|
||||
inventory_path / 'group_vars' / 'all' / 'vars.yml',
|
||||
{'ad_dc_ip', 'prole_domain', 'kerberos_realm', 'krb5_realm', 'kerberos_kdc', 'krb5_kdc', 'kerberos_kdc_ip'}
|
||||
)
|
||||
domain = vars_vals.get('prole_domain', '')
|
||||
ad_dc_ip = vars_vals.get('ad_dc_ip', '')
|
||||
explicit_realm = vars_vals.get('kerberos_realm') or vars_vals.get('krb5_realm') or ''
|
||||
explicit_kdc = vars_vals.get('kerberos_kdc') or vars_vals.get('krb5_kdc') or vars_vals.get('kerberos_kdc_ip') or ''
|
||||
if explicit_kdc and not ad_dc_ip:
|
||||
ad_dc_ip = explicit_kdc
|
||||
dns_domain, ip_map, fqdn_records = _parse_internal_a_records(inventory_path / 'group_vars' / 'all' / 'dns.yml')
|
||||
if not domain and dns_domain:
|
||||
domain = dns_domain
|
||||
ad_vars = _parse_yaml_scalar_values(
|
||||
inventory_path / 'group_vars' / 'ad_dc' / 'vars.yml',
|
||||
{'samba_dns_server'}
|
||||
)
|
||||
samba_dns_server = ad_vars.get('samba_dns_server', '')
|
||||
ad_dc_host = ''
|
||||
ad_dc_hosts = groups.get('ad_dc') or []
|
||||
if ad_dc_hosts:
|
||||
ad_dc_host = ad_dc_hosts[0]
|
||||
if samba_dns_server:
|
||||
ad_dc_host = ad_dc_host or samba_dns_server
|
||||
if not ad_dc_ip:
|
||||
ad_dc_ip = _resolve_host_ip(samba_dns_server, domain, ip_map)
|
||||
if not ad_dc_ip and ad_dc_host:
|
||||
ad_dc_ip = _resolve_host_ip(ad_dc_host, domain, ip_map)
|
||||
kdc_ip = explicit_kdc or ad_dc_ip or ''
|
||||
realm = explicit_realm or (domain.upper() if domain else '')
|
||||
|
||||
all_hosts = set()
|
||||
for hosts in groups.values():
|
||||
all_hosts.update(hosts)
|
||||
if ad_dc_host:
|
||||
all_hosts.add(ad_dc_host)
|
||||
host_ip_map = {}
|
||||
unmapped_hosts = []
|
||||
for host in sorted(all_hosts):
|
||||
ip = _resolve_host_ip(host, domain, ip_map)
|
||||
if ip:
|
||||
host_ip_map[host] = ip
|
||||
else:
|
||||
unmapped_hosts.append(host)
|
||||
|
||||
topology = {
|
||||
'domain': domain,
|
||||
'realm': realm,
|
||||
'internal_records': fqdn_records,
|
||||
'ad_dc': {
|
||||
'host': ad_dc_host,
|
||||
'ip': ad_dc_ip,
|
||||
},
|
||||
'groups': groups,
|
||||
'hosts': host_ip_map,
|
||||
'unmapped_hosts': unmapped_hosts,
|
||||
}
|
||||
try:
|
||||
topology_json = json.dumps(topology, separators=(',', ':'))
|
||||
except Exception:
|
||||
topology_json = ''
|
||||
return {
|
||||
'infrastructure_path': str(infra_path),
|
||||
'inventory_path': str(inventory_path),
|
||||
'domain': domain,
|
||||
'realm': realm,
|
||||
'ad_dc_ip': ad_dc_ip,
|
||||
'ad_dc_host': ad_dc_host,
|
||||
'kdc_ip': kdc_ip,
|
||||
'groups': groups,
|
||||
'hosts': host_ip_map,
|
||||
'unmapped_hosts': unmapped_hosts,
|
||||
'topology': topology,
|
||||
'topology_json': topology_json,
|
||||
}
|
||||
|
||||
|
||||
def _format_ansible_topology_summary(info: dict) -> str:
|
||||
if not info:
|
||||
return ''
|
||||
lines = []
|
||||
inv = info.get('inventory_path') or ''
|
||||
if inv:
|
||||
lines.append(f"Ansible inventory detected at {inv}.")
|
||||
domain = info.get('domain') or ''
|
||||
realm = info.get('realm') or ''
|
||||
if domain or realm:
|
||||
if domain and realm:
|
||||
lines.append(f"Domain: {domain} (Realm: {realm})")
|
||||
elif domain:
|
||||
lines.append(f"Domain: {domain}")
|
||||
else:
|
||||
lines.append(f"Realm: {realm}")
|
||||
ad_dc_host = info.get('ad_dc_host') or ''
|
||||
ad_dc_ip = info.get('ad_dc_ip') or ''
|
||||
if ad_dc_host or ad_dc_ip:
|
||||
if ad_dc_host and ad_dc_ip:
|
||||
lines.append(f"AD DC: {ad_dc_host} -> {ad_dc_ip}")
|
||||
elif ad_dc_ip:
|
||||
lines.append(f"AD DC IP: {ad_dc_ip}")
|
||||
else:
|
||||
lines.append(f"AD DC Host: {ad_dc_host}")
|
||||
groups = info.get('groups') or {}
|
||||
if groups:
|
||||
group_bits = []
|
||||
for name in sorted(groups.keys()):
|
||||
group_bits.append(f"{name}({len(groups[name])})")
|
||||
lines.append("Groups: " + ", ".join(group_bits))
|
||||
hosts = info.get('hosts') or {}
|
||||
if hosts:
|
||||
host_items = sorted(hosts.items())
|
||||
preview = host_items[:8]
|
||||
host_str = ", ".join([f"{h}={ip}" for h, ip in preview])
|
||||
if len(host_items) > len(preview):
|
||||
host_str += f", +{len(host_items) - len(preview)} more"
|
||||
lines.append("Hosts: " + host_str)
|
||||
unmapped = info.get('unmapped_hosts') or []
|
||||
if unmapped:
|
||||
lines.append(f"Hosts without IPs: {len(unmapped)}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _render_prole_cfg(inputs: dict, globals_to_save: dict, sections: dict, generated_at: str | None = None) -> str:
|
||||
content = []
|
||||
content.append('; Prole Master Configuration File')
|
||||
@ -164,7 +488,7 @@ class ProleController:
|
||||
pg_version_file = self.project_root / "conf" / "postgresql" / ".version"
|
||||
release_file = self.project_root / "prole-db" / ".version"
|
||||
pg_version = pg_version_file.read_text().strip() if pg_version_file.exists() else "17.7"
|
||||
release = release_file.read_text().strip() if release_file.exists() else "41"
|
||||
release = release_file.read_text().strip() if release_file.exists() else "43"
|
||||
if release.isdigit():
|
||||
release = release.zfill(3)
|
||||
return f"{pg_version}-{release}"
|
||||
@ -499,6 +823,12 @@ class ProleInstaller:
|
||||
self.kerberos_kdc = tk.StringVar()
|
||||
self.supabase_enabled = tk.BooleanVar(value=False)
|
||||
self.at_rest_encryption_enabled = tk.BooleanVar(value=False)
|
||||
self.ansible_topology = {}
|
||||
self.ansible_topology_summary = ''
|
||||
try:
|
||||
self._apply_ansible_topology_defaults()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Create pages
|
||||
self.page_frames = {}
|
||||
@ -684,6 +1014,43 @@ class ProleInstaller:
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _apply_ansible_topology_defaults(self):
|
||||
info = _detect_ansible_topology(PROJECT_ROOT)
|
||||
if not info:
|
||||
return
|
||||
self.ansible_topology = info
|
||||
self.ansible_topology_summary = _format_ansible_topology_summary(info)
|
||||
try:
|
||||
net = self.prole_cfg_data.get('Network', {})
|
||||
if info.get('topology_json'):
|
||||
net['ANSIBLE_TOPOLOGY'] = info['topology_json']
|
||||
if info.get('inventory_path'):
|
||||
net['ANSIBLE_INVENTORY'] = info['inventory_path']
|
||||
if info.get('infrastructure_path'):
|
||||
net['ANSIBLE_INFRASTRUCTURE'] = info['infrastructure_path']
|
||||
if info.get('domain'):
|
||||
net['ANSIBLE_DOMAIN'] = info['domain']
|
||||
if info.get('realm'):
|
||||
net['ANSIBLE_REALM'] = info['realm']
|
||||
if info.get('ad_dc_host'):
|
||||
net['AD_DC_HOST'] = info['ad_dc_host']
|
||||
if info.get('ad_dc_ip'):
|
||||
net['AD_DC_IP'] = info['ad_dc_ip']
|
||||
if info.get('kdc_ip'):
|
||||
net['KDC_ANSIBLE_DETECTED'] = info['kdc_ip']
|
||||
self.prole_cfg_data['Network'] = net
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
if not self.kerberos_kdc.get().strip() and info.get('kdc_ip'):
|
||||
self.kerberos_kdc.set(info['kdc_ip'])
|
||||
self.kerberos_enabled.set(True)
|
||||
if not self.kerberos_realm.get().strip() and info.get('realm'):
|
||||
self.kerberos_realm.set(info['realm'])
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _create_sidebar_nav(self):
|
||||
"""Create the left-hand navigation menu."""
|
||||
tk.Label(self.sidebar, text="INSTALLER", bg='#F5F5DC', fg='#8B8B7A',
|
||||
@ -1901,12 +2268,12 @@ class ProleInstaller:
|
||||
font=('SF Pro Text', 18), anchor='ne')
|
||||
|
||||
self._render_title('Network Configuration Scan', y=150)
|
||||
self._render_paragraph("The network prole-scan will detect Kerberos services, Active Directory controllers, and other configuration details required for deployment.", y=210)
|
||||
self._render_paragraph("The network prole-agent will detect Kerberos services, Active Directory controllers, and other configuration details required for deployment.", y=210)
|
||||
|
||||
y = 280
|
||||
self.scan_status_var = tk.StringVar(value="Ready to prole-scan")
|
||||
self.scan_status_var = tk.StringVar(value="Ready to prole-agent")
|
||||
# Ready to scan text (drawn on canvas)
|
||||
status_item = ui.canvas_text(self, 48, y, "Ready to prole-scan", fill='black', font=('SF Pro Text', 12))
|
||||
status_item = ui.canvas_text(self, 48, y, "Ready to prole-agent", fill='black', font=('SF Pro Text', 12))
|
||||
|
||||
# Link variable to canvas text
|
||||
def update_status_text(*args):
|
||||
@ -1920,6 +2287,10 @@ class ProleInstaller:
|
||||
self.scan_results_console = self._create_console_output(y=320, title="Scan Output", width=880, height=380)
|
||||
self.scan_results_text = self.scan_results_console.text
|
||||
|
||||
if getattr(self, 'ansible_topology_summary', ''):
|
||||
self.scan_results_console.write(self.ansible_topology_summary + "\n")
|
||||
self.scan_results_console.write("Ansible topology loaded; network scan can refine detection.\n")
|
||||
|
||||
y = 740
|
||||
|
||||
# Start Network Scan Button (placed on canvas)
|
||||
@ -1973,7 +2344,7 @@ class ProleInstaller:
|
||||
self._scan_running = True
|
||||
|
||||
self.scan_results_console.clear()
|
||||
self.scan_results_console.write("Initializing network scan using prole-net/prole-scan ...\n")
|
||||
self.scan_results_console.write("Initializing network scan using prole-net/prole-agent ...\n")
|
||||
self.scan_status_var.set("Scanning...")
|
||||
|
||||
# Start animation
|
||||
@ -1983,7 +2354,7 @@ class ProleInstaller:
|
||||
def worker():
|
||||
try:
|
||||
# Use the new scan binary
|
||||
scan_binary = get_resource_path("prole-net/prole-scan")
|
||||
scan_binary = get_resource_path("prole-net/prole-agent")
|
||||
if not scan_binary.exists():
|
||||
self.safe_after(lambda: self.scan_results_console.write(f"Scan binary not found at {scan_binary}\n"))
|
||||
self.safe_after(lambda: self.scan_status_var.set("Scan failed"))
|
||||
@ -1998,7 +2369,7 @@ class ProleInstaller:
|
||||
|
||||
# Run scan from writable directory with 10s timeout and summary analysis
|
||||
# Set bufsize=0 for truly unbuffered binary stream reading
|
||||
# Ensure PYTHONUNBUFFERED=1 is set in case prole-scan is/uses Python
|
||||
# Ensure PYTHONUNBUFFERED=1 is set in case prole-agent is/uses Python
|
||||
env = os.environ.copy()
|
||||
env['PYTHONUNBUFFERED'] = '1'
|
||||
process = subprocess.Popen([str(scan_binary), "-t", "10", "-s"],
|
||||
@ -3189,6 +3560,7 @@ class ProleInstaller:
|
||||
('CloudNative-PG', 'init_cloudnative_pg.sh'),
|
||||
]
|
||||
if self.kerberos_enabled.get():
|
||||
scripts.append(('Authority', 'init_authority.sh'))
|
||||
scripts.append(('Kerberos Realm', 'init_kerberos.sh'))
|
||||
scripts.extend([
|
||||
('Prole DB', 'init_prole-db.sh'),
|
||||
@ -3548,7 +3920,49 @@ class ProleInstaller:
|
||||
else:
|
||||
self.script_consoles["init_cloudnative_pg.sh"].write("Skipping CloudNative-PG initialization because previous steps failed.\n")
|
||||
|
||||
# 4. init_kerberos.sh initialize (optional)
|
||||
# 4. init_authority.sh start (optional)
|
||||
if overall_success and self.kerberos_enabled.get():
|
||||
script = "init_authority.sh"
|
||||
if script in self.script_consoles:
|
||||
_select_tab(script)
|
||||
self.script_consoles[script].clear()
|
||||
self.script_consoles[script].write(f"Running {script} start...\n")
|
||||
|
||||
log_path = _log_path_for(script)
|
||||
self._record_install_log(log_path)
|
||||
try:
|
||||
log_fp = log_path.open('a', encoding='utf-8')
|
||||
except Exception:
|
||||
log_fp = None
|
||||
|
||||
def _auth_line(line):
|
||||
self.script_consoles[script].write(line)
|
||||
if log_fp:
|
||||
try:
|
||||
log_fp.write(line)
|
||||
log_fp.flush()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
rc_auth = self.controller.run_script(
|
||||
script,
|
||||
args=['start'],
|
||||
env=env,
|
||||
on_line=_auth_line
|
||||
)
|
||||
if log_fp:
|
||||
try:
|
||||
log_fp.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if rc_auth != 0:
|
||||
self.script_consoles[script].write(f"\nERROR: {script} start failed with code {rc_auth}\n")
|
||||
overall_success = False
|
||||
else:
|
||||
self.script_consoles[script].write(f"\n{script} completed successfully.\n")
|
||||
|
||||
# 5. init_kerberos.sh initialize (optional)
|
||||
if overall_success and self.kerberos_enabled.get():
|
||||
script = "init_kerberos.sh"
|
||||
if script in self.script_consoles:
|
||||
@ -3596,7 +4010,7 @@ class ProleInstaller:
|
||||
elif not self.kerberos_enabled.get():
|
||||
if "init_kerberos.sh" in self.script_consoles:
|
||||
self.script_consoles["init_kerberos.sh"].write("Kerberos auth disabled; skipping init_kerberos.sh.\n")
|
||||
# 5. init_prole-db.sh start
|
||||
# 6. init_prole-db.sh start
|
||||
if overall_success:
|
||||
script = "init_prole-db.sh"
|
||||
_select_tab(script)
|
||||
@ -3639,7 +4053,7 @@ class ProleInstaller:
|
||||
else:
|
||||
self.script_consoles["init_prole-db.sh"].write("Skipping Prole DB initialization because previous steps failed.\n")
|
||||
|
||||
# 6. init_prole-db-backup.sh start
|
||||
# 7. init_prole-db-backup.sh start
|
||||
if overall_success:
|
||||
script = "init_prole-db-backup.sh"
|
||||
_select_tab(script)
|
||||
@ -3682,7 +4096,7 @@ class ProleInstaller:
|
||||
else:
|
||||
self.script_consoles["init_prole-db-backup.sh"].write("Skipping Prole DB backup because previous steps failed.\n")
|
||||
|
||||
# 7. init_supabase.sh start (optional)
|
||||
# 8. init_supabase.sh start
|
||||
if overall_success:
|
||||
script = "init_supabase.sh"
|
||||
_select_tab(script)
|
||||
@ -3718,7 +4132,7 @@ class ProleInstaller:
|
||||
pass
|
||||
if rc_sb != 0:
|
||||
self.script_consoles[script].write(f"\nERROR: {script} start failed with code {rc_sb}\n")
|
||||
overall_success = False
|
||||
# overall_success = False # Keep as False if you want to block port forwards on failure
|
||||
else:
|
||||
self.script_consoles[script].write(f"\n{script} completed successfully.\n")
|
||||
else:
|
||||
@ -3726,7 +4140,7 @@ class ProleInstaller:
|
||||
else:
|
||||
self.script_consoles["init_supabase.sh"].write("Skipping Supabase because previous steps failed.\n")
|
||||
|
||||
# 8. init_port_forwards.sh start
|
||||
# 9. init_port_forwards.sh start
|
||||
if overall_success:
|
||||
script = "init_port_forwards.sh"
|
||||
_select_tab(script)
|
||||
@ -6585,16 +6999,16 @@ esac
|
||||
base_dir = PROJECT_ROOT / rel_dir
|
||||
if not base_dir.exists():
|
||||
continue
|
||||
for path in base_dir.glob('*.yaml'):
|
||||
try:
|
||||
for line in path.read_text().splitlines():
|
||||
m = re.match(r'^\s*image:\s*([^\s#]+)', line)
|
||||
if m:
|
||||
img = m.group(1).strip().strip('"').strip("'")
|
||||
if img:
|
||||
images.add(img)
|
||||
except Exception:
|
||||
continue
|
||||
images.update(_collect_images_from_files(list(base_dir.glob('*.yaml'))))
|
||||
|
||||
if include_supabase:
|
||||
supa_home = _resolve_supabase_home(PROJECT_ROOT)
|
||||
if supa_home:
|
||||
docker_dir = supa_home / 'docker'
|
||||
compose_files = [docker_dir / 'docker-compose.yml']
|
||||
if os.environ.get('SUPABASE_USE_DEV_COMPOSE') == '1':
|
||||
compose_files.append(docker_dir / 'dev' / 'docker-compose.dev.yml')
|
||||
images.update(_collect_images_from_files(compose_files))
|
||||
|
||||
if not include_supabase:
|
||||
images = {img for img in images if 'supabase' not in img}
|
||||
@ -6757,7 +7171,7 @@ esac
|
||||
if not manifest_path.exists():
|
||||
continue
|
||||
content = manifest_path.read_text()
|
||||
# Update imageName: prole-db:17.7-041
|
||||
# Update imageName: prole-db:17.7-043
|
||||
new_content = re.sub(r'imageName:\s*.*', f'imageName: {image_tag}', content)
|
||||
|
||||
# Update Kerberos realm in manifest if enabled
|
||||
@ -7460,16 +7874,16 @@ class ProleSilentInstaller:
|
||||
base_dir = self.project_root / rel_dir
|
||||
if not base_dir.exists():
|
||||
continue
|
||||
for path in base_dir.glob('*.yaml'):
|
||||
try:
|
||||
for line in path.read_text().splitlines():
|
||||
m = re.match(r'^\s*image:\s*([^\s#]+)', line)
|
||||
if m:
|
||||
img = m.group(1).strip().strip('"').strip("'")
|
||||
if img:
|
||||
images.add(img)
|
||||
except Exception:
|
||||
continue
|
||||
images.update(_collect_images_from_files(list(base_dir.glob('*.yaml'))))
|
||||
|
||||
if include_supabase:
|
||||
supa_home = _resolve_supabase_home(self.project_root)
|
||||
if supa_home:
|
||||
docker_dir = supa_home / 'docker'
|
||||
compose_files = [docker_dir / 'docker-compose.yml']
|
||||
if os.environ.get('SUPABASE_USE_DEV_COMPOSE') == '1':
|
||||
compose_files.append(docker_dir / 'dev' / 'docker-compose.dev.yml')
|
||||
images.update(_collect_images_from_files(compose_files))
|
||||
|
||||
if not include_supabase:
|
||||
images = {img for img in images if 'supabase' not in img}
|
||||
@ -7531,11 +7945,52 @@ class ProleSilentInstaller:
|
||||
defaults = self._default_inputs()
|
||||
loaded = self._load_inputs_from_cfg()
|
||||
self.inputs = {**defaults, **loaded}
|
||||
try:
|
||||
self._apply_ansible_defaults(set(loaded.keys()))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
for k in ('env_setup.PROLE_HOME', 'env_setup.PROLE_CONF', 'env_setup.PROLE_DATA', 'env_setup.PROLE_LOGS', 'env_setup.PROLE_SERVICE'):
|
||||
if k in self.inputs:
|
||||
self.inputs[k] = _expand_path(self.inputs[k])
|
||||
|
||||
def _apply_ansible_defaults(self, loaded_keys: set[str] | None = None):
|
||||
info = _detect_ansible_topology(self.project_root)
|
||||
if not info:
|
||||
return
|
||||
loaded_keys = loaded_keys or set()
|
||||
try:
|
||||
net = self.prole_cfg_data.get('Network', {})
|
||||
if info.get('topology_json'):
|
||||
net['ANSIBLE_TOPOLOGY'] = info['topology_json']
|
||||
if info.get('inventory_path'):
|
||||
net['ANSIBLE_INVENTORY'] = info['inventory_path']
|
||||
if info.get('infrastructure_path'):
|
||||
net['ANSIBLE_INFRASTRUCTURE'] = info['infrastructure_path']
|
||||
if info.get('domain'):
|
||||
net['ANSIBLE_DOMAIN'] = info['domain']
|
||||
if info.get('realm'):
|
||||
net['ANSIBLE_REALM'] = info['realm']
|
||||
if info.get('ad_dc_host'):
|
||||
net['AD_DC_HOST'] = info['ad_dc_host']
|
||||
if info.get('ad_dc_ip'):
|
||||
net['AD_DC_IP'] = info['ad_dc_ip']
|
||||
if info.get('kdc_ip'):
|
||||
net['KDC_ANSIBLE_DETECTED'] = info['kdc_ip']
|
||||
self.prole_cfg_data['Network'] = net
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if 'kerberos_config.kdc' not in loaded_keys:
|
||||
if not self._get_input('kerberos_config.kdc', '').strip() and info.get('kdc_ip'):
|
||||
self.inputs['kerberos_config.kdc'] = info['kdc_ip']
|
||||
if 'kerberos_config.realm' not in loaded_keys:
|
||||
if not self._get_input('kerberos_config.realm', '').strip() and info.get('realm'):
|
||||
self.inputs['kerberos_config.realm'] = info['realm']
|
||||
if 'kerberos_config.enabled' not in loaded_keys:
|
||||
if info.get('kdc_ip') and not self._get_input_bool('kerberos_config.enabled', False):
|
||||
self.inputs['kerberos_config.enabled'] = _bool_str(True)
|
||||
|
||||
def _write_cfg(self):
|
||||
globals_to_save = {
|
||||
'PROLE_HOME': self._get_input('env_setup.PROLE_HOME', ''),
|
||||
@ -7608,8 +8063,8 @@ class ProleSilentInstaller:
|
||||
if not self._get_input_bool('network_scan.run', DEFAULT_ACTION_FLAGS.get('network_scan.run', True)):
|
||||
self.log("[SKIP] Network scan disabled.")
|
||||
return
|
||||
self.log("==> Network scan (prole-scan)")
|
||||
scan_binary = get_resource_path("prole-net/prole-scan")
|
||||
self.log("==> Network scan (prole-agent)")
|
||||
scan_binary = get_resource_path("prole-net/prole-agent")
|
||||
if not scan_binary.exists():
|
||||
self.err(f"[ERROR] Scan binary not found at {scan_binary}")
|
||||
return
|
||||
|
||||
@ -22,7 +22,7 @@ datas = [
|
||||
|
||||
# Binaries to include (with execute permissions)
|
||||
binaries = [
|
||||
('prole-net/prole-scan', 'prole-net'),
|
||||
('prole-net/prole-agent', 'prole-net'),
|
||||
]
|
||||
|
||||
# Hidden imports
|
||||
|
||||
@ -12,8 +12,8 @@ data:
|
||||
|
||||
[realms]
|
||||
PROLE.ORG = {
|
||||
kdc = 10.0.0.3
|
||||
admin_server = 10.0.0.3
|
||||
kdc = kdc.prole.org
|
||||
admin_server = kdc.prole.org
|
||||
}
|
||||
|
||||
[domain_realm]
|
||||
|
||||
@ -5,7 +5,7 @@ metadata:
|
||||
name: prole-db
|
||||
spec:
|
||||
instances: 3
|
||||
imageName: prole-db:17.7-041
|
||||
imageName: prole-db:17.7-053
|
||||
postgresUID: 100
|
||||
postgresGID: 101
|
||||
maxSyncReplicas: 1
|
||||
|
||||
@ -4,7 +4,7 @@ metadata:
|
||||
name: prole-db
|
||||
spec:
|
||||
instances: 3
|
||||
imageName: prole-db:17.7-041
|
||||
imageName: prole-db:17.7-053
|
||||
postgresUID: 100
|
||||
postgresGID: 101
|
||||
maxSyncReplicas: 1
|
||||
|
||||
@ -1 +1 @@
|
||||
41
|
||||
53
|
||||
|
||||
@ -62,16 +62,6 @@ RUN set -eux; \
|
||||
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
|
||||
cmake \
|
||||
libssl-dev \
|
||||
libkrb5-dev \
|
||||
krb5-user \
|
||||
sssd \
|
||||
sssd-krb5 \
|
||||
sssd-krb5-common \
|
||||
sssd-tools \
|
||||
libpam-sss \
|
||||
libnss-sss \
|
||||
oddjob \
|
||||
oddjob-mkhomedir \
|
||||
libgdal-dev \
|
||||
libproj-dev \
|
||||
libgeos-dev \
|
||||
@ -84,7 +74,7 @@ RUN set -eux; \
|
||||
|
||||
# Copy and install pg_prolelog from local .deb package
|
||||
ARG TARGETARCH
|
||||
COPY pg_prolelog/percona-postgresql-17-prolelog_1.0-1_${TARGETARCH:-amd64}.deb /tmp/pg_prolelog.deb
|
||||
COPY pg_prolelog/percona-postgresql-17-pg-prolelog_1.0-1_${TARGETARCH:-amd64}.deb /tmp/pg_prolelog.deb
|
||||
RUN set -eux; \
|
||||
apt-get update; \
|
||||
apt-get install -y --no-install-recommends /tmp/pg_prolelog.deb; \
|
||||
@ -99,6 +89,7 @@ RUN set -eux; \
|
||||
percona-postgresql-17-pgvector \
|
||||
percona-postgresql-17-postgis-3 \
|
||||
percona-postgresql-contrib-17 \
|
||||
percona-postgresql-contrib \
|
||||
freetds-dev \
|
||||
; \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
@ -123,7 +114,6 @@ RUN set -eux; \
|
||||
echo " CREATE EXTENSION IF NOT EXISTS tds_fdw;" >> /docker-entrypoint-initdb.d/20_create_extensions.sh; \
|
||||
echo " CREATE EXTENSION IF NOT EXISTS postgis;" >> /docker-entrypoint-initdb.d/20_create_extensions.sh; \
|
||||
echo " CREATE EXTENSION IF NOT EXISTS postgis_topology;" >> /docker-entrypoint-initdb.d/20_create_extensions.sh; \
|
||||
echo " CREATE EXTENSION IF NOT EXISTS postgis_tiger_geocoder;" >> /docker-entrypoint-initdb.d/20_create_extensions.sh; \
|
||||
echo " CREATE EXTENSION IF NOT EXISTS pg_prolelog;" >> /docker-entrypoint-initdb.d/20_create_extensions.sh; \
|
||||
echo "EOSQL" >> /docker-entrypoint-initdb.d/20_create_extensions.sh; \
|
||||
chmod +x /docker-entrypoint-initdb.d/20_create_extensions.sh
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -41,6 +41,7 @@ httpcore==1.0.9
|
||||
httpx==0.28.1
|
||||
idna==3.10
|
||||
importlib_metadata==8.7.0
|
||||
langchain
|
||||
isodate==0.6.1
|
||||
Jinja2==3.1.6
|
||||
jiter==0.10.0
|
||||
|
||||
@ -8,6 +8,7 @@ cryptography
|
||||
ghp-import
|
||||
griffe
|
||||
idna
|
||||
langchain
|
||||
Jinja2
|
||||
jsonpatch
|
||||
jsonpointer
|
||||
|
||||
@ -7,7 +7,7 @@ export POSTGRES_PASSWORD=$(cat $PROLE_HOME/postgres-password.txt)
|
||||
PG_VERSION=$(cat "$PROLE_HOME/conf/postgresql/.version" 2>/dev/null | tr -d '[:space:]')
|
||||
RELEASE=$(cat "$PROLE_HOME/prole-db/.version" 2>/dev/null | tr -d '[:space:]')
|
||||
if [[ "$RELEASE" =~ ^[0-9]+$ ]]; then RELEASE=$(printf "%03d" "$RELEASE"); fi
|
||||
IMAGE="prole-db:${PG_VERSION:-17.7}-${RELEASE:-041}"
|
||||
IMAGE="prole-db:${PG_VERSION:-17.7}-${RELEASE:-043}"
|
||||
|
||||
docker run --rm -it -u root \
|
||||
--hostname=prole-db-00n \
|
||||
|
||||
@ -1,3 +1,4 @@
|
||||
set -eux
|
||||
# export POSTGRES_PASSWORD=$(openssl rand -base64 32)
|
||||
# echo "$POSTGRES_PASSWORD" > postgres-password.txt
|
||||
export POSTGRES_PASSWORD=$(cat postgres-password.txt)
|
||||
@ -6,7 +7,7 @@ PROLE_HOME=~/dev/prole
|
||||
PG_VERSION=$(cat "$PROLE_HOME/conf/postgresql/.version" 2>/dev/null | tr -d '[:space:]')
|
||||
RELEASE=$(cat "$PROLE_HOME/prole-db/.version" 2>/dev/null | tr -d '[:space:]')
|
||||
if [[ "$RELEASE" =~ ^[0-9]+$ ]]; then RELEASE=$(printf "%03d" "$RELEASE"); fi
|
||||
IMAGE="prole-db:${PG_VERSION:-17.7}-${RELEASE:-041}"
|
||||
IMAGE="prole-db:${PG_VERSION:-17.7}-${RELEASE:-043}"
|
||||
|
||||
docker run --rm -it --hostname=prole-db-00n \
|
||||
-e POSTGRES_PASSWORD="$POSTGRES_PASSWORD" \
|
||||
|
||||
@ -27,7 +27,7 @@ datas = [
|
||||
|
||||
# Binaries to include (with execute permissions)
|
||||
binaries = [
|
||||
('prole-net/prole-scan', 'prole-net'),
|
||||
('prole-net/prole-agent', 'prole-net'),
|
||||
]
|
||||
|
||||
# Hidden imports
|
||||
|
||||
@ -40,7 +40,7 @@ for img in ['img/proleIcon.png', 'img/proleLogo.png', 'img/proleLogoSepia.png']:
|
||||
sys.exit(1)
|
||||
"
|
||||
|
||||
run_test "Binary executable" test -x prole-net/prole-scan
|
||||
run_test "Binary executable" test -x prole-net/prole-agent
|
||||
run_test "App bundle" test -d "prole-app/dist/Prole Tools.app"
|
||||
|
||||
# Build system tests
|
||||
@ -51,7 +51,7 @@ run_test "Icon conversion" make build/prole.icns
|
||||
echo -n "[$((TESTS_TOTAL + 1))] Spec includes all resources... "
|
||||
TESTS_TOTAL=$((TESTS_TOTAL + 1))
|
||||
if grep -q "prole-app/dist/Prole Tools.app" installer.spec && \
|
||||
grep -q "prole-net/prole-scan" installer.spec && \
|
||||
grep -q "prole-net/prole-agent" installer.spec && \
|
||||
grep -q "img.*img" installer.spec; then
|
||||
echo "✓"
|
||||
TESTS_PASSED=$((TESTS_PASSED + 1))
|
||||
@ -73,13 +73,13 @@ if [ $TESTS_PASSED -eq $TESTS_TOTAL ]; then
|
||||
echo " ✓ Ncurses terminal interface"
|
||||
echo " ✓ Automatic display detection"
|
||||
echo " ✓ Image resources embedded"
|
||||
echo " ✓ Binary executables embedded (prole-scan)"
|
||||
echo " ✓ Binary executables embedded (prole-agent)"
|
||||
echo " ✓ App bundles embedded (Prole Tools.app)"
|
||||
echo " ✓ Build system configured"
|
||||
echo ""
|
||||
echo "Embedded resources:"
|
||||
echo " • Images: ~11 MB"
|
||||
echo " • prole-scan: 6.8 MB (universal)"
|
||||
echo " • prole-agent: 6.8 MB (universal)"
|
||||
echo " • Prole Tools.app: ~12 MB"
|
||||
echo " • Expected final size: ~50-100 MB"
|
||||
echo ""
|
||||
|
||||
@ -47,8 +47,8 @@ test_item "proleLogoSepia.png" test -f img/proleLogoSepia.png
|
||||
# Binary Resources
|
||||
echo ""
|
||||
echo "BINARY RESOURCES"
|
||||
test_item "prole-scan exists" test -f prole-net/prole-scan
|
||||
test_item "prole-scan executable" test -x prole-net/prole-scan
|
||||
test_item "prole-agent exists" test -f prole-net/prole-agent
|
||||
test_item "prole-agent executable" test -x prole-net/prole-agent
|
||||
|
||||
# App Bundle
|
||||
echo ""
|
||||
@ -67,7 +67,7 @@ echo "SPEC FILE VERIFICATION"
|
||||
python3 scripts/generate_spec.py > /dev/null 2>&1
|
||||
test_item "Spec includes img" grep -q "img" installer.spec
|
||||
test_item "Spec includes prole-db" grep -q "prole-db" installer.spec
|
||||
test_item "Spec includes prole-scan" grep -q "prole-scan" installer.spec
|
||||
test_item "Spec includes prole-agent" grep -q "prole-agent" installer.spec
|
||||
test_item "Spec includes Prole Tools" grep -q "Prole Tools" installer.spec
|
||||
|
||||
# Resource Path Tests
|
||||
@ -85,7 +85,7 @@ from pathlib import Path
|
||||
import sys
|
||||
sys.path.insert(0, str(Path.cwd()))
|
||||
from install import get_resource_path
|
||||
assert get_resource_path('prole-net/prole-scan').exists()
|
||||
assert get_resource_path('prole-net/prole-agent').exists()
|
||||
"
|
||||
test_item "Docker context works" python3 -c "
|
||||
from pathlib import Path
|
||||
@ -113,14 +113,14 @@ if [ $TESTS_PASSED -eq $TESTS_TOTAL ]; then
|
||||
echo " ✓ Automatic display detection"
|
||||
echo " ✓ Static universal binary build system"
|
||||
echo " ✓ Image resources embedded"
|
||||
echo " ✓ prole-scan binary embedded"
|
||||
echo " ✓ prole-agent binary embedded"
|
||||
echo " ✓ Prole Tools.app embedded"
|
||||
echo " ✓ Docker build context embedded"
|
||||
echo " ✓ Docker build uses ~/.prole/build (writable)"
|
||||
echo ""
|
||||
echo "PACKAGE WILL INCLUDE:"
|
||||
echo " • Images (~11 MB)"
|
||||
echo " • prole-scan (6.8 MB)"
|
||||
echo " • prole-agent (6.8 MB)"
|
||||
echo " • Prole Tools.app (~12 MB)"
|
||||
echo " • prole-db build context (~1-5 MB)"
|
||||
echo " • Python runtime + deps (~30-50 MB)"
|
||||
|
||||
@ -36,7 +36,7 @@ test_item "Build directory writable" python3 -c "from pathlib import Path; f = P
|
||||
|
||||
echo ""
|
||||
echo "NETWORK SCAN FIX"
|
||||
test_item "prole-scan binary exists" test -x prole-net/prole-scan
|
||||
test_item "prole-agent binary exists" test -x prole-net/prole-agent
|
||||
test_item "Scan directory creation" python3 -c "from pathlib import Path; d = Path.home() / '.prole' / 'scan'; d.mkdir(parents=True, exist_ok=True); assert d.exists()"
|
||||
test_item "Scan directory writable" python3 -c "from pathlib import Path; f = Path.home() / '.prole' / 'scan' / 'test.txt'; f.write_text('test'); f.unlink()"
|
||||
|
||||
@ -51,7 +51,7 @@ echo ""
|
||||
echo "SPEC FILE VERIFICATION"
|
||||
python3 scripts/generate_spec.py > /dev/null 2>&1
|
||||
test_item "Spec includes prole-db" grep -q "prole-db" installer.spec
|
||||
test_item "Spec includes prole-scan" grep -q "prole-scan" installer.spec
|
||||
test_item "Spec includes prole-agent" grep -q "prole-agent" installer.spec
|
||||
|
||||
# Cleanup
|
||||
echo ""
|
||||
|
||||
@ -28,10 +28,10 @@ if [ $? -ne 0 ]; then exit 1; fi
|
||||
|
||||
# Test 2: Binaries
|
||||
echo "2. Testing binary resources..."
|
||||
if [ -f "prole-net/prole-scan" ] && [ -x "prole-net/prole-scan" ]; then
|
||||
echo " ✓ prole-scan binary present and executable"
|
||||
if [ -f "prole-net/prole-agent" ] && [ -x "prole-net/prole-agent" ]; then
|
||||
echo " ✓ prole-agent binary present and executable"
|
||||
else
|
||||
echo " ✗ prole-scan missing or not executable"
|
||||
echo " ✗ prole-agent missing or not executable"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@ -57,7 +57,7 @@ fi
|
||||
# Test 5: Verify spec includes everything
|
||||
echo "5. Verifying spec includes all resources..."
|
||||
grep -q "prole-app/dist/Prole Tools.app" installer.spec && \
|
||||
grep -q "prole-net/prole-scan" installer.spec && \
|
||||
grep -q "prole-net/prole-agent" installer.spec && \
|
||||
grep -q "img" installer.spec
|
||||
if [ $? -eq 0 ]; then
|
||||
echo " ✓ All resources included in spec"
|
||||
@ -68,8 +68,8 @@ fi
|
||||
|
||||
# Test 6: Check binary size
|
||||
echo "6. Checking binary sizes..."
|
||||
SCAN_SIZE=$(ls -lh prole-net/prole-scan | awk '{print $5}')
|
||||
echo " prole-scan: $SCAN_SIZE (universal binary)"
|
||||
SCAN_SIZE=$(ls -lh prole-net/prole-agent | awk '{print $5}')
|
||||
echo " prole-agent: $SCAN_SIZE (universal binary)"
|
||||
APP_SIZE=$(du -sh "prole-app/dist/Prole Tools.app" | awk '{print $1}')
|
||||
echo " Prole Tools.app: $APP_SIZE"
|
||||
|
||||
@ -78,7 +78,7 @@ echo "=== All Embedded Resource Tests Passed ==="
|
||||
echo ""
|
||||
echo "Resources ready for packaging:"
|
||||
echo " ✓ Images (proleIcon.png, proleLogo.png, proleLogoSepia.png)"
|
||||
echo " ✓ Binary (prole-net/prole-scan)"
|
||||
echo " ✓ Binary (prole-net/prole-agent)"
|
||||
echo " ✓ App bundle (prole-app/dist/Prole Tools.app)"
|
||||
echo ""
|
||||
echo "Build with: make package"
|
||||
|
||||
@ -36,28 +36,28 @@ except Exception as e:
|
||||
PYTEST
|
||||
if [ $? -ne 0 ]; then exit 1; fi
|
||||
|
||||
# Test 3: prole-scan binary exists
|
||||
echo "3. Testing prole-scan binary..."
|
||||
if [ -x "prole-net/prole-scan" ]; then
|
||||
echo " ✓ prole-scan binary exists and is executable"
|
||||
# Test 3: prole-agent binary exists
|
||||
echo "3. Testing prole-agent binary..."
|
||||
if [ -x "prole-net/prole-agent" ]; then
|
||||
echo " ✓ prole-agent binary exists and is executable"
|
||||
else
|
||||
echo " ✗ prole-scan binary missing or not executable"
|
||||
echo " ✗ prole-agent binary missing or not executable"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Test 4: prole-scan can run from scan directory
|
||||
echo "4. Testing prole-scan runs from writable directory..."
|
||||
# Test 4: prole-agent can run from scan directory
|
||||
echo "4. Testing prole-agent runs from writable directory..."
|
||||
mkdir -p /tmp/scan-test
|
||||
cd /tmp/scan-test
|
||||
/Users/chrisfu/dev/prole/prole-net/prole-scan 2>&1 &
|
||||
/Users/chrisfu/dev/prole/prole-net/prole-agent 2>&1 &
|
||||
SCAN_PID=$!
|
||||
sleep 2
|
||||
if ps -p $SCAN_PID > /dev/null 2>&1; then
|
||||
echo " ✓ prole-scan runs successfully"
|
||||
echo " ✓ prole-agent runs successfully"
|
||||
kill $SCAN_PID 2>/dev/null
|
||||
wait 2>/dev/null
|
||||
else
|
||||
echo " ✗ prole-scan failed to start"
|
||||
echo " ✗ prole-agent failed to start"
|
||||
exit 1
|
||||
fi
|
||||
cd /Users/chrisfu/dev/prole
|
||||
@ -85,5 +85,5 @@ echo " • From source: Uses writable cwd"
|
||||
echo " • From package: Uses ~/.prole/scan"
|
||||
echo ""
|
||||
echo "Scan directory: ~/.prole/scan"
|
||||
echo "Purpose: Writable working directory for prole-scan"
|
||||
echo "Purpose: Writable working directory for prole-agent"
|
||||
echo ""
|
||||
|
||||
Loading…
Reference in New Issue
Block a user