prole/docs/PROLE-HOME-DIRECTORY.md
chrisfu a069989315 Rename prole-db to knoe-db, add knoe-auth as cluster-internal KDC
Itemized changes:

1. knoe-auth: New cluster-internal KDC and SSO gateway service
   - Created etc/init_knoe_auth.sh based on init_kdc.sh with knoe-auth naming
   - Namespace defaults to SERVICE_NAMESPACE (knoe-system)
   - ConfigMap: knoe-auth-kdc-config, Secret: knoe-auth-secrets
   - Legacy cleanup removes old auth/dog/authority deployments

2. Orchestration: knoe-auth initializes before CloudNativePG
   - Updated prole.sh to insert init_knoe_auth.sh as step 2 (before CNPG)
   - Renumbered all subsequent initialization steps

3. Kong routing: Updated init_kong.sh to route to knoe-auth in SERVICE_NAMESPACE

4. Comment/reference updates for knoe-auth
   - Updated init_common_services.sh, init_service_layer.sh, init_kerberos.sh

5. prole-db renamed to knoe-db across the entire codebase
   - Renamed prole-db/ directory to knoe-db/
   - Renamed all prole-db Kubernetes manifests (deploy/opentofu, k8s/)
   - Renamed scripts: docker-root-knoe-db.sh, docker-run-knoe-db.sh, test-cnpg-knoe-db.sh
   - Renamed etc/init_prole-db-reset.sh to etc/init_knoe-db-reset.sh
   - Renamed etc/prole-db-passwwd.sh to etc/knoe-db-passwwd.sh
   - Renamed mock_val counterparts accordingly
   - Renamed tests/etc/test_init_prole-db-reset.sh to test_init_knoe-db-reset.sh
   - Renamed docs/prole-db-documentation-mcp-architecture.md to knoe-db variant
   - Renamed modes/k3d/prole-db/ to modes/k3d/knoe-db/
   - Renamed prole-db.iml to knoe-db.iml

6. Configuration updates
   - Updated conf/dev, conf/prod, conf/test, conf/service prole.cfg files
   - Updated conf/port-mapping.cfg
   - Updated etc/prole_cfg.sh and mock_val/prole_cfg.sh
   - Updated service/prole.cfg

7. Kubernetes manifests and deploy configuration
   - Updated deploy/opentofu/k3s ArgoCD application YAMLs
   - Updated kong-configmap.yaml and kustomization.yaml
   - Updated k3s/kong-config.yml and prole-resources.yaml
   - Updated prole-mssql-db deployment YAMLs
   - Updated supabase helm render and deploy scripts

8. Infrastructure and GCP Terraform
   - Updated deploy/gcp/terraform: folders, groups, IAM, service-projects

9. Python/installer code updates
   - Updated knoe/core: actions, build_context, controller, env, milestones
   - Updated knoe/milestone.py
   - Updated knoe/ui/screens: cfg, database, database_options, deploy, docker,
     navigation, security, services, validate
   - Updated knoe.spec, status.py

10. Shell script updates
    - Updated etc/: build_db, init_cloudnative_pg, init_cnpg_backup,
      init_db_manager, init_forgejo, init_gitlab, init_monitoring, init_openbao,
      init_port_forwards, init_postgrest, init_supabase_ports, status
    - Updated mock_val/ counterparts for all above scripts
    - Updated prole-net/init-prole-dns.sh
    - Updated bin/prole-kpf.sh, gitea/deploy.sh, supabase/deploy.sh

11. Test updates
    - Updated tests/etc/: test_init_cloudnative_pg*, test_init_cnpg_backup*,
      test_init_kdc*, test_init_kerberos*, test_init_kong*, test_prole_cfg*
    - Updated tests/installer/: test_actions_helpers, test_cfg_save_kubecontext,
      test_controller, test_core_classes, test_milestones, test_milestones_extended,
      test_namespace_propagation
    - Updated tests/: test_database_options, test_navigation,
      test_render_supabase_hostname, test_docker_build_fix,
      test_all_prole_home_fixes, silent_install_test, final_test

12. Documentation updates
    - Updated docs/: DOCKER-BUILD-FIX, PROLE-CFG-SECRETS, PROLE-HOME-DIRECTORY,
      build-system, patent
    - Updated scan/network_description.txt
    - Updated pom.xml

13. Miscellaneous script updates
    - Updated root-level: _adopt_replica_pvcs, _fix_replica_merlin, _import_pi,
      _patch_cluster, _prebind_pvcs, _rebind_d002, _rebind_d002b, test_resolve
    - Updated scripts/generate_spec.py

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 22:16:21 -07:00

7.4 KiB
Raw Blame History

Prole Home Directory Structure

Overview

The Prole Installer uses $PROLE_HOME/ (defaulting to $HOME/.prole/) for writable storage when running from a packaged binary. This is necessary because PyInstaller extracts resources to a read-only temporary directory.

Directory Structure

$PROLE_HOME/
├── build/               # Docker build contexts
│   └── knoe-db/        # PostgreSQL Docker build
│       ├── Dockerfile
│       ├── conf/
│       └── ...
└── scan/               # Network scan output and cache
    └── (scan results, temporary files)

Purpose of Each Directory

build/

Purpose: Writable location for Docker build contexts

Why: PyInstaller extracts resources to /tmp/_MEIxxxxxx/ which is read-only. Docker requires a writable build context directory to create images.

Usage:

prole_home = resolve_prole_home()
build_dir = prole_home / "build" / "knoe-db"
build_dir.mkdir(parents=True, exist_ok=True)

# Copy build context from embedded resources
source_dir = get_resource_path("knoe-db")
shutil.copytree(source_dir, build_dir)

# Docker build
subprocess.Popen(['docker', 'build', '-t', 'knoe-db:TAG', '.'], cwd=build_dir)

Contents:

  • Complete copy of knoe-db/ directory
  • Dockerfile and all dependencies
  • Refreshed on each build (old content removed)

Size: ~1-5 MB

scan/

Purpose: Writable working directory for network scan operations

Why: The prole-agent binary may need to write output files, cache data, or store temporary results. Running from a read-only directory causes failures.

Usage:

prole_home = resolve_prole_home()
scan_dir = prole_home / "scan"
scan_dir.mkdir(parents=True, exist_ok=True)

# Run scan with writable cwd
subprocess.Popen([str(scan_binary)], cwd=str(scan_dir))

Contents:

  • Network scan results (temporary)
  • Ollama API interaction cache
  • Any intermediate files created by prole-agent

Size: Varies, typically < 1 MB

Creation and Cleanup

Automatic Creation

All directories are created automatically when needed:

# Build directory
(resolve_prole_home() / "build" / "knoe-db").mkdir(parents=True, exist_ok=True)

# Scan directory
(resolve_prole_home() / "scan").mkdir(parents=True, exist_ok=True)

Manual Cleanup

To remove all Prole working directories:

rm -rf ~/.prole

Or from Python:

import shutil
from pathlib import Path

prole_home = resolve_prole_home()
if prole_home.exists():
    shutil.rmtree(prole_home)

Automatic Cleanup (Future)

Consider adding cleanup options to the installer:

def cleanup_prole_home():
    """Clean up .prole working directories."""
    prole_home = Path.home() / ".prole"
    if prole_home.exists():
        # Keep or remove based on user preference
        if messagebox.askyesno("Cleanup", "Remove temporary files?"):
            shutil.rmtree(prole_home)

Disk Space

Expected Usage

Directory Size When Created Persistent
build/knoe-db/ 1-5 MB First Docker build Yes
scan/ < 1 MB First network scan Yes
Total ~2-6 MB On first use Yes

Growth

  • build/ Overwritten on each build, doesn't grow
  • scan/ May accumulate cache files over time

Troubleshooting

Permission Errors

Error: Permission denied creating .prole directory

Cause: Home directory not writable

Solution:

ls -ld ~
chmod u+w ~

Disk Space Issues

Error: No space left on device

Cause: Disk full

Solution:

df -h ~
rm -rf ~/.prole  # Free up space

Stale Build Context

Issue: Docker build uses old files

Solution:

# Build directory is refreshed automatically
if build_dir.exists():
    shutil.rmtree(build_dir)
shutil.copytree(source_dir, build_dir)

Security Considerations

File Permissions

The .prole directory inherits user's home directory permissions:

ls -ld ~/.prole
# drwxr-xr-x  user  group  ~/.prole

Only the user should have write access.

Sensitive Data

Avoid storing sensitive data in .prole/:

  • ✓ Build contexts (public)
  • ✓ Scan results (network info, semi-sensitive)
  • ✗ Passwords, keys, credentials

Cleanup on Uninstall

If distributing the installer, consider:

  1. Document cleanup:

    To completely remove Prole:
    1. Delete the app: rm -rf /Applications/Prole\ Installer.app
    2. Clean up data: rm -rf ~/.prole
    
  2. Provide uninstall script:

    #!/bin/bash
    # uninstall-prole.sh
    rm -rf /Applications/Prole\ Installer.app
    rm -rf ~/.prole
    echo "Prole uninstalled"
    

Docker Build Hang

Problem: Docker build hangs when running from packaged binary

Solution: Copy build context to ~/.prole/build/knoe-db/

See: DOCKER-BUILD-FIX.md

Network Scan Failure

Problem: Network scan fails to make API calls from packaged binary

Solution: Run scan with cwd=~/.prole/scan

Reason: Scan binary needs writable directory for output/cache

Implementation Details

Code Location

Build directory creation: install.py, run_db_build() method (line ~1791)

prole_home = Path.home() / ".prole"
build_dir = prole_home / "build" / "knoe-db"
build_dir.mkdir(parents=True, exist_ok=True)

Scan directory creation: install.py, network scan worker (line ~1102)

prole_home = Path.home() / ".prole"
scan_dir = prole_home / "scan"
scan_dir.mkdir(parents=True, exist_ok=True)

Resource Path Resolution

Both use get_resource_path() to find embedded resources:

def get_resource_path(relative_path):
    """Get absolute path to resource, works for dev and for PyInstaller."""
    try:
        base_path = Path(sys._MEIPASS)  # PyInstaller temp dir
    except AttributeError:
        base_path = PROJECT_ROOT  # Running from source

    return base_path / relative_path

Future Enhancements

Persistent Cache

Store network scan results between runs:

scan_cache = Path.home() / ".prole" / "scan" / "cache.json"
if scan_cache.exists():
    # Load previous scan
    results = json.loads(scan_cache.read_text())
else:
    # Run new scan
    results = run_scan()
    scan_cache.write_text(json.dumps(results))

Build Artifacts

Store built Docker images as tarballs:

image_cache = Path.home() / ".prole" / "build" / "knoe-db-TAG.tar"
if not image_cache.exists():
    # Build and save
    subprocess.run(['docker', 'build', '-t', 'knoe-db:TAG', '.'])
    subprocess.run(['docker', 'save', '-o', str(image_cache), 'knoe-db:TAG'])
else:
    # Load cached image
    subprocess.run(['docker', 'load', '-i', str(image_cache)])

Configuration Storage

Store user preferences:

config_file = Path.home() / ".prole" / "config.json"
config = {
    'cluster_env': 'development',
    'kerberos_enabled': False,
    'last_scan': '2025-01-19',
}
config_file.write_text(json.dumps(config, indent=2))

Summary

The ~/.prole/ directory provides:

  • ✓ Writable storage for packaged binary operations
  • ✓ Separate from extracted read-only resources
  • ✓ User-specific, secure location
  • ✓ Easy to clean up manually
  • ✓ Small disk footprint (< 10 MB)

This architecture ensures the installer works correctly whether running from source or from a PyInstaller package.