prole/docs/build-system.md
chrisfu a069989315 Rename prole-db to knoe-db, add knoe-auth as cluster-internal KDC
Itemized changes:

1. knoe-auth: New cluster-internal KDC and SSO gateway service
   - Created etc/init_knoe_auth.sh based on init_kdc.sh with knoe-auth naming
   - Namespace defaults to SERVICE_NAMESPACE (knoe-system)
   - ConfigMap: knoe-auth-kdc-config, Secret: knoe-auth-secrets
   - Legacy cleanup removes old auth/dog/authority deployments

2. Orchestration: knoe-auth initializes before CloudNativePG
   - Updated prole.sh to insert init_knoe_auth.sh as step 2 (before CNPG)
   - Renumbered all subsequent initialization steps

3. Kong routing: Updated init_kong.sh to route to knoe-auth in SERVICE_NAMESPACE

4. Comment/reference updates for knoe-auth
   - Updated init_common_services.sh, init_service_layer.sh, init_kerberos.sh

5. prole-db renamed to knoe-db across the entire codebase
   - Renamed prole-db/ directory to knoe-db/
   - Renamed all prole-db Kubernetes manifests (deploy/opentofu, k8s/)
   - Renamed scripts: docker-root-knoe-db.sh, docker-run-knoe-db.sh, test-cnpg-knoe-db.sh
   - Renamed etc/init_prole-db-reset.sh to etc/init_knoe-db-reset.sh
   - Renamed etc/prole-db-passwwd.sh to etc/knoe-db-passwwd.sh
   - Renamed mock_val counterparts accordingly
   - Renamed tests/etc/test_init_prole-db-reset.sh to test_init_knoe-db-reset.sh
   - Renamed docs/prole-db-documentation-mcp-architecture.md to knoe-db variant
   - Renamed modes/k3d/prole-db/ to modes/k3d/knoe-db/
   - Renamed prole-db.iml to knoe-db.iml

6. Configuration updates
   - Updated conf/dev, conf/prod, conf/test, conf/service prole.cfg files
   - Updated conf/port-mapping.cfg
   - Updated etc/prole_cfg.sh and mock_val/prole_cfg.sh
   - Updated service/prole.cfg

7. Kubernetes manifests and deploy configuration
   - Updated deploy/opentofu/k3s ArgoCD application YAMLs
   - Updated kong-configmap.yaml and kustomization.yaml
   - Updated k3s/kong-config.yml and prole-resources.yaml
   - Updated prole-mssql-db deployment YAMLs
   - Updated supabase helm render and deploy scripts

8. Infrastructure and GCP Terraform
   - Updated deploy/gcp/terraform: folders, groups, IAM, service-projects

9. Python/installer code updates
   - Updated knoe/core: actions, build_context, controller, env, milestones
   - Updated knoe/milestone.py
   - Updated knoe/ui/screens: cfg, database, database_options, deploy, docker,
     navigation, security, services, validate
   - Updated knoe.spec, status.py

10. Shell script updates
    - Updated etc/: build_db, init_cloudnative_pg, init_cnpg_backup,
      init_db_manager, init_forgejo, init_gitlab, init_monitoring, init_openbao,
      init_port_forwards, init_postgrest, init_supabase_ports, status
    - Updated mock_val/ counterparts for all above scripts
    - Updated prole-net/init-prole-dns.sh
    - Updated bin/prole-kpf.sh, gitea/deploy.sh, supabase/deploy.sh

11. Test updates
    - Updated tests/etc/: test_init_cloudnative_pg*, test_init_cnpg_backup*,
      test_init_kdc*, test_init_kerberos*, test_init_kong*, test_prole_cfg*
    - Updated tests/installer/: test_actions_helpers, test_cfg_save_kubecontext,
      test_controller, test_core_classes, test_milestones, test_milestones_extended,
      test_namespace_propagation
    - Updated tests/: test_database_options, test_navigation,
      test_render_supabase_hostname, test_docker_build_fix,
      test_all_prole_home_fixes, silent_install_test, final_test

12. Documentation updates
    - Updated docs/: DOCKER-BUILD-FIX, PROLE-CFG-SECRETS, PROLE-HOME-DIRECTORY,
      build-system, patent
    - Updated scan/network_description.txt
    - Updated pom.xml

13. Miscellaneous script updates
    - Updated root-level: _adopt_replica_pvcs, _fix_replica_merlin, _import_pi,
      _patch_cluster, _prebind_pvcs, _rebind_d002, _rebind_d002b, test_resolve
    - Updated scripts/generate_spec.py

Co-authored-by: Junie <junie@jetbrains.com>
2026-03-22 22:16:21 -07:00

350 lines
9.4 KiB
Markdown

# Prole Installer Build System
## Overview
The Prole Database Installer includes a comprehensive build system that creates self-contained, universal binaries for distribution. The build system produces a macOS application bundle (.app) that:
- **Double-click launch**: Opens the GUI installer when launched from Finder
- **Command-line capable**: Can be run from terminal with full argument support
- **Auto-detects display**: Automatically switches to ncurses mode when no GUI is available
- **Self-contained**: Includes all dependencies and resources
- **Icon embedded**: Uses the Prole icon for the application bundle
## Quick Start
```bash
# Install build dependencies
make install
# Build everything (creates .app bundle)
make package
# Test the build
make test
# Clean build artifacts
make clean
```
## Makefile Targets
### `make help` (default)
Display help information about available targets.
### `make install`
Install required build dependencies:
- PyInstaller (for creating standalone executables)
- Pillow (for image processing, if needed)
### `make build`
Build the static binary using PyInstaller:
1. Generates icon in .icns format
2. Creates PyInstaller spec file
3. Builds standalone executable
4. Creates macOS .app bundle
**Output:**
- `dist/prole-installer` - Standalone executable
- `dist/Prole Installer.app` - macOS application bundle
### `make package`
Alias for `make build` - creates the complete package.
### `make test`
Run basic tests on the built binary:
- Test `--help` output
- Verify app bundle structure
- Test executable within app bundle
### `make clean`
Remove all build artifacts:
- `build/` directory
- `dist/` directory
- `installer.spec` file
- Python cache files
### `make rebuild`
Clean and rebuild from scratch (`make clean build`).
### `make spec`
Generate only the PyInstaller spec file (intermediate step).
## Build Process Details
### 1. Icon Conversion
The build system converts `img/proleIcon.png` to macOS `.icns` format with multiple resolutions:
- 16x16, 32x32 (standard and @2x)
- 128x128, 256x256 (standard and @2x)
- 512x512, 1024x1024 (standard and @2x)
**Tool:** macOS `sips` and `iconutil` commands
**Output:** `build/prole.icns`
### 2. Spec File Generation
A Python script (`scripts/generate_spec.py`) creates the PyInstaller specification:
**Included Data Files:**
- `installer/` - Installer Python modules
- `conf/` - Configuration files
- `etc/` - Scripts and utilities
- `img/` - Images and icons (including proleIcon.png, proleLogoSepia.png, proleLogo.png)
- `docs/` - Documentation
- `knoe-db/` - Docker build context for PostgreSQL database
- `prole-app/dist/Prole Tools.app` - Pre-built Prole Tools application bundle (entire .app)
**Included Binaries:**
- `prole-net/prole-agent` - Network scanner binary (universal: x86_64 + arm64, 6.8 MB)
**Resource Path Resolution:**
The installer uses a `get_resource_path()` helper function that automatically resolves paths correctly whether running from source or from a PyInstaller bundle:
- **From source**: Uses `PROJECT_ROOT / relative_path`
- **From bundle**: Uses `sys._MEIPASS / relative_path` (PyInstaller temp directory)
This ensures all images and resources are found correctly in both development and production.
**Docker Build Context:**
Docker builds require a writable directory. When running from a PyInstaller bundle, the extracted resources are in a read-only temporary directory. To solve this:
- Docker build context is copied to `$HOME/.prole/build/knoe-db/`
- This provides a writable location for Docker to operate
- See [DOCKER-BUILD-FIX.md](DOCKER-BUILD-FIX.md) for details
**Hidden Imports:**
PyInstaller can't always auto-detect imports, so we explicitly include:
- All installer modules
- curses/ncurses libraries
**Output:** `installer.spec`
### 3. Binary Creation
PyInstaller processes the spec file to create:
**Single-file Executable:**
- All Python code and dependencies bundled
- Extracts to temporary directory at runtime
- Console-enabled (works with both GUI and ncurses)
**macOS App Bundle:**
- Standard `.app` directory structure
- Icon embedded in bundle
- Info.plist with metadata
- Executable in `Contents/MacOS/`
## Using the Built Application
### GUI Mode (Double-click)
Simply double-click `Prole Installer.app` in Finder to launch the graphical installer.
### Command-line Mode
```bash
# Run with auto-detection (GUI if available, otherwise ncurses)
./dist/Prole\ Installer.app/Contents/MacOS/prole-installer
# Force ncurses mode
./dist/Prole\ Installer.app/Contents/MacOS/prole-installer --no-gui
# Force GUI mode
./dist/Prole\ Installer.app/Contents/MacOS/prole-installer --gui
# Show help
./dist/Prole\ Installer.app/Contents/MacOS/prole-installer --help
```
### Installation to /Applications
```bash
cp -r "dist/Prole Installer.app" /Applications/
```
After installation, the app is available:
- In Finder under Applications
- Via Spotlight search
- From command line as `/Applications/Prole\ Installer.app/Contents/MacOS/prole-installer`
## Display Auto-detection
The installer automatically detects whether a GUI display is available:
1. **macOS**: Attempts to create a Tk root window
2. **Linux/Unix**: Checks for `DISPLAY` environment variable
3. **Fallback**: Uses ncurses mode if GUI unavailable
**Override behavior:**
- `--gui` flag: Force GUI mode (fails if no display)
- `--no-gui` flag: Force ncurses mode
## Architecture Support
The build system detects the host architecture:
- **Apple Silicon (arm64)**: Native ARM64 binary
- **Intel (x86_64)**: Native x86_64 binary
For universal binaries supporting both architectures, you would need to:
1. Build on arm64 machine
2. Build on x86_64 machine
3. Use `lipo` to combine binaries
## Build Requirements
### System Requirements
- macOS 10.13 or later
- Python 3.8 or later
- Make build system
### Python Dependencies
- pyinstaller >= 5.0
- pillow (optional, for advanced image processing)
- All runtime dependencies of install.py
### macOS Developer Tools
- Command Line Tools (for `sips`, `iconutil`, `codesign`)
Install with: `xcode-select --install`
## Troubleshooting
### Build Fails with "No module named 'installer'"
**Cause:** PyInstaller can't find the installer package.
**Solution:** Ensure you're running from project root and `installer/` directory exists.
### Icon Not Showing in App Bundle
**Cause:** Icon conversion failed or .icns file is invalid.
**Solution:**
```bash
make clean
make build/prole.icns
file build/prole.icns # Should say "Mac OS X icon"
```
### Binary Won't Run on Other Machines
**Cause:** PyInstaller doesn't include all dependencies, or architecture mismatch.
**Solutions:**
- Check target machine has same or newer macOS version
- Verify architecture matches (arm64 vs x86_64)
- Check Console.app for error messages
- Rebuild with `--debug` in spec file for verbose output
### GUI Doesn't Launch When Double-clicking
**Cause:** App not signed, or console mode preventing GUI launch.
**Solutions:**
- Right-click → Open (first time only, to bypass Gatekeeper)
- Check that `console=True` in spec allows both modes
- Run from Terminal to see error messages
### Ncurses Mode Garbled Output
**Cause:** Terminal doesn't support required features.
**Solution:** Use a modern terminal emulator (Terminal.app, iTerm2, etc.)
## Advanced Configuration
### Custom Icon
Replace `img/proleIcon.png` with your icon (PNG format, preferably 1024x1024).
### Additional Data Files
Edit `scripts/generate_spec.py` and add to `datas` list:
```python
datas = [
('installer', 'installer'),
('your_data', 'your_data'), # Add this
]
```
### Code Signing
For distribution outside development:
1. Get Apple Developer certificate
2. Modify Makefile to add signing step:
```makefile
codesign --deep --force --verify --verbose \
--sign "Developer ID Application: Your Name" \
"dist/Prole Installer.app"
```
### Notarization
For Gatekeeper approval on macOS 10.15+:
1. Sign the app
2. Create a DMG or ZIP
3. Submit to Apple:
```bash
xcrun notarytool submit prole-installer.zip \
--apple-id your@email.com \
--password app-specific-password \
--team-id TEAMID
```
## Distribution
### DMG Creation
```bash
hdiutil create -volname "Prole Installer" \
-srcfolder "dist/Prole Installer.app" \
-ov -format UDZO \
prole-installer.dmg
```
### ZIP Archive
```bash
cd dist
zip -r ../prole-installer.zip "Prole Installer.app"
```
## Performance Considerations
**Build Time:** 30-60 seconds typical
**Binary Size:** 50-100 MB (includes Python runtime + dependencies)
**Startup Time:** 1-3 seconds (PyInstaller extraction overhead)
To improve startup time, consider using PyInstaller's `--onedir` mode instead of `--onefile`, though this increases distribution complexity.
## CI/CD Integration
Example GitHub Actions workflow:
```yaml
name: Build Installer
on: [push]
jobs:
build:
runs-on: macos-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: '3.11'
- run: make install
- run: make package
- uses: actions/upload-artifact@v2
with:
name: prole-installer
path: dist/Prole Installer.app
```
## References
- [PyInstaller Documentation](https://pyinstaller.org/)
- [macOS App Bundle Structure](https://developer.apple.com/library/archive/documentation/CoreFoundation/Conceptual/CFBundles/BundleTypes/BundleTypes.html)
- [macOS Code Signing](https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution)