prole/docs/EMBEDDED-RESOURCES.md
chrisfu cbfe930b78 feat: add ncurses interface, build system, and embedded resources
Major feature additions and infrastructure improvements for the Prole
Database Installer, enabling command-line operation and packaged binary
distribution.

## Ncurses Terminal Interface

- Add installer/ncurses_ui.py: UI primitives (CursesWindow, TerminalConsole,
  NavFooter, InputField, Checkbox)
- Add installer/ncurses_installer.py: Complete terminal UI with all 11 screens
- Implement same screen flow as GUI (welcome, deps, network scan, env setup,
  kerberos, password, build, cluster, scripts, deploy, installer creation)
- Add keyboard navigation (arrows, hjkl, vim-style)
- Support both GUI and ncurses modes in single binary

## Automatic Display Detection

- Add has_display() function to detect GUI availability
- Auto-select GUI if display available, ncurses otherwise
- Add --gui and --no-gui command-line flags
- Fallback to ncurses on GUI failure

## Build System and Packaging

- Add Makefile with targets: build, package, clean, test, install
- Add scripts/generate_spec.py: PyInstaller spec generator
- Add installer.spec: PyInstaller configuration
- Automatic PNG to ICNS icon conversion
- Create self-contained macOS .app bundle with embedded icon
- Support both Intel (x86_64) and Apple Silicon (arm64)

## Embedded Resources

- Add get_resource_path() helper for PyInstaller compatibility
- Embed all images (proleIcon.png, proleLogo.png, proleLogoSepia.png)
- Embed prole-net/prole-scan binary (6.8 MB universal binary)
- Embed prole-app/dist/Prole Tools.app (12 MB app bundle)
- Embed prole-db/ Docker build context

## Writable Directory Fixes

- Create ~/.prole/build/prole-db/ for Docker builds (fixes read-only _MEIPASS)
- Create ~/.prole/scan/ for network scan output (fixes API call failures)
- Copy build context to writable location before Docker operations
- Run prole-scan from writable working directory

## Documentation

- docs/build-system.md: Complete build system guide
- docs/ncurses-installer.md: Ncurses interface documentation
- docs/RELEASE-NOTES.md: Feature overview and release notes
- docs/IMAGE-RESOURCES.md: Image resource management
- docs/EMBEDDED-RESOURCES.md: Binary and app bundle embedding
- docs/DOCKER-BUILD-FIX.md: Docker build hang solution
- docs/PROLE-HOME-DIRECTORY.md: ~/.prole directory structure
- BUILD.md: Quick build reference

## Key Changes

install.py:
- Add get_resource_path() for embedded resource resolution
- Update image paths to use get_resource_path()
- Update Docker build to use ~/.prole/build/prole-db/
- Update network scan to use ~/.prole/scan/
- Add display detection and mode selection
- Add --gui and --no-gui argument parsing

## Testing

All features tested and verified:
- Ncurses interface navigation
- Display auto-detection
- Resource path resolution
- Docker build from package
- Network scan from package
- Icon conversion and embedding

Package size: ~50-100 MB (includes Python runtime, all resources)
Disk usage: ~/.prole/ uses ~2-6 MB

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-19 23:10:37 -08:00

309 lines
7.7 KiB
Markdown

# Embedded Resources in Prole Installer
## Overview
The Prole Installer package includes several embedded resources that must be accessible both when running from source and when packaged as a standalone binary.
## Embedded Resources
### 1. Images (img/)
- **proleIcon.png** (1.5 MB) - Application icon
- **proleLogo.png** (2.2 MB) - Main logo
- **proleLogoSepia.png** (2.7 MB) - Sepia background for GUI
- **proleLogoBlueprint.png** (3.1 MB) - Blueprint variant
- **proleIconblueprint.png** (1.6 MB) - Blueprint icon variant
### 2. Binary Executables
- **prole-net/prole-scan** (6.8 MB) - Network scanner
- Universal binary (x86_64 + arm64)
- Used by network scan screen
- Detects Kerberos, Active Directory, etc.
### 3. Application Bundles
- **prole-app/dist/Prole Tools.app** (~12 MB) - Pre-built Prole Tools
- Complete macOS .app bundle
- Used by installer creation screen
- Can be copied to DMG or USB installer
## How Embedding Works
### Resource Path Resolution
All resources use the `get_resource_path()` helper function:
```python
def get_resource_path(relative_path):
"""Get absolute path to resource, works for dev and for PyInstaller."""
try:
# PyInstaller creates a temp folder and stores path in _MEIPASS
base_path = Path(sys._MEIPASS)
except AttributeError:
# Running from source
base_path = PROJECT_ROOT
return base_path / relative_path
```
### Usage Examples
**Image Loading:**
```python
bg_path = get_resource_path('img/proleLogoSepia.png')
if bg_path.exists():
image = Image.open(str(bg_path))
```
**Binary Execution:**
```python
scan_binary = get_resource_path("prole-net/prole-scan")
if scan_binary.exists():
process = subprocess.Popen([str(scan_binary)], ...)
```
**App Bundle Access:**
```python
app_src = get_resource_path('prole-app/dist/Prole Tools.app')
if app_src.exists():
# Copy to destination
shutil.copytree(app_src, dest)
```
## PyInstaller Configuration
### Spec File (scripts/generate_spec.py)
**Data Files:**
```python
datas = [
('installer', 'installer'),
('conf', 'conf'),
('etc', 'etc'),
('img', 'img'),
('docs', 'docs'),
('prole-app/dist/Prole Tools.app', 'prole-app/dist/Prole Tools.app'),
]
```
**Binaries:**
```python
binaries = [
('prole-net/prole-scan', 'prole-net'),
]
```
The `binaries` list ensures executable permissions are preserved.
## File Sizes
Total embedded resources: ~30-35 MB
Breakdown:
- Images: ~11 MB
- prole-scan: 6.8 MB
- Prole Tools.app: ~12 MB
- Other resources: ~5-10 MB
Final installer bundle: ~50-100 MB (includes Python runtime)
## Usage in Installer
### Network Scan Screen
The network scan screen uses `prole-scan` to detect services:
```python
scan_binary = get_resource_path("prole-net/prole-scan")
process = subprocess.Popen([str(scan_binary)],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True)
```
Output is captured in real-time and displayed in the scan results text widget.
### Installer Creation Screen
The installer creation screen copies Prole Tools.app to DMG:
```python
app_src = get_resource_path('prole-app/dist/Prole Tools.app')
if app_src.exists():
shutil.copytree(app_src, staging / 'Prole Tools.app')
```
## Testing
### Verify Resources from Source
```bash
python3 test_embedded_resources.sh
```
This tests:
- ✓ All image files exist
- ✓ prole-scan binary exists and is executable
- ✓ Prole Tools.app bundle exists
- ✓ Spec file includes all resources
- ✓ Resource sizes
### Verify Resources in Built Package
After building:
```bash
# Check extracted resources
./dist/prole-installer --help
# In another terminal, while installer is running:
ls -la /tmp/_MEI*/prole-net/
ls -la /tmp/_MEI*/img/
ls -la "/tmp/_MEI*/prole-app/dist/Prole Tools.app"
```
## Troubleshooting
### Binary Not Found Error
**Error:** `Scan binary not found at /var/folders/.../prole-net/prole-scan`
**Cause:** Binary not included in package or path not using `get_resource_path()`
**Solution:**
1. Verify spec includes binary: `grep prole-scan installer.spec`
2. Check code uses `get_resource_path()`: `grep "get_resource_path.*prole-scan" install.py`
3. Rebuild: `make clean && make package`
### Binary Not Executable
**Error:** `Permission denied` when running prole-scan
**Cause:** Binary permissions not preserved in package
**Solution:**
Ensure binary is in `binaries` list (not `datas`) in spec file:
```python
binaries = [
('prole-net/prole-scan', 'prole-net'), # Correct - preserves +x
]
# NOT in datas:
# datas = [('prole-net/prole-scan', 'prole-net')] # Wrong - loses +x
```
### App Bundle Not Found
**Error:** `Prole Tools.app not found`
**Cause:** App not built before packaging installer
**Solution:**
1. Build Prole Tools.app first (in prole-app directory)
2. Verify it exists: `ls "prole-app/dist/Prole Tools.app"`
3. Then build installer: `make package`
### Large Bundle Size
**Cause:** Including large binary files significantly increases bundle size
**Optimization Options:**
1. **Compress app bundle:**
```bash
cd prole-app/dist
zip -r "Prole Tools.zip" "Prole Tools.app"
# Include zip instead of .app
```
2. **Download on demand:**
Instead of embedding, download from server when needed
3. **Exclude debug symbols:**
Strip binaries before packaging:
```bash
strip prole-net/prole-scan
```
## Build Process
When running `make package`:
1. **Icon conversion:** PNG → ICNS
2. **Spec generation:** Creates installer.spec with all resources
3. **PyInstaller:**
- Analyzes install.py
- Collects dependencies
- Copies data files (preserves directory structure)
- Copies binaries (preserves execute permissions)
- Creates single-file executable
4. **Bundle creation:** Packages into .app with icon
### Resource Extraction at Runtime
When the packaged installer runs:
1. PyInstaller extracts resources to `/tmp/_MEI<random>/`
2. Sets `sys._MEIPASS` to extraction directory
3. `get_resource_path()` uses `_MEIPASS` to find resources
4. Resources deleted automatically when installer exits
## Security Considerations
### Code Signing
Embedded binaries should be code signed:
```bash
codesign --force --sign "Developer ID Application: Your Name" prole-net/prole-scan
```
Then build the installer - the signed binary will be included.
### Verification
Users can verify embedded binaries:
```bash
# Check signature of prole-scan after extraction
codesign --verify --verbose /tmp/_MEI*/prole-net/prole-scan
# Check installer bundle signature
codesign --verify --verbose "dist/Prole Installer.app"
```
## Future Enhancements
### Lazy Loading
For large resources, consider lazy loading:
```python
def get_prole_tools_app():
"""Download or extract Prole Tools.app only when needed."""
app_path = get_resource_path('prole-app/dist/Prole Tools.app')
if not app_path.exists():
# Download from server
download_prole_tools(app_path)
return app_path
```
### Compression
Compress large resources:
```python
# In spec file
datas = [
('prole-app/dist/Prole Tools.zip', 'prole-app/dist'), # Compressed
]
# In code
def extract_prole_tools():
zip_path = get_resource_path('prole-app/dist/Prole Tools.zip')
extract_dir = Path(tempfile.mkdtemp())
shutil.unpack_archive(zip_path, extract_dir)
return extract_dir / 'Prole Tools.app'
```
## References
- [PyInstaller Data Files](https://pyinstaller.org/en/stable/spec-files.html#adding-data-files)
- [PyInstaller Binaries](https://pyinstaller.org/en/stable/spec-files.html#adding-binaries)
- [macOS App Bundle Structure](https://developer.apple.com/library/archive/documentation/CoreFoundation/Conceptual/CFBundles/BundleTypes/BundleTypes.html)