prole/docs/EMBEDDED-RESOURCES.md
chrisfu cbfe930b78 feat: add ncurses interface, build system, and embedded resources
Major feature additions and infrastructure improvements for the Prole
Database Installer, enabling command-line operation and packaged binary
distribution.

## Ncurses Terminal Interface

- Add installer/ncurses_ui.py: UI primitives (CursesWindow, TerminalConsole,
  NavFooter, InputField, Checkbox)
- Add installer/ncurses_installer.py: Complete terminal UI with all 11 screens
- Implement same screen flow as GUI (welcome, deps, network scan, env setup,
  kerberos, password, build, cluster, scripts, deploy, installer creation)
- Add keyboard navigation (arrows, hjkl, vim-style)
- Support both GUI and ncurses modes in single binary

## Automatic Display Detection

- Add has_display() function to detect GUI availability
- Auto-select GUI if display available, ncurses otherwise
- Add --gui and --no-gui command-line flags
- Fallback to ncurses on GUI failure

## Build System and Packaging

- Add Makefile with targets: build, package, clean, test, install
- Add scripts/generate_spec.py: PyInstaller spec generator
- Add installer.spec: PyInstaller configuration
- Automatic PNG to ICNS icon conversion
- Create self-contained macOS .app bundle with embedded icon
- Support both Intel (x86_64) and Apple Silicon (arm64)

## Embedded Resources

- Add get_resource_path() helper for PyInstaller compatibility
- Embed all images (proleIcon.png, proleLogo.png, proleLogoSepia.png)
- Embed prole-net/prole-scan binary (6.8 MB universal binary)
- Embed prole-app/dist/Prole Tools.app (12 MB app bundle)
- Embed prole-db/ Docker build context

## Writable Directory Fixes

- Create ~/.prole/build/prole-db/ for Docker builds (fixes read-only _MEIPASS)
- Create ~/.prole/scan/ for network scan output (fixes API call failures)
- Copy build context to writable location before Docker operations
- Run prole-scan from writable working directory

## Documentation

- docs/build-system.md: Complete build system guide
- docs/ncurses-installer.md: Ncurses interface documentation
- docs/RELEASE-NOTES.md: Feature overview and release notes
- docs/IMAGE-RESOURCES.md: Image resource management
- docs/EMBEDDED-RESOURCES.md: Binary and app bundle embedding
- docs/DOCKER-BUILD-FIX.md: Docker build hang solution
- docs/PROLE-HOME-DIRECTORY.md: ~/.prole directory structure
- BUILD.md: Quick build reference

## Key Changes

install.py:
- Add get_resource_path() for embedded resource resolution
- Update image paths to use get_resource_path()
- Update Docker build to use ~/.prole/build/prole-db/
- Update network scan to use ~/.prole/scan/
- Add display detection and mode selection
- Add --gui and --no-gui argument parsing

## Testing

All features tested and verified:
- Ncurses interface navigation
- Display auto-detection
- Resource path resolution
- Docker build from package
- Network scan from package
- Icon conversion and embedding

Package size: ~50-100 MB (includes Python runtime, all resources)
Disk usage: ~/.prole/ uses ~2-6 MB

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-19 23:10:37 -08:00

7.7 KiB

Embedded Resources in Prole Installer

Overview

The Prole Installer package includes several embedded resources that must be accessible both when running from source and when packaged as a standalone binary.

Embedded Resources

1. Images (img/)

  • proleIcon.png (1.5 MB) - Application icon
  • proleLogo.png (2.2 MB) - Main logo
  • proleLogoSepia.png (2.7 MB) - Sepia background for GUI
  • proleLogoBlueprint.png (3.1 MB) - Blueprint variant
  • proleIconblueprint.png (1.6 MB) - Blueprint icon variant

2. Binary Executables

  • prole-net/prole-scan (6.8 MB) - Network scanner
    • Universal binary (x86_64 + arm64)
    • Used by network scan screen
    • Detects Kerberos, Active Directory, etc.

3. Application Bundles

  • prole-app/dist/Prole Tools.app (~12 MB) - Pre-built Prole Tools
    • Complete macOS .app bundle
    • Used by installer creation screen
    • Can be copied to DMG or USB installer

How Embedding Works

Resource Path Resolution

All resources use the get_resource_path() helper function:

def get_resource_path(relative_path):
    """Get absolute path to resource, works for dev and for PyInstaller."""
    try:
        # PyInstaller creates a temp folder and stores path in _MEIPASS
        base_path = Path(sys._MEIPASS)
    except AttributeError:
        # Running from source
        base_path = PROJECT_ROOT
    return base_path / relative_path

Usage Examples

Image Loading:

bg_path = get_resource_path('img/proleLogoSepia.png')
if bg_path.exists():
    image = Image.open(str(bg_path))

Binary Execution:

scan_binary = get_resource_path("prole-net/prole-scan")
if scan_binary.exists():
    process = subprocess.Popen([str(scan_binary)], ...)

App Bundle Access:

app_src = get_resource_path('prole-app/dist/Prole Tools.app')
if app_src.exists():
    # Copy to destination
    shutil.copytree(app_src, dest)

PyInstaller Configuration

Spec File (scripts/generate_spec.py)

Data Files:

datas = [
    ('installer', 'installer'),
    ('conf', 'conf'),
    ('etc', 'etc'),
    ('img', 'img'),
    ('docs', 'docs'),
    ('prole-app/dist/Prole Tools.app', 'prole-app/dist/Prole Tools.app'),
]

Binaries:

binaries = [
    ('prole-net/prole-scan', 'prole-net'),
]

The binaries list ensures executable permissions are preserved.

File Sizes

Total embedded resources: ~30-35 MB

Breakdown:

  • Images: ~11 MB
  • prole-scan: 6.8 MB
  • Prole Tools.app: ~12 MB
  • Other resources: ~5-10 MB

Final installer bundle: ~50-100 MB (includes Python runtime)

Usage in Installer

Network Scan Screen

The network scan screen uses prole-scan to detect services:

scan_binary = get_resource_path("prole-net/prole-scan")
process = subprocess.Popen([str(scan_binary)],
                         stdout=subprocess.PIPE,
                         stderr=subprocess.STDOUT,
                         text=True)

Output is captured in real-time and displayed in the scan results text widget.

Installer Creation Screen

The installer creation screen copies Prole Tools.app to DMG:

app_src = get_resource_path('prole-app/dist/Prole Tools.app')
if app_src.exists():
    shutil.copytree(app_src, staging / 'Prole Tools.app')

Testing

Verify Resources from Source

python3 test_embedded_resources.sh

This tests:

  • ✓ All image files exist
  • ✓ prole-scan binary exists and is executable
  • ✓ Prole Tools.app bundle exists
  • ✓ Spec file includes all resources
  • ✓ Resource sizes

Verify Resources in Built Package

After building:

# Check extracted resources
./dist/prole-installer --help

# In another terminal, while installer is running:
ls -la /tmp/_MEI*/prole-net/
ls -la /tmp/_MEI*/img/
ls -la "/tmp/_MEI*/prole-app/dist/Prole Tools.app"

Troubleshooting

Binary Not Found Error

Error: Scan binary not found at /var/folders/.../prole-net/prole-scan

Cause: Binary not included in package or path not using get_resource_path()

Solution:

  1. Verify spec includes binary: grep prole-scan installer.spec
  2. Check code uses get_resource_path(): grep "get_resource_path.*prole-scan" install.py
  3. Rebuild: make clean && make package

Binary Not Executable

Error: Permission denied when running prole-scan

Cause: Binary permissions not preserved in package

Solution: Ensure binary is in binaries list (not datas) in spec file:

binaries = [
    ('prole-net/prole-scan', 'prole-net'),  # Correct - preserves +x
]
# NOT in datas:
# datas = [('prole-net/prole-scan', 'prole-net')]  # Wrong - loses +x

App Bundle Not Found

Error: Prole Tools.app not found

Cause: App not built before packaging installer

Solution:

  1. Build Prole Tools.app first (in prole-app directory)
  2. Verify it exists: ls "prole-app/dist/Prole Tools.app"
  3. Then build installer: make package

Large Bundle Size

Cause: Including large binary files significantly increases bundle size

Optimization Options:

  1. Compress app bundle:

    cd prole-app/dist
    zip -r "Prole Tools.zip" "Prole Tools.app"
    # Include zip instead of .app
    
  2. Download on demand: Instead of embedding, download from server when needed

  3. Exclude debug symbols: Strip binaries before packaging:

    strip prole-net/prole-scan
    

Build Process

When running make package:

  1. Icon conversion: PNG → ICNS
  2. Spec generation: Creates installer.spec with all resources
  3. PyInstaller:
    • Analyzes install.py
    • Collects dependencies
    • Copies data files (preserves directory structure)
    • Copies binaries (preserves execute permissions)
    • Creates single-file executable
  4. Bundle creation: Packages into .app with icon

Resource Extraction at Runtime

When the packaged installer runs:

  1. PyInstaller extracts resources to /tmp/_MEI<random>/
  2. Sets sys._MEIPASS to extraction directory
  3. get_resource_path() uses _MEIPASS to find resources
  4. Resources deleted automatically when installer exits

Security Considerations

Code Signing

Embedded binaries should be code signed:

codesign --force --sign "Developer ID Application: Your Name" prole-net/prole-scan

Then build the installer - the signed binary will be included.

Verification

Users can verify embedded binaries:

# Check signature of prole-scan after extraction
codesign --verify --verbose /tmp/_MEI*/prole-net/prole-scan

# Check installer bundle signature
codesign --verify --verbose "dist/Prole Installer.app"

Future Enhancements

Lazy Loading

For large resources, consider lazy loading:

def get_prole_tools_app():
    """Download or extract Prole Tools.app only when needed."""
    app_path = get_resource_path('prole-app/dist/Prole Tools.app')
    if not app_path.exists():
        # Download from server
        download_prole_tools(app_path)
    return app_path

Compression

Compress large resources:

# In spec file
datas = [
    ('prole-app/dist/Prole Tools.zip', 'prole-app/dist'),  # Compressed
]

# In code
def extract_prole_tools():
    zip_path = get_resource_path('prole-app/dist/Prole Tools.zip')
    extract_dir = Path(tempfile.mkdtemp())
    shutil.unpack_archive(zip_path, extract_dir)
    return extract_dir / 'Prole Tools.app'

References