prole/docs/build-system.md

Knoe Installer Build System

Overview

The Knoe Database Installer includes a comprehensive build system that creates self-contained, universal binaries for distribution. The build system produces a macOS application bundle (.app) that:

• Double-click launch: Opens the GUI installer when launched from Finder
• Command-line capable: Can be run from terminal with full argument support
• Auto-detects display: Automatically switches to ncurses mode when no GUI is available
• Self-contained: Includes all dependencies and resources
• Icon embedded: Uses the Knoe icon for the application bundle

Quick Start

# Install build dependencies
make install

# Build everything (creates .app bundle)
make package

# Test the build
make test

# Clean build artifacts
make clean

Makefile Targets

make help (default)

Display help information about available targets.

make install

Install required build dependencies:

• PyInstaller (for creating standalone executables)
• Pillow (for image processing, if needed)

make build

Build the static binary using PyInstaller:

1. Generates icon in .icns format
2. Creates PyInstaller spec file
3. Builds standalone executable
4. Creates macOS .app bundle

Output:

• dist/knoe-installer - Standalone executable
• dist/Knoe Installer.app - macOS application bundle

make package

Alias for make build - creates the complete package.

make test

Run basic tests on the built binary:

• Test --help output
• Verify app bundle structure
• Test executable within app bundle

make clean

Remove all build artifacts:

• build/ directory
• dist/ directory
• installer.spec file
• Python cache files

make rebuild

Clean and rebuild from scratch (make clean build).

make spec

Generate only the PyInstaller spec file (intermediate step).

Build Process Details

1. Icon Conversion

The build system converts img/knoeIcon.png to macOS .icns format with multiple resolutions:

• 16x16, 32x32 (standard and @2x)
• 128x128, 256x256 (standard and @2x)
• 512x512, 1024x1024 (standard and @2x)

Tool: macOS sips and iconutil commands

Output: build/knoe.icns

2. Spec File Generation

A Python script (scripts/generate_spec.py) creates the PyInstaller specification:

Included Data Files:

• installer/ - Installer Python modules
• conf/ - Configuration files
• etc/ - Scripts and utilities
• img/ - Images and icons (including knoeIcon.png, knoeLogoSepia.png, knoeLogo.png)
• docs/ - Documentation
• knoe-db/ - Docker build context for PostgreSQL database
• knoe-app/dist/Knoe Tools.app - Pre-built Knoe Tools application bundle (entire .app)

Included Binaries:

• scan/network-agent - Network scanner binary (universal: x86_64 + arm64, 6.8 MB)

Resource Path Resolution: The installer uses a get_resource_path() helper function that automatically resolves paths correctly whether running from source or from a PyInstaller bundle:

• From source: Uses PROJECT_ROOT / relative_path
• From bundle: Uses sys._MEIPASS / relative_path (PyInstaller temp directory)

This ensures all images and resources are found correctly in both development and production.

Docker Build Context: Docker builds require a writable directory. When running from a PyInstaller bundle, the extracted resources are in a read-only temporary directory. To solve this:

• Docker build context is copied to $HOME/.knoe/build/knoe-db/
• This provides a writable location for Docker to operate
• See DOCKER-BUILD-FIX.md for details

Hidden Imports: PyInstaller can't always auto-detect imports, so we explicitly include:

• All installer modules
• curses/ncurses libraries

Output: installer.spec

3. Binary Creation

PyInstaller processes the spec file to create:

Single-file Executable:

• All Python code and dependencies bundled
• Extracts to temporary directory at runtime
• Console-enabled (works with both GUI and ncurses)

macOS App Bundle:

• Standard .app directory structure
• Icon embedded in bundle
• Info.plist with metadata
• Executable in Contents/MacOS/

Using the Built Application

GUI Mode (Double-click)

Simply double-click Knoe Installer.app in Finder to launch the graphical installer.

Command-line Mode

# Run with auto-detection (GUI if available, otherwise ncurses)
./dist/Knoe\ Installer.app/Contents/MacOS/knoe-installer

# Force ncurses mode
./dist/Knoe\ Installer.app/Contents/MacOS/knoe-installer --no-gui

# Force GUI mode
./dist/Knoe\ Installer.app/Contents/MacOS/knoe-installer --gui

# Show help
./dist/Knoe\ Installer.app/Contents/MacOS/knoe-installer --help

Installation to /Applications

cp -r "dist/Knoe Installer.app" /Applications/

After installation, the app is available:

• In Finder under Applications
• Via Spotlight search
• From command line as /Applications/Knoe\ Installer.app/Contents/MacOS/knoe-installer

Display Auto-detection

The installer automatically detects whether a GUI display is available:

1. macOS: Attempts to create a Tk root window
2. Linux/Unix: Checks for DISPLAY environment variable
3. Fallback: Uses ncurses mode if GUI unavailable

Override behavior:

• --gui flag: Force GUI mode (fails if no display)
• --no-gui flag: Force ncurses mode

Architecture Support

The build system detects the host architecture:

• Apple Silicon (arm64): Native ARM64 binary
• Intel (x86_64): Native x86_64 binary

For universal binaries supporting both architectures, you would need to:

1. Build on arm64 machine
2. Build on x86_64 machine
3. Use lipo to combine binaries

Build Requirements

System Requirements

• macOS 10.13 or later
• Python 3.8 or later
• Make build system

Python Dependencies

• pyinstaller >= 5.0
• pillow (optional, for advanced image processing)
• All runtime dependencies of install.py

macOS Developer Tools

• Command Line Tools (for sips, iconutil, codesign)

Install with: xcode-select --install

Troubleshooting

Build Fails with "No module named 'installer'"

Cause: PyInstaller can't find the installer package.

Solution: Ensure you're running from project root and installer/ directory exists.

Icon Not Showing in App Bundle

Cause: Icon conversion failed or .icns file is invalid.

Solution:

make clean
make build/knoe.icns
file build/knoe.icns  # Should say "Mac OS X icon"

Binary Won't Run on Other Machines

Cause: PyInstaller doesn't include all dependencies, or architecture mismatch.

Solutions:

• Check target machine has same or newer macOS version
• Verify architecture matches (arm64 vs x86_64)
• Check Console.app for error messages
• Rebuild with --debug in spec file for verbose output

GUI Doesn't Launch When Double-clicking

Cause: App not signed, or console mode preventing GUI launch.

Solutions:

• Right-click → Open (first time only, to bypass Gatekeeper)
• Check that console=True in spec allows both modes
• Run from Terminal to see error messages

Ncurses Mode Garbled Output

Cause: Terminal doesn't support required features.

Solution: Use a modern terminal emulator (Terminal.app, iTerm2, etc.)

Advanced Configuration

Custom Icon

Replace img/knoeIcon.png with your icon (PNG format, preferably 1024x1024).

Additional Data Files

Edit scripts/generate_spec.py and add to datas list:

datas = [
    ('installer', 'installer'),
    ('your_data', 'your_data'),  # Add this
]

Code Signing

For distribution outside development:

1. Get Apple Developer certificate
2. Modify Makefile to add signing step:

codesign --deep --force --verify --verbose \
  --sign "Developer ID Application: Your Name" \
  "dist/Knoe Installer.app"

Notarization

For Gatekeeper approval on macOS 10.15+:

1. Sign the app
2. Create a DMG or ZIP
3. Submit to Apple:

xcrun notarytool submit knoe-installer.zip \
  --apple-id your@email.com \
  --password app-specific-password \
  --team-id TEAMID

Distribution

DMG Creation

hdiutil create -volname "Knoe Installer" \
  -srcfolder "dist/Knoe Installer.app" \
  -ov -format UDZO \
  knoe-installer.dmg

ZIP Archive

cd dist
zip -r ../knoe-installer.zip "Knoe Installer.app"

Performance Considerations

Build Time: 30-60 seconds typical Binary Size: 50-100 MB (includes Python runtime + dependencies) Startup Time: 1-3 seconds (PyInstaller extraction overhead)

To improve startup time, consider using PyInstaller's --onedir mode instead of --onefile, though this increases distribution complexity.

CI/CD Integration

Example GitHub Actions workflow:

name: Build Installer
on: [push]
jobs:
  build:
    runs-on: macos-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: '3.11'
      - run: make install
      - run: make package
      - uses: actions/upload-artifact@v2
        with:
          name: knoe-installer
          path: dist/Knoe Installer.app

References

• PyInstaller Documentation
• macOS App Bundle Structure
• macOS Code Signing