Auto-UI Architecture

Overview

Photon's Auto-UI system automatically generates appropriate UI components based on the data returned from .photon.ts methods. This allows developers to focus on business logic while the runtime handles presentation across different interfaces (CLI, MCP, Web).

Architecture

Core Components (in @portel/photon-core)

1. Auto-UI System (auto-ui.ts)

   • Component type detection
   • JSDoc hint extraction
   • UI component generation
   • Layout inference

2. Progress Renderer (progress.ts)

   • Ephemeral progress indicators
   • Spinner for indeterminate progress
   • Progress bar for determinate progress
   • Auto-clears when complete

3. CLI UI Renderer (cli-ui-renderer.ts)

   • Terminal-based component rendering
   • Implements UIRenderer interface
   • Formats data for CLI display

Runtime Integration (in @portel/photon)

The runtime consumes photon-core components and provides:

1. MCP Server (server.ts)

   • Serves Photon methods via MCP protocol
   • Includes /playground for interactive testing
   • Auto-generates UI hints in tool responses

2. CLI Interface (cli.ts)

   • Uses ProgressRenderer from core
   • Formats output using core renderers
   • Ephemeral progress messages

3. Auto-UI Components (src/auto-ui/)

   • Extended components for runtime-specific features
   • Table, Tree, List, Card, Form, Progress
   • ComponentRegistry for routing

How It Works

1. Method Introspection

When a Photon method is called, the runtime:

// Extract JSDoc hints
const hints = extractUIHints(methodJSDoc);

// Generate UI component
const component = generateUIComponent(result, hints);

// Render based on target (CLI/MCP/Web)
renderUIComponent(component, renderer);

2. JSDoc Annotations

Developers can provide hints in JSDoc:

/**
 * Search GitHub repositories
 * @format table
 * @ui-component table
 * @ui-title Search Results
 * @ui-sortable
 * @ui-filterable
 */
async searchRepos(query: string) {
  return [...]; // Runtime auto-formats as table
}

3. Automatic Detection

Without hints, the system infers the best UI:

// Returns array of objects → table
async listUsers() {
  return [{ name: 'Alice', age: 30 }, ...];
}

// Returns tree structure → tree view
async getFileSystem() {
  return { name: 'root', children: [...] };
}

// Returns single value → text
async getCount() {
  return 42;
}

4. Progress Indication

Generators can emit progress updates:

async *searchWithProgress(query: string) {
  // Indeterminate progress (spinner)
  yield { emit: 'status', message: 'Searching...' };
  
  // Determinate progress (bar)
  yield { emit: 'progress', value: 0.5, message: 'Processing...' };
  
  // Progress clears automatically when done
  return results;
}

Supported UI Components

Data Display

• text: Single values, strings
• number: Numeric values
• boolean: True/false
• list: Simple arrays
• table: Array of objects
• tree: Hierarchical data
• card: Rich object display
• json: Raw JSON view

Input

• form: Interactive forms (via generators)

Feedback

• progress: Loading states
• toast: Notifications
• thinking: AI status

Code & Content

• code: Syntax-highlighted code
• markdown: Formatted text

MCP Integration

Client Capability Detection

Photon uses progressive enhancement — it detects the client's capabilities during the MCP initialize handshake and adapts responses accordingly. Developer code is unchanged; the runtime handles everything.

Detection Logic

During initialization, the client sends clientInfo (who it is) and capabilities (what it supports). Photon checks both:

1. capabilities.experimental["io.modelcontextprotocol/ui"] — official MCP Apps capability negotiation. If present, the client supports UI widgets.
2. clientInfo.name — fallback for known UI-capable clients that may not yet announce the capability key. Currently: "chatgpt", "mcpjam", "mcp-inspector", and "beam" (see UI_CAPABLE_CLIENTS in server.ts).

Client Tiers

Tier	Detection	What we send
basic	Unknown client, no UI capability	content only. No structuredContent, no _meta.ui
mcp-apps	Has experimental["io.modelcontextprotocol/ui"] OR known UI-capable clientInfo.name	content + structuredContent + _meta.ui.resourceUri
beam	clientInfo.name === 'beam'	Same as mcp-apps (Beam's transport handles rendering via its own pipeline)

Example: Same Tool, Different Clients

// UI-capable client (ChatGPT, MCPJam, Beam) — tool definition
{
  name: "search",
  description: "Search repositories",
  inputSchema: {...},
  _meta: {
    ui: { resourceUri: "ui://github/search-results" }
  }
}

// Basic client (Claude Desktop without MCP Apps, unknown clients) — tool definition
{
  name: "search",
  description: "Search repositories",
  inputSchema: {...}
  // No _meta — client wouldn't use it
}

Tool responses follow the same pattern: UI-capable clients receive structuredContent + _meta.ui, basic clients receive only content[].text.

Tool Output Templates

UI-capable clients receive _meta.ui.resourceUri pointing to the tool's linked UI template. The ui:// URI scheme follows the MCP Apps standard (SEP-1865).

Photon detects the client at handshake and serves the right format automatically. Developer code stays unchanged — just use the @ui docblock tag.

Beam UI

The Beam UI (photon beam) provides:

1. Interactive Testing: Select tools and run them
2. Auto-Generated Forms: Input parameters based on JSON Schema
3. UI Preview: See linked templates and custom UIs
4. Progress Indication: Real-time status updates via SSE
5. Status Panel: Runtime health monitoring
6. Theme Support: Automatic light/dark theme switching

Access via: photon beam command, which opens the interactive web UI for all installed Photons.

Web UI Integration

For web-based UIs (like Lumina or NCP Dashboard):

1. Component Props: Runtime returns structured UI descriptors
2. Lit Web Components: Beam's frontend uses Lit internally (custom UIs in iframes can use any framework)
3. Custom Styling: CSS injection support
4. Real-time Updates: SSE via MCP Streamable HTTP transport

Example component descriptor:

{
  "component": "Table",
  "props": {
    "data": [...],
    "columns": ["name", "stars", "forks"],
    "sortable": true,
    "filterable": true
  }
}

Best Practices

For Photon Developers

1. Return Clean Data: Let Auto-UI handle formatting

   // Good
   return { name: 'Alice', score: 95 };
   
   // Avoid
   return `Name: Alice\nScore: 95`;

2. Use JSDoc Hints for complex UIs:

   /** @ui-component tree @ui-expandable */

3. Emit Progress for long operations:

   yield { emit: 'progress', value: 0.5 };

4. Group Related Data for card layout:

   return {
     user: { name, email },
     stats: { followers, repos },
     metadata: { created, updated }
   };

For Runtime Developers

1. Use Core Components: Import from @portel/photon-core
2. Implement UIRenderer: For custom output targets
3. Handle Progress: Subscribe to emit events
4. Clear Ephemeral Output: Use ProgressRenderer API

API Reference

Auto-UI Core

// Extract hints from JSDoc
extractUIHints(jsdoc: string): AutoUIConfig

// Generate component from data
generateUIComponent(data: any, config?: AutoUIConfig): UIComponent

// Suggest alternative components
suggestComponents(data: any): UIComponentType[]

// Render component
renderUIComponent(component: UIComponent, renderer: UIRenderer): void

Progress Renderer

// Start spinner (indeterminate)
startSpinner(message: string): void

// Show progress bar (determinate)
showProgress(value: number, message?: string): void

// Update message
updateProgressMessage(message: string): void

// Stop and clear
stopProgress(): void

// Check if active
isProgressActive(): boolean

CLI UI Renderer

// Get singleton instance
const renderer = cliRenderer();

// Render components
renderer.renderTable(data);
renderer.renderTree(data);
renderer.renderList(data);
renderer.renderCard(data);
renderer.renderProgress(value, total);

Implementation Status

Completed

• [x] Custom Components: Photons can bundle custom UI via @ui directive
• [x] Themes: Light/dark theme support with automatic switching
• [x] Interactive Components: Click handlers, forms in Beam UI
• [x] Streaming UI: Progressive rendering via SSE events

Future Enhancements

• [ ] Responsive Layouts: Adapt to terminal size / viewport
• [ ] Chart Components: Graphs and visualizations in CLI
• [ ] Advanced filtering and sorting in CLI output

Related Standards

• MCP (Model Context Protocol): Anthropic's tool protocol
• ChatGPT Actions UI: OpenAI's UI guidelines
• Anthropic UI Paper: Combined MCP + UI specification

Examples

See:

• photon-examples for example Photon implementations
• photon beam command for interactive demos
• Tests in tests/ for component usage