Nexus - API Gateway & Frontend Proxy¶

Port: 443 (HTTPS) Database: None (stateless proxy) Repository: hivematrix-nexus Version: 1.0

Table of Contents¶

Overview
Architecture
Gateway Responsibilities
Technology Stack
Production Features
API Reference
Authentication Endpoints
Proxy Routes
Utility Endpoints
Authentication Flow
OAuth2 Login Flow
Token Validation
Session Management
Logout Flow
Reverse Proxy
Service Routing
Request Forwarding
Response Handling
SSE Streaming
HTML Injection
Global CSS Injection
Side Panel Navigation
Theme Application
JavaScript Injection
Keycloak Proxy
URL Rewriting
Cookie Handling
Content Modification
User Preferences
Theme Management
Home Page Routing
Preference Caching
Configuration
Environment Variables
SSL Certificates
Service Registry
Security
TLS Termination
Session Security
CSRF Protection
Rate Limiting
Monitoring & Observability
Health Checks
Logging
Development
Running Locally
Testing
SSL Certificate Generation
Troubleshooting
See Also

Overview¶

Nexus is the single entry point for all HiveMatrix services. It acts as a reverse proxy, authentication gateway, and frontend compositor that provides a unified HTTPS interface to the entire platform.

Gateway Responsibilities¶

HTTPS Termination
Only service that binds to 0.0.0.0 (external network)
SSL/TLS termination at port 443
All backend services remain localhost-only
Enforces secure communication
Reverse Proxy
Routes /<service>/* to backend services
Strips service prefix before forwarding
Adds X-Forwarded-* headers for proxy awareness
Streams Server-Sent Events (SSE) responses
Handles redirects and cookies
Authentication Gateway
Manages OAuth2 flow with Keycloak
Validates JWT tokens with Core
Maintains user sessions in cookies
Enforces authentication on all routes
Graceful fallback if Core unavailable
Frontend Composition
Injects global CSS into all HTML responses
Adds side panel navigation to every page
Applies user theme preferences (light/dark)
Injects JavaScript for interactive features
Cache-busting for static assets
Keycloak Proxy
Proxies /keycloak/* to Keycloak server
Rewrites URLs for external access
Modifies cookies for proxy compatibility
Enables external users to access Keycloak through HTTPS

Architecture¶

Technology Stack¶

Framework: Flask 3.0.0
HTTP Client: Requests (for proxying)
HTML Parser: BeautifulSoup4 (for injection)
JWT Validation: PyJWT with PyJWKClient
Rate Limiting: Flask-Limiter
Logging: Structured JSON logging with correlation IDs
API Documentation: Flasgger (OpenAPI/Swagger)
Health Checks: Custom HealthChecker library

Production Features¶

Nexus includes several production-ready features introduced in version 4.1:

Per-User Rate Limiting¶

Rate limits applied per user (JWT subject) instead of IP address
Higher limits for gateway service:
20,000 requests/hour
1,000 requests/minute
Suitable for LAN environments with multiple users
Falls back to IP-based limiting for unauthenticated requests

Structured Logging¶

JSON-formatted logs with correlation IDs for distributed tracing
Request/response logging with timing
Error context preservation
Centralized logging to Helm service
Configurable log levels (DEBUG, INFO, WARNING, ERROR)

RFC 7807 Problem Details¶

Standardized machine-readable error responses
Consistent error format across all endpoints
Includes error type, title, detail, status, and instance
Enables automated error handling by clients

OpenAPI/Swagger Documentation¶

Auto-generated API documentation at /docs
Interactive API testing interface
Request/response schemas
Authentication examples
Available at https://localhost/docs

Comprehensive Health Checks¶

Component-level health monitoring at /health
Checks: Core availability, Keycloak availability, disk space
Status: healthy (200), degraded, or unhealthy (503)
Useful for load balancer health probes

Graceful Degradation¶

Continues operation if Core is temporarily unavailable
Falls back to local JWT signature validation
Logs warnings but maintains service availability
Prevents cascading failures

API Reference¶

Nexus exposes both authentication endpoints and proxy routes.

Authentication Endpoints¶

`GET /login`¶

Initiates OAuth2 login flow with Keycloak.

Query Parameters: - next (optional): URL to redirect to after successful login

Process: 1. Saves next URL in session 2. Generates OAuth state and nonce (CSRF protection) 3. Builds Keycloak authorization URL 4. Redirects to Keycloak through /keycloak/ proxy

Response: 302 Redirect to Keycloak authorization endpoint

Example:

# Redirect user to login
curl -L https://localhost/login?next=/codex/

Authorization URL Format:

https://localhost/keycloak/realms/hivematrix/protocol/openid-connect/auth
  ?response_type=code
  &client_id=core-client
  &redirect_uri=https://localhost/keycloak-callback
  &scope=openid%20email%20profile
  &state=<random-state>
  &nonce=<random-nonce>

`GET /keycloak-callback`¶

OAuth2 callback endpoint. Keycloak redirects here after authentication.

Query Parameters: - code (required, from Keycloak): Authorization code - state (required, from Keycloak): CSRF protection token - error (optional): Error code if authentication failed

Process: 1. Verify State: Check state parameter matches session (CSRF protection) 2. Exchange Code: Trade authorization code for Keycloak access token 3. Get HiveMatrix JWT: Send access token to Core for exchange 4. Validate JWT: Verify JWT signature and session status 5. Store Session: Save JWT and user data in session cookie 6. Redirect: Send user to original destination (next_url)

Success Response: 302 Redirect to next_url or user's home page

Error Responses: - 401 Unauthorized: Invalid state, authentication failed, or token exchange failed - 502 Bad Gateway: Keycloak or Core unreachable

Example Flow:

1. User authenticates → Keycloak redirects to:
   https://localhost/keycloak-callback?code=abc123&state=xyz789

2. Nexus exchanges code for Keycloak access token
3. Nexus sends access token to Core: POST /api/token/exchange
4. Core returns HiveMatrix JWT
5. Nexus validates JWT locally and with Core
6. Nexus stores JWT in session cookie
7. Nexus redirects to /codex/ (or wherever user was going)

`GET /logout`¶

Ends user session, revokes tokens, and clears all browser storage.

Process: 1. Revoke Token: Call Core's /api/token/revoke endpoint 2. Clear Session: Delete server-side session cookie 3. Return HTML: Page with JavaScript to clear browser storage 4. Redirect: After 1 second, redirect to home page with cache-busting

Response: HTML page that clears storage and redirects

Headers Set:

Cache-Control: no-cache, no-store, must-revalidate, max-age=0, private
Pragma: no-cache
Expires: 0
Clear-Site-Data: "cache", "cookies", "storage"

JavaScript Actions: - Clears sessionStorage - Clears localStorage - Deletes all cookies - Redirects to /?logout=<timestamp> (cache-busting)

Example:

curl https://localhost/logout

User Experience: 1. User clicks "Logout" in sidebar 2. Nexus revokes JWT session at Core 3. Page displays "You have been logged out" 4. JavaScript clears all browser storage 5. After 1 second, redirects to login page 6. User cannot use back button to access authenticated pages

Proxy Routes¶

`GET /<service>/*`¶

Reverse proxy to backend services. This is the main gateway route.

Route Pattern: /<service>/<path>

Examples: - /codex/ → http://localhost:5010/ - /ledger/api/invoices → http://localhost:5030/api/invoices - /beacon/helpdesk → http://localhost:5001/helpdesk

Request Processing:

Authentication Check:
If no token in session → redirect to /login
If token exists → validate with Core
If token invalid → clear session and redirect to /login
Service Lookup:
Extract service name from first path segment
Look up service configuration in services.json
Check user has permission to access service
Return 404 if service not found
Permission Check:
Skip if service is visible to all users
Check admin_only flag (requires admin permission)
Check billing_or_admin_only flag (requires admin or billing)
Deny access if user lacks required permission
Request Forwarding:
Build backend URL: {service.url}/{remaining_path}
Copy request headers (except Host)
Add Authorization: Bearer {jwt} header
Add X-Forwarded-* headers for proxy awareness
Forward method, query params, body, and cookies
Response Handling:
If Content-Type: text/event-stream → stream response
If Content-Type: text/html → inject CSS and side panel
Otherwise → return response as-is

Headers Added to Backend Request:

Authorization: Bearer eyJhbGciOiJSUzI1NiIs...
X-Forwarded-For: 192.168.1.100
X-Forwarded-Proto: https
X-Forwarded-Host: your-server.com
X-Forwarded-Prefix: /codex

HTML Injection (for text/html responses): 1. Parse HTML with BeautifulSoup 2. Add data-theme attribute to <html> tag 3. Inject global.css and side-panel.css with cache-busting 4. Inject sidebar initialization script in <head> 5. Inject side panel navigation and theme toggle in <body> 6. Return modified HTML

Response: Proxied response from backend service

`GET /keycloak/<path>`¶

Proxies requests to Keycloak server for external access.

Purpose: - Allows external browsers to access Keycloak through Nexus's HTTPS endpoint - Rewrites URLs so Keycloak works behind a proxy - Enables OAuth flow to work with external clients

URL Rewriting: - Backend: http://localhost:8080/ (where Keycloak actually runs) - Frontend: https://your-server/keycloak/ (external-facing URL)

Process: 1. Forward request to http://localhost:8080/{path} 2. Add X-Forwarded-* headers 3. Rewrite Location headers in redirects 4. Modify Set-Cookie headers (remove domain, set path to /keycloak) 5. Rewrite HTML/JS/CSS content to use proxy URLs

Example:

# External request
GET https://your-server/keycloak/realms/hivematrix/.well-known/openid-configuration

# Proxied to
GET http://localhost:8080/realms/hivematrix/.well-known/openid-configuration

# Response URLs rewritten from:
"issuer": "http://localhost:8080/realms/hivematrix"

# To:
"issuer": "https://your-server/keycloak/realms/hivematrix"

Cookie Modifications:

# Original Set-Cookie from Keycloak:
Set-Cookie: AUTH_SESSION_ID=abc123; Path=/auth; Domain=localhost; HttpOnly

# Modified by Nexus:
Set-Cookie: AUTH_SESSION_ID=abc123; Path=/keycloak; HttpOnly; SameSite=Lax

Utility Endpoints¶

`POST /api/invalidate-cache`¶

Invalidates user preference cache (theme, home page).

Purpose: Called after user updates settings in Codex to force cache refresh.

Response:

{
  "success": true,
  "message": "Cache invalidated"
}

Cache Cleared: - cached_theme - cached_theme_time - cached_home_page - cached_home_page_time

Example:

// After saving theme to Codex
await fetch('/api/invalidate-cache', {
    method: 'POST',
    credentials: 'same-origin'
});

`GET /health`¶

Comprehensive health check endpoint.

Checks: - Disk space usage - Core service availability (GET /health) - Keycloak availability (GET /health)

Response (200 OK - Healthy):

{
  "service": "nexus",
  "status": "healthy",
  "timestamp": "2025-11-22T10:30:00.000Z",
  "checks": {
    "disk": {
      "status": "healthy",
      "usage_percent": 45.67,
      "free_gb": 123.45,
      "total_gb": 250.0
    },
    "dependencies": {
      "core": {
        "status": "healthy",
        "response_time_ms": 15
      },
      "keycloak": {
        "status": "healthy",
        "response_time_ms": 42
      }
    }
  }
}

Response (503 Service Unavailable - Degraded):

{
  "service": "nexus",
  "status": "degraded",
  "timestamp": "2025-11-22T10:30:00.000Z",
  "checks": {
    "disk": {
      "status": "healthy",
      "usage_percent": 45.67,
      "free_gb": 123.45,
      "total_gb": 250.0
    },
    "dependencies": {
      "core": {
        "status": "unhealthy",
        "error": "connection_refused"
      },
      "keycloak": {
        "status": "healthy",
        "response_time_ms": 42
      }
    }
  }
}

Use for Load Balancer: - 200 = Healthy, route traffic - 503 = Unhealthy/degraded, remove from pool

`GET /docs`¶

Interactive OpenAPI/Swagger API documentation.

Response: HTML page with interactive API explorer

Access: https://localhost/docs

`GET /helpdesk`¶

Convenience redirect to Beacon's helpdesk view.

Response: 302 Redirect to /beacon/helpdesk

`GET /professional-services`¶

Convenience redirect to Beacon's professional services view.

Response: 302 Redirect to /beacon/professional-services

Authentication Flow¶

Complete OAuth2 authorization code flow with Keycloak:

┌─────────┐           ┌───────┐           ┌──────────┐           ┌──────┐
│ Browser │           │ Nexus │           │ Keycloak │           │ Core │
└────┬────┘           └───┬───┘           └────┬─────┘           └───┬──┘
     │                    │                    │                     │
     │  1. GET /codex/    │                    │                     │
     ├───────────────────>│                    │                     │
     │                    │                    │                     │
     │                    │  2. No session     │                     │
     │                    │  cookie found      │                     │
     │                    │                    │                     │
     │  3. 302 Redirect   │                    │                     │
     │  to /login?next=/codex/                 │                     │
     │<───────────────────┤                    │                     │
     │                    │                    │                     │
     │  4. GET /login?next=/codex/             │                     │
     ├───────────────────>│                    │                     │
     │                    │                    │                     │
     │                    │  5. Generate state │                     │
     │                    │  and nonce, save   │                     │
     │                    │  in session        │                     │
     │                    │                    │                     │
     │  6. 302 Redirect to Keycloak auth URL   │                     │
     │<───────────────────┤                    │                     │
     │                    │                    │                     │
     │  7. GET /keycloak/realms/.../auth       │                     │
     ├────────────────────────────────────────>│                     │
     │                    │                    │                     │
     │  8. Login form (username/password)      │                     │
     │<────────────────────────────────────────┤                     │
     │                    │                    │                     │
     │  9. POST credentials                    │                     │
     ├────────────────────────────────────────>│                     │
     │                    │                    │                     │
     │  10. 302 Redirect with code             │                     │
     │  /keycloak-callback?code=xxx&state=yyy  │                     │
     │<────────────────────────────────────────┤                     │
     │                    │                    │                     │
     │  11. GET /keycloak-callback?code&state  │                     │
     ├───────────────────>│                    │                     │
     │                    │                    │                     │
     │                    │  12. Verify state  │                     │
     │                    │  matches session   │                     │
     │                    │                    │                     │
     │                    │  13. POST /realms/.../token              │
     │                    │  (exchange code    │                     │
     │                    │  for access token) │                     │
     │                    ├───────────────────>│                     │
     │                    │                    │                     │
     │                    │  14. Access token  │                     │
     │                    │<───────────────────┤                     │
     │                    │                    │                     │
     │                    │  15. POST /api/token/exchange            │
     │                    │  {access_token}    │                     │
     │                    ├─────────────────────────────────────────>│
     │                    │                    │                     │
     │                    │                    │  16. Validate with  │
     │                    │                    │  Keycloak, create   │
     │                    │                    │  session, mint JWT  │
     │                    │                    │                     │
     │                    │  17. {token: JWT}  │                     │
     │                    │<─────────────────────────────────────────┤
     │                    │                    │                     │
     │                    │  18. Validate JWT  │                     │
     │                    │  signature locally │                     │
     │                    │                    │                     │
     │                    │  19. POST /api/token/validate            │
     │                    │  {token: JWT}      │                     │
     │                    ├─────────────────────────────────────────>│
     │                    │                    │                     │
     │                    │  20. {valid: true, user: {...}}          │
     │                    │<─────────────────────────────────────────┤
     │                    │                    │                     │
     │                    │  21. Store JWT in  │                     │
     │                    │  session cookie    │                     │
     │                    │                    │                     │
     │  22. 302 Redirect to /codex/            │                     │
     │<───────────────────┤                    │                     │
     │                    │                    │                     │
     │  23. GET /codex/   │                    │                     │
     │  (with session cookie)                  │                     │
     ├───────────────────>│                    │                     │
     │                    │                    │                     │
     │                    │  24. Validate JWT, │                     │
     │                    │  proxy to backend  │                     │
     │                    │                    │                     │
     │  25. HTML page with injected CSS & nav  │                     │
     │<───────────────────┤                    │                     │

Key Points: - Nexus manages entire OAuth flow (not using Authlib's automatic flow) - State parameter prevents CSRF attacks - Nonce prevents replay attacks - JWT stored in HttpOnly session cookie - JWT validated both locally (signature) and remotely (session status)

Token Validation¶

On every proxied request, Nexus validates the JWT:

def validate_token(token):
    """
    Validates a JWT token by checking signature and session status.
    Returns decoded data if valid, None otherwise.
    """
    try:
        # 1. Verify signature locally using Core's public key
        client = get_jwks_client()
        signing_key = client.get_signing_key_from_jwt(token)
        data = jwt.decode(
            token,
            signing_key.key,
            algorithms=["RS256"],
            issuer="hivematrix-core",
            options={"verify_exp": True}
        )

        # 2. Check with Core if session is still valid (not revoked)
        validation_response = requests.post(
            f"{core_url}/api/token/validate",
            json={'token': token},
            timeout=2
        )

        if validation_response.status_code == 200:
            return data  # Session is valid
        else:
            return None  # Session revoked or invalid

    except requests.exceptions.RequestException as e:
        # If Core is unreachable, fall back to local validation
        # This ensures the system keeps working even if Core is down
        logger.warning(f"Core unreachable: {e}, falling back to local validation")
        return data  # Trust local signature verification

    except jwt.ExpiredSignatureError:
        return None  # Token expired
    except jwt.PyJWTError as e:
        return None  # Invalid token

Validation Steps: 1. Fetch Core's public key from JWKS endpoint (cached) 2. Verify JWT signature using public key 3. Check token expiration (exp claim) 4. Check issuer is hivematrix-core 5. Call Core's /api/token/validate to check session status 6. If Core unavailable, trust local signature verification (graceful degradation)

Why Dual Validation? - Local: Fast signature verification (prevents tampering) - Remote: Session status check (handles revocation/logout) - Fallback: If Core down, local validation keeps system working

Session Management¶

Nexus stores JWT in Flask session cookie:

Session Cookie Settings:

app.config['SESSION_COOKIE_HTTPONLY'] = True   # Prevents JavaScript access
app.config['SESSION_COOKIE_SAMESITE'] = 'Lax'  # CSRF protection
app.config['PERMANENT_SESSION_LIFETIME'] = 3600  # 1 hour

Session Data:

session['token'] = 'eyJhbGciOiJSUzI1NiIs...'
session['user'] = {
    'sub': 'user-uuid',
    'name': 'John Doe',
    'email': 'john@example.com',
    'permission_level': 'admin',
    'groups': ['/admins']
}

Security Features: - HttpOnly: Cookie cannot be accessed by JavaScript (prevents XSS) - SameSite: Cookie only sent with same-site requests (prevents CSRF) - Secure: Cookie only sent over HTTPS (production) - Encrypted: Cookie signed with SECRET_KEY

Logout Flow¶

Complete session termination:

┌─────────┐           ┌───────┐           ┌──────┐
│ Browser │           │ Nexus │           │ Core │
└────┬────┘           └───┬───┘           └───┬──┘
     │                    │                   │
     │  1. GET /logout    │                   │
     ├───────────────────>│                   │
     │                    │                   │
     │                    │  2. Extract JWT   │
     │                    │  from session     │
     │                    │                   │
     │                    │  3. POST /api/token/revoke │
     │                    │  {token: JWT}     │
     │                    ├──────────────────>│
     │                    │                   │
     │                    │  4. Mark session  │
     │                    │  as revoked       │
     │                    │                   │
     │                    │  5. {message: "Session revoked"} │
     │                    │<──────────────────┤
     │                    │                   │
     │                    │  6. Clear session │
     │                    │  cookie           │
     │                    │                   │
     │  7. HTML page with storage-clearing JS │
     │  + Clear-Site-Data header              │
     │<───────────────────┤                   │
     │                    │                   │
     │  8. JavaScript     │                   │
     │  clears all        │                   │
     │  browser storage   │                   │
     │                    │                   │
     │  9. 302 Redirect   │                   │
     │  to /?logout=timestamp                 │
     │                    │                   │
     │  10. GET /?logout=...                  │
     │  (no session cookie)                   │
     ├───────────────────>│                   │
     │                    │                   │
     │  11. 302 Redirect to /login            │
     │<───────────────────┤                   │

Logout Actions: 1. Revoke JWT session at Core 2. Clear Flask session cookie server-side 3. Return HTML with storage-clearing JavaScript 4. Set Clear-Site-Data header 5. JavaScript clears sessionStorage, localStorage, all cookies 6. Redirect to home page with cache-busting timestamp 7. User redirected to login (no valid session)

Reverse Proxy¶

Service Routing¶

Nexus routes requests based on the first path segment.

Routing Table (from services.json):

Path	Backend URL	Description
`/core/*`	`http://localhost:5000`	Authentication service
`/codex/*`	`http://localhost:5010`	Master data management
`/ledger/*`	`http://localhost:5030`	Billing and invoicing
`/beacon/*`	`http://localhost:5001`	Ticket dashboard
`/knowledgetree/*`	`http://localhost:5020`	Knowledge base
`/brainhair/*`	`http://localhost:5050`	AI assistant
`/helm/*`	`http://localhost:5004`	Orchestration and config
`/keycloak/*`	`http://localhost:8080`	Identity provider

Service Configuration:

{
  "codex": {
    "name": "HiveMatrix Codex",
    "url": "http://localhost:5010",
    "port": 5010,
    "visible": true,
    "admin_only": false,
    "billing_or_admin_only": false
  }
}

Permission-Based Access: - visible: false → Hidden from all users (infrastructure services) - admin_only: true → Only admins can access - billing_or_admin_only: true → Admins and billing users only - No flags → All authenticated users can access

Request Forwarding¶

When proxying a request to a backend service:

1. Build Backend URL:

# User requests: GET /codex/api/companies
service_name = 'codex'
service_path = 'api/companies'
backend_url = 'http://localhost:5010/api/companies'

2. Copy Request Headers:

headers = {key: value for (key, value) in request.headers if key != 'Host'}

3. Add Authentication:

headers['Authorization'] = f"Bearer {jwt_token}"

4. Add Proxy Headers:

headers['X-Forwarded-For'] = request.remote_addr        # Client IP
headers['X-Forwarded-Proto'] = 'https'                  # Original protocol
headers['X-Forwarded-Host'] = request.host              # Original host
headers['X-Forwarded-Prefix'] = f'/{service_name}'      # Service prefix

Why X-Forwarded Headers? - Backend services use ProxyFix to reconstruct original URL - X-Forwarded-Prefix enables correct url_for() behavior - Services can detect they're behind a proxy - Logging shows original client IP, not 127.0.0.1

5. Forward Request:

resp = requests.request(
    method=request.method,
    url=backend_url,
    headers=headers,
    params=request.args,      # Query parameters
    data=request.get_data(),  # Request body
    cookies=request.cookies,
    allow_redirects=False,    # Handle redirects manually
    stream=True,              # Enable SSE streaming
    timeout=30
)

Response Handling¶

Nexus processes responses differently based on content type:

1. Server-Sent Events (SSE):

if 'text/event-stream' in resp.headers.get('Content-Type', ''):
    # Stream response directly without buffering
    def generate():
        for chunk in resp.iter_content(chunk_size=1024):
            if chunk:
                yield chunk
    return Response(generate(), resp.status_code, response_headers)

Use Case: Brainhair's AI streaming responses

2. HTML Responses:

if 'text/html' in resp.headers.get('Content-Type', ''):
    # Parse HTML
    soup = BeautifulSoup(content, 'html.parser')

    # Add theme to <html> tag
    html_tag = soup.find('html')
    if html_tag:
        theme = get_user_theme(token_data)
        html_tag['data-theme'] = theme

    # Inject CSS
    head = soup.find('head')
    if head:
        css_link = soup.new_tag('link', rel='stylesheet',
                                href=f"/static/css/global.css?v={VERSION}")
        head.append(css_link)

        panel_css_link = soup.new_tag('link', rel='stylesheet',
                                      href=f"/static/css/side-panel.css?v={VERSION}")
        head.append(panel_css_link)

        # Inject sidebar initialization script
        sidebar_init = soup.new_tag('script')
        sidebar_init.string = '''
            (function() {
                if (localStorage.getItem('sidebar-collapsed') === 'true') {
                    document.documentElement.classList.add('sidebar-collapsed');
                }
            })();
        '''
        head.append(sidebar_init)

    # Inject side panel navigation
    inject_side_panel(soup, service_name, token_data)

    content = str(soup)
    return Response(content, resp.status_code, response_headers)

Injection Order: 1. Parse HTML with BeautifulSoup 2. Add data-theme="light" or "dark" to <html> tag 3. Inject global.css and side-panel.css in <head> 4. Inject sidebar init script in <head> (prevents flash) 5. Wrap <body> content in layout structure 6. Inject side panel HTML 7. Inject theme toggle JavaScript 8. Return modified HTML

3. All Other Responses:

# JSON, images, downloads, etc. - return as-is
return Response(content, resp.status_code, response_headers)

SSE Streaming¶

Nexus supports Server-Sent Events for real-time streaming:

Detection:

if 'text/event-stream' in resp.headers.get('Content-Type', ''):
    # This is an SSE stream

Streaming:

def generate():
    for chunk in resp.iter_content(chunk_size=1024, decode_unicode=False):
        if chunk:
            yield chunk

return Response(generate(), resp.status_code, response_headers)

Example Use Case - Brainhair AI:

Client → Nexus → Brainhair
  ↓
  SSE stream: data: {"token": "Hello"}
  SSE stream: data: {"token": " world"}
  SSE stream: data: {"done": true}

Why Not Buffer? - SSE requires incremental delivery - Buffering would defeat real-time purpose - Generator yields chunks as they arrive - Client sees tokens as they're generated

HTML Injection¶

Global CSS Injection¶

Nexus injects global styles into every HTML response:

CSS Files: - static/css/global.css - Global styles, design system, utilities - static/css/side-panel.css - Side panel navigation styles

Injection:

<head>
    <!-- Existing service head content -->
    <link rel="stylesheet" href="/static/css/global.css?v=4.1">
    <link rel="stylesheet" href="/static/css/side-panel.css?v=4.1">
    <script>
        (function() {
            if (localStorage.getItem('sidebar-collapsed') === 'true') {
                document.documentElement.classList.add('sidebar-collapsed');
            }
        })();
    </script>
</head>

Cache-Busting: - ?v=4.1 parameter changes with version - Forces browsers to fetch new CSS after updates - Prevents stale CSS from being served

Nexus injects a side panel into every HTML page:

Structure:



href="#__codelineno-30-1"><div class="hivematrix-layout"> <div class="hivematrix-side-panel" id="side-panel"> <!-- Header with toggle --> <div class="side-panel__header"> <button class="side-panel__toggle" id="sidebar-toggle">☰</button> <h3 class="side-panel__title">HiveMatrix</h3> </div> <!-- Service navigation --> <nav class="side-panel__nav"> <ul class="side-panel__list"> <li class="side-panel__item side-panel__item--active"> <a href="/codex/" class="side-panel__link"> <span class="side-panel__icon">[SVG icon]</span> <span class="side-panel__label">Codex</span> </a> </li> <!-- More services... --> </ul> </nav> <!-- Footer with settings & logout --> <div class="side-panel__footer"> <a href="/codex/settings" class="side-panel__link"> <span class="side-panel__icon">[Settings icon]</span> <span class="side-panel__label">Settings</span> </a> <a href="/logout" class="side-panel__link"> <span class="side-panel__icon">[Logout icon]</span> <span class="side-panel__label">Logout</span> </a> <button class="theme-toggle" id="theme-toggle-btn"> [Theme toggle UI] </button> </div> </div> <div class="hivematrix-content"> <!-- Original page content --> </div> class="p"></div>
Service Icons:
Lucide-style SVG icons for each service:
- Codex: Database icon
- Ledger: Receipt icon
- Beacon: Radio tower icon
- KnowledgeTree: Tree icon
- Brainhair: Brain icon
- Helm: Ship wheel icon
Permission Filtering:
Only shows services the user has permission to access:
# Skip if not visible
if not service_config.get('visible', True):
    continue

# Skip if admin_only and user is not admin
if service_config.get('admin_only', False) and user_permission != 'admin':
    continue

# Skip if billing_or_admin_only and user is neither
if service_config.get('billing_or_admin_only', False) and user_permission not in ['admin', 'billing']:
    continue

Active State:
Current service gets side-panel__item--active class for highlighting.

Theme Application¶
Nexus applies user's theme preference to every page:
1. Fetch Theme:
def get_user_theme(token_data):
    """Fetch user's theme from Codex with 5-minute cache."""
    user_email = token_data.get('email')

    # Check session cache
    cached_theme = session.get('cached_theme')
    cache_time = session.get('cached_theme_time', 0)

    if cached_theme and (time.time() - cache_time) < 300:
        return cached_theme

    # Call Codex API
    response = call_service('codex', '/api/public/user/theme',
                           params={'email': user_email}, timeout=2)

    if response.status_code == 200:
        theme = response.json().get('theme', 'light')
        # Cache in session
        session['cached_theme'] = theme
        session['cached_theme_time'] = time.time()
        return theme

    # Default to light
    return 'light'

2. Apply to HTML:
html_tag = soup.find('html')
if html_tag:
    theme = get_user_theme(token_data)
    html_tag['data-theme'] = theme  # Sets data-theme="light" or "dark"

3. CSS Selects Theme:
/* Light theme (default) */
:root {
    --color-bg: #ffffff;
    --color-text: #000000;
}

/* Dark theme */
[data-theme="dark"] {
    --color-bg: #1a1a1a;
    --color-text: #ffffff;
}

Performance:
- Theme cached in session for 5 minutes
- Avoids calling Codex on every page load
- Falls back to light if Codex unavailable

JavaScript Injection¶
Nexus injects JavaScript for interactive features:
1. Sidebar Initialization (in <head>):
(function() {
    if (localStorage.getItem('sidebar-collapsed') === 'true') {
        document.documentElement.classList.add('sidebar-collapsed');
    }
})();

Purpose: Applies sidebar state before render to prevent flash.

2. Theme Toggle (in <body>):
async function toggleTheme() {
    const currentTheme = document.documentElement.getAttribute('data-theme') || 'light';
    const newTheme = currentTheme === 'dark' ? 'light' : 'dark';

    // Apply immediately
    document.documentElement.setAttribute('data-theme', newTheme);

    // Save to Codex
    const response = await fetch('/codex/api/my/settings', {
        method: 'PUT',
        headers: {'Content-Type': 'application/json'},
        credentials: 'same-origin',
        body: JSON.stringify({ theme_preference: newTheme })
    });

    if (response.ok) {
        // Invalidate Nexus cache
        await fetch('/api/invalidate-cache', {
            method: 'POST',
            credentials: 'same-origin'
        });
    }
}

Purpose: Toggle theme, save to Codex, invalidate cache.

3. Sidebar Toggle:
function toggleSidebar() {
    const sidePanel = document.getElementById('side-panel');
    sidePanel.classList.toggle('collapsed');
    const isCollapsed = sidePanel.classList.contains('collapsed');
    localStorage.setItem('sidebar-collapsed', isCollapsed);
}

Purpose: Toggle sidebar visibility, persist state in localStorage.

Keycloak Proxy¶
URL Rewriting¶
Nexus rewrites Keycloak URLs to enable external access:
Problem:
- Keycloak runs on http://localhost:8080
- External browsers cannot reach localhost
- OAuth redirects fail if URLs point to localhost
Solution:
- Proxy /keycloak/* to http://localhost:8080
- Rewrite all URLs in responses to use /keycloak/ prefix
- External browsers access Keycloak through Nexus
Example Rewriting:
# Original Keycloak response:
{
  "issuer": "http://localhost:8080/realms/hivematrix",
  "authorization_endpoint": "http://localhost:8080/realms/hivematrix/protocol/openid-connect/auth"
}

# Rewritten by Nexus:
{
  "issuer": "https://your-server/keycloak/realms/hivematrix",
  "authorization_endpoint": "https://your-server/keycloak/realms/hivematrix/protocol/openid-connect/auth"
}

Implementation:
keycloak_backend = 'http://localhost:8080'
nexus_proxy_url = 'https://your-server/keycloak'

# Rewrite HTML/JS/CSS content
if 'text/html' in content_type or 'application/javascript' in content_type:
    text = content.decode('utf-8')
    text = text.replace(keycloak_backend, nexus_proxy_url)
    content = text.encode('utf-8')


Cookie Handling¶
Nexus modifies Keycloak's Set-Cookie headers for proxy compatibility:
Original Set-Cookie:
Set-Cookie: AUTH_SESSION_ID=abc123; Path=/auth; Domain=localhost; HttpOnly

Problems:
- Domain=localhost prevents cookie from being sent to your-server
- Path=/auth prevents cookie from being sent to /keycloak/auth
Modified by Nexus:
Set-Cookie: AUTH_SESSION_ID=abc123; Path=/keycloak; HttpOnly; SameSite=Lax

Changes:
- Remove Domain restriction (cookie works for any domain)
- Change Path=/auth to Path=/keycloak (matches proxy path)
- Add SameSite=Lax for security
- Keep HttpOnly for XSS protection
Implementation:
if name.lower() == 'set-cookie':
    # Remove domain restrictions
    value = re.sub(r'; Domain=[^;]+', '', value)
    # Set path to /keycloak
    value = re.sub(r'; Path=[^;]+', '; Path=/keycloak', value)
    # Ensure SameSite
    if 'SameSite' not in value:
        value += '; SameSite=Lax'


Content Modification¶
Nexus rewrites HTML forms to use proxy URLs:
Original Keycloak Form:
<form action="/realms/hivematrix/login-actions/authenticate" method="post">

Problem: Form submits to /realms/... which is not proxied.
Rewritten by Nexus:
<form action="https://your-server/keycloak/realms/hivematrix/login-actions/authenticate" method="post">

Implementation:
# Replace relative form actions
text = re.sub(r'action="/', f'action="{nexus_proxy_url}/', text)


User Preferences¶
Theme Management¶
Theme Storage:
- Stored in Codex's agents table
- theme_preference column: 'light' or 'dark'
- Requires user to be synced from Keycloak first
Theme Retrieval:
# GET /api/public/user/theme?email=user@example.com
{
  "theme": "dark"
}

Theme Caching:
- Cached in Flask session for 5 minutes
- Cache key: cached_theme
- Cache timestamp: cached_theme_time
- Reduces Codex API calls
Theme Toggle Flow:
1. User clicks theme toggle button
2. JavaScript applies theme immediately (instant feedback)
3. JavaScript calls /codex/api/my/settings to save
4. If saved successfully, calls /api/invalidate-cache
5. Next page load fetches new theme from Codex
Error Handling:
- If user not synced: Alert with instructions
- If auth error: Alert to refresh page
- If network error: Theme applied locally but not persisted

Home Page Routing¶
Home Page Storage:
- Stored in Codex's agents table
- home_page_preference column: service slug (e.g., 'codex', 'beacon')
Home Page Retrieval:
# GET /api/public/user/home-page?email=user@example.com
{
  "home_page": "codex"
}

Home Page Routing Logic:
if not path:  # User requested root /
    # Get user's preference
    preferred_home = get_user_home_page(token_data)

    # Validate user has access
    if preferred_home in services:
        service_config = services[preferred_home]
        # Check permissions
        if service_config.get('admin_only') and user_permission != 'admin':
            preferred_home = None  # Fall back
        elif not service_config.get('visible'):
            preferred_home = None  # Fall back

    # If valid, redirect
    if preferred_home:
        return redirect(f'/{preferred_home}/')

    # Otherwise, use first accessible service
    # ...fallback logic...

Fallback Chain:
1. User's preference (if valid and accessible)
2. First accessible service based on permissions
3. Error message if no services available

Preference Caching¶
Cache TTL: 5 minutes (300 seconds)
Cache Keys:
session['cached_theme'] = 'dark'
session['cached_theme_time'] = 1700000000
session['cached_home_page'] = 'codex'
session['cached_home_page_time'] = 1700000000

Cache Validation:
cached_theme = session.get('cached_theme')
cache_time = session.get('cached_theme_time', 0)

if cached_theme and (time.time() - cache_time) < 300:
    return cached_theme  # Use cached value
else:
    # Fetch from Codex and update cache

Cache Invalidation:
# POST /api/invalidate-cache
def invalidate_cache_endpoint():
    session.pop('cached_theme', None)
    session.pop('cached_theme_time', None)
    session.pop('cached_home_page', None)
    session.pop('cached_home_page_time', None)
    return jsonify({'success': True})

When to Invalidate:
- After user updates settings in Codex
- After theme toggle
- After home page change
- Ensures next page load uses fresh data

Configuration¶
Environment Variables¶
Nexus is configured via environment variables (loaded from .flaskenv).
⚠️ Important: .flaskenv is auto-generated by config_manager.py. Do not edit manually.
Flask Configuration¶
# Flask application settings
FLASK_APP=run.py
FLASK_ENV=development  # or production
SECRET_KEY=<generated-secret>  # Session encryption key
SERVICE_NAME=nexus

Service URLs¶
# Backend service URLs
CORE_SERVICE_URL=http://localhost:5000
NEXUS_SERVICE_URL=https://localhost  # External-facing URL
HELM_SERVICE_URL=http://localhost:5004
CODEX_SERVICE_URL=http://localhost:5010

Keycloak Configuration¶
# Keycloak OAuth2 settings
KEYCLOAK_SERVER_URL=https://your-server/keycloak  # External-facing URL
KEYCLOAK_BACKEND_URL=http://localhost:8080        # Internal backend URL
KEYCLOAK_REALM=hivematrix
KEYCLOAK_CLIENT_ID=core-client
KEYCLOAK_CLIENT_SECRET=<client-secret>

Important Distinction:
- KEYCLOAK_SERVER_URL: External URL browsers use (through Nexus proxy)
- KEYCLOAK_BACKEND_URL: Internal URL Nexus uses for server-to-server calls
Logging Configuration¶
# Logging settings
LOG_LEVEL=INFO  # DEBUG, INFO, WARNING, ERROR
ENABLE_JSON_LOGGING=true  # Structured JSON logs with correlation IDs


SSL Certificates¶
Nexus requires SSL certificates for HTTPS (port 443).
Certificate Locations:
hivematrix-nexus/
├── certs/
│   ├── nexus.crt  # SSL certificate
│   └── nexus.key  # Private key

Production Certificates:
# Use Let's Encrypt or purchase from CA
# Copy to certs directory
cp /etc/letsencrypt/live/your-domain/fullchain.pem certs/nexus.crt
cp /etc/letsencrypt/live/your-domain/privkey.pem certs/nexus.key

# Set permissions
chmod 644 certs/nexus.crt
chmod 600 certs/nexus.key

Development Certificates (Self-Signed):
cd hivematrix-nexus

# Generate self-signed certificate (valid 365 days)
openssl req -x509 -newkey rsa:4096 \
  -keyout certs/nexus.key \
  -out certs/nexus.crt \
  -days 365 \
  -nodes \
  -subj "/CN=localhost"

# Set permissions
chmod 644 certs/nexus.crt
chmod 600 certs/nexus.key

Certificate Renewal:
# For Let's Encrypt (automated renewal)
certbot renew

# Copy new certificates
cp /etc/letsencrypt/live/your-domain/fullchain.pem certs/nexus.crt
cp /etc/letsencrypt/live/your-domain/privkey.pem certs/nexus.key

# Restart Nexus
cd ../hivematrix-helm
python cli.py restart nexus


Service Registry¶
Nexus reads service configuration from services.json.
Location: hivematrix-nexus/services.json (symlink to ../hivematrix-helm/services.json)
Format:
{
  "codex": {
    "name": "HiveMatrix Codex",
    "description": "Master Data Management",
    "url": "http://localhost:5010",
    "port": 5010,
    "visible": true,
    "admin_only": false,
    "billing_or_admin_only": false
  },
  "core": {
    "name": "HiveMatrix Core",
    "description": "Authentication Service",
    "url": "http://localhost:5000",
    "port": 5000,
    "visible": false,
    "admin_only": false,
    "billing_or_admin_only": false
  }
}

Fields:
- name: Display name
- description: Service description
- url: Backend URL (localhost)
- port: Port number
- visible: Show in side panel navigation
- admin_only: Restrict to admin users
- billing_or_admin_only: Restrict to admin and billing users
Updating Service Registry:
cd hivematrix-helm

# Edit apps_registry.json
vim apps_registry.json

# Regenerate services.json
python install_manager.py update-config

# Restart Nexus to reload config
python cli.py restart nexus


Security¶
TLS Termination¶
Nexus is the only service that binds to external network (0.0.0.0).
Security Model:
# Nexus (port 443) - EXTERNAL ACCESS
app.run(host='0.0.0.0', port=443, ssl_context=('certs/nexus.crt', 'certs/nexus.key'))

# All backend services (ports 5000-5999) - LOCALHOST ONLY
app.run(host='127.0.0.1', port=5010)

Benefits:
- Single SSL certificate for entire platform
- Backend services hidden from external access
- All requests authenticated by Nexus before reaching backends
- Firewall blocks ports 5000-5999 from external access
Network Architecture:
Internet
   │
   │ HTTPS (443)
   ▼
┌──────────────┐
│    Nexus     │  0.0.0.0:443 (TLS termination)
│   (443)      │
└──────┬───────┘
       │
       │ HTTP (localhost)
       ├──────────────────────────┐
       │                          │
       ▼                          ▼
┌────────────┐            ┌────────────┐
│    Core    │            │   Codex    │
│ (5000)     │            │  (5010)    │
│ 127.0.0.1  │            │ 127.0.0.1  │
└────────────┘            └────────────┘


Session Security¶
Cookie Settings:
SESSION_COOKIE_HTTPONLY = True   # Cannot be accessed by JavaScript
SESSION_COOKIE_SAMESITE = 'Lax'  # Only sent with same-site requests
PERMANENT_SESSION_LIFETIME = 3600  # 1 hour expiration

Production Settings:
# In production with HTTPS, set:
SESSION_COOKIE_SECURE = True  # Cookie only sent over HTTPS

Cookie Contents (Encrypted):
{
  'token': 'eyJhbGciOiJSUzI1NiIs...',  # HiveMatrix JWT
  'user': {...},                        # Decoded user data
  'oauth_state': '...',                 # OAuth CSRF token (during login)
  'oauth_nonce': '...',                 # OAuth replay protection
  'cached_theme': 'dark',               # Theme preference (5 min cache)
  'cached_home_page': 'codex'           # Home page preference (5 min cache)
}

Security Features:
- Encryption: Cookie signed with SECRET_KEY (prevents tampering)
- HttpOnly: JavaScript cannot access cookie (prevents XSS)
- SameSite: Cookie only sent with same-site requests (prevents CSRF)
- Secure: Cookie only sent over HTTPS in production
- Expiration: Cookie expires after 1 hour

CSRF Protection¶
OAuth CSRF Protection:
# Generate random state during login
state = secrets.token_urlsafe(32)
session['oauth_state'] = state

# Include in authorization URL
redirect_to_keycloak(state=state)

# Verify state matches during callback
if request.args.get('state') != session.get('oauth_state'):
    return "Invalid state parameter", 401

Why State Parameter?
- Prevents CSRF attacks on OAuth flow
- Ensures callback originated from our login initiation
- Required by OAuth2 specification
Session CSRF Protection:
- SESSION_COOKIE_SAMESITE = 'Lax' prevents cookie from being sent with cross-site requests
- Protects against CSRF attacks on authenticated endpoints

Rate Limiting¶
Nexus implements per-user rate limiting:
Limits:
default_limits=["20000 per hour", "1000 per minute"]

Rate Limit Key:
# Per-user rate limiting (from app.rate_limit_key import get_user_id_or_ip)
def get_user_id_or_ip():
    auth_header = request.headers.get('Authorization', '')
    if auth_header.startswith('Bearer '):
        token = auth_header[7:]
        try:
            payload = jwt.decode(token, options={"verify_signature": False})
            return f"user:{payload.get('sub', 'unknown')}"
        except:
            pass
    return f"ip:{request.remote_addr}"

Storage:
storage_uri="memory://"  # In-memory storage for Nexus

Why Higher Limits?
- Nexus is a gateway, not an endpoint
- All user requests flow through Nexus
- Each page load = multiple proxied requests (HTML, CSS, images, API calls)
- Higher limits prevent legitimate users from being blocked

Monitoring & Observability¶
Health Checks¶
Nexus provides comprehensive health monitoring:
Endpoint: GET /health
Components Checked:
1. Disk Space: Usage percentage, free space
2. Core Service: Availability and response time
3. Keycloak: Availability and response time
Example Response (Healthy):
{
  "service": "nexus",
  "status": "healthy",
  "timestamp": "2025-11-22T10:30:00.000Z",
  "checks": {
    "disk": {
      "status": "healthy",
      "usage_percent": 45.67,
      "free_gb": 123.45,
      "total_gb": 250.0
    },
    "dependencies": {
      "core": {
        "status": "healthy",
        "response_time_ms": 15
      },
      "keycloak": {
        "status": "healthy",
        "response_time_ms": 42
      }
    }
  }
}

Status Codes:
- 200 OK: All checks healthy
- 503 Service Unavailable: One or more checks unhealthy or degraded
Use for Load Balancing:
# Nginx upstream health check
upstream nexus {
    server 192.168.1.10:443 max_fails=3 fail_timeout=30s;
    check interval=10000 rise=2 fall=3 timeout=5000 type=http;
    check_http_send "GET /health HTTP/1.0\r\n\r\n";
    check_http_expect_alive http_2xx;
}


Logging¶
Nexus implements structured JSON logging with correlation IDs.
Log Format:
{
  "timestamp": "2025-11-22T10:30:00.123Z",
  "level": "INFO",
  "service": "nexus",
  "correlation_id": "a1b2c3d4-e5f6-7890",
  "message": "Proxying request to codex",
  "method": "GET",
  "path": "/codex/api/companies",
  "backend_url": "http://localhost:5010/api/companies",
  "user": "john@example.com",
  "status_code": 200,
  "duration_ms": 145
}

Centralized Logging:
from app.helm_logger import init_helm_logger

helm_logger = init_helm_logger('nexus', 'http://localhost:5004')
helm_logger.info("Request proxied", extra={'backend': 'codex', 'status': 200})

View Logs:
cd hivematrix-helm
source pyenv/bin/activate

# View Nexus logs
python logs_cli.py nexus --tail 50

# Filter by level
python logs_cli.py nexus --level ERROR --tail 100


Development¶
Running Locally¶
Prerequisites:
- Python 3.9+
- SSL certificates (self-signed for development)
- Core service running (port 5000)
- Keycloak running (port 8080)
Setup:
cd hivematrix-nexus

# Create virtual environment
python3 -m venv pyenv
source pyenv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Generate self-signed SSL certificate
mkdir -p certs
openssl req -x509 -newkey rsa:4096 \
  -keyout certs/nexus.key \
  -out certs/nexus.crt \
  -days 365 \
  -nodes \
  -subj "/CN=localhost"

# Set permissions
chmod 644 certs/nexus.crt
chmod 600 certs/nexus.key

# Configure environment
# .flaskenv is auto-generated by config_manager.py
cd ../hivematrix-helm
python config_manager.py write-dotenv nexus
cd ../hivematrix-nexus

# Run Nexus
sudo python run.py
# Note: sudo required for port 443

Expected Output:
SessionManager: Using Redis for persistent sessions
Flask-Limiter: Using in-memory storage for rate limiting
 * Running on https://0.0.0.0:443
 * Debug mode: on

Access Points:
- Gateway: https://localhost
- API Docs: https://localhost/docs
- Health Check: https://localhost/health
Browser Warning:
- Self-signed certificates trigger browser warnings
- Click "Advanced" → "Proceed to localhost" (development only)
- For production, use valid SSL certificates

Testing¶
Manual Testing¶
1. Test Health Check:
curl -k https://localhost/health | jq

Expected:
{
  "service": "nexus",
  "status": "healthy",
  "timestamp": "2025-11-22T10:30:00.000Z",
  "checks": {...}
}


2. Test Login Flow:
# Visit in browser
open https://localhost

# Expected flow:
# 1. Redirect to /login
# 2. Redirect to Keycloak
# 3. Login with admin/admin
# 4. Redirect back to Nexus
# 5. Land on home page (Helm dashboard)


3. Test Proxy:
# Get authenticated session cookie
# (Use browser developer tools to copy cookie)

# Test proxying to Codex
curl -k https://localhost/codex/ \
  -H "Cookie: session=<session-cookie>"

# Should return Codex HTML with injected CSS and side panel


4. Test Theme Toggle:
# Visit any page
open https://localhost/codex/

# Click theme toggle in sidebar
# Expected: Page switches to dark mode immediately
# Expected: Console logs show theme saved to Codex
# Expected: Next page load uses dark theme


Load Testing¶
Test proxy performance and rate limiting:
# Install Apache Bench
sudo apt install apache2-utils

# Test health endpoint (no rate limit)
ab -n 1000 -c 10 -k https://localhost/health

# Test proxy endpoint (requires authentication)
# Note: -H flag adds session cookie
ab -n 100 -c 5 -k \
  -H "Cookie: session=<session-cookie>" \
  https://localhost/codex/


SSL Certificate Generation¶
Self-Signed Certificate (Development):
cd hivematrix-nexus/certs

# Generate certificate (valid 365 days)
openssl req -x509 -newkey rsa:4096 \
  -keyout nexus.key \
  -out nexus.crt \
  -days 365 \
  -nodes \
  -subj "/CN=localhost"

# Set permissions
chmod 644 nexus.crt
chmod 600 nexus.key

Production Certificate (Let's Encrypt):
# Install certbot
sudo apt install certbot

# Generate certificate (requires DNS or HTTP challenge)
sudo certbot certonly --standalone \
  -d your-domain.com \
  -d www.your-domain.com

# Copy to Nexus
sudo cp /etc/letsencrypt/live/your-domain.com/fullchain.pem \
  /path/to/hivematrix-nexus/certs/nexus.crt
sudo cp /etc/letsencrypt/live/your-domain.com/privkey.pem \
  /path/to/hivematrix-nexus/certs/nexus.key

# Set ownership and permissions
sudo chown $USER:$USER certs/nexus.*
chmod 644 certs/nexus.crt
chmod 600 certs/nexus.key

# Auto-renewal (certbot handles this automatically)
sudo systemctl enable certbot.timer

Certificate Verification:
# Verify certificate
openssl x509 -in certs/nexus.crt -text -noout

# Check expiration date
openssl x509 -in certs/nexus.crt -noout -enddate

# Test certificate with server
openssl s_client -connect localhost:443 -showcerts


Troubleshooting¶
502 Bad Gateway¶
Symptoms:
- Nexus returns 502 error
- "Proxy error: ..." message
Causes & Solutions:
1. Backend Service Not Running:
# Check which service
# Error message shows: "Proxy error: Connection refused"

# Check service status
cd hivematrix-helm
python cli.py status

# Start missing service
python cli.py start codex

2. Backend Service URL Wrong:
# Check services.json
cat services.json | jq '.codex.url'

# Should be: "http://localhost:5010"
# If wrong, regenerate config:
python install_manager.py update-config
python cli.py restart nexus

3. Backend Service Crashed:
# Check service logs
python logs_cli.py codex --tail 50

# Look for errors, restart service
python cli.py restart codex


Authentication Redirect Loop¶
Symptoms:
- Browser keeps redirecting between /login and /keycloak-callback
- Never reaches authenticated pages
Causes & Solutions:
1. Keycloak Not Running:
# Check Keycloak
curl http://localhost:8080/health

# If connection refused, start Keycloak:
cd ../keycloak-26.4.0
bin/kc.sh start-dev

2. Client Secret Mismatch:
# Check Nexus config
grep KEYCLOAK_CLIENT_SECRET .flaskenv

# Check Core config
grep KEYCLOAK_CLIENT_SECRET ../hivematrix-core/.flaskenv

# Should match! If not, regenerate:
cd ../hivematrix-helm
python config_manager.py sync-all

3. Callback URL Wrong:
# Check Keycloak client configuration
# Admin console: http://localhost:8080
# Realm: hivematrix
# Client: core-client
# Valid Redirect URIs should include: https://your-server/*

# If wrong, reconfigure Keycloak:
cd ../hivematrix-helm
rm instance/configs/master_config.json
./start.sh

4. State Parameter Mismatch:
# Check browser console for errors
# Look for: "Invalid state parameter"

# This happens if:
# - Cookies disabled
# - Incognito mode (session not preserved)
# - Multiple tabs (state collision)

# Solution: Clear cookies, use single tab


CSS Not Loading¶
Symptoms:
- Pages load but have no styling
- Browser console shows 404 for CSS files
Causes & Solutions:
1. Static Files Missing:
# Check static directory exists
ls static/css/

# Should contain: global.css, side-panel.css

# If missing, check repository:
git status
git checkout static/css/

2. Cache-Busting Version Wrong:
# Check version in app/version.py
cat app/version.py

# Should match what's requested:
# <link rel="stylesheet" href="/static/css/global.css?v=4.1">

# If mismatch, restart Nexus:
cd ../hivematrix-helm
python cli.py restart nexus

3. Static File Caching Disabled:
# Check app/__init__.py
grep SEND_FILE_MAX_AGE app/__init__.py

# Should be: app.config['SEND_FILE_MAX_AGE_DEFAULT'] = 0

# If wrong, fix and restart

4. BeautifulSoup Injection Failing:
# Check Nexus logs for BeautifulSoup errors
cd ../hivematrix-helm
python logs_cli.py nexus --tail 50 | grep -i beautiful

# If errors, check backend service HTML is valid


Theme Not Applying¶
Symptoms:
- Theme toggle doesn't work
- Always shows light theme
- Browser console shows errors
Causes & Solutions:
1. User Not Synced from Keycloak:
# Check if agent exists in Codex
curl -s -H "Authorization: Bearer $TOKEN" \
  http://localhost:5010/api/public/user/theme?email=admin@example.com

# If 404: User not synced

# Solution: Sync from Keycloak
# Visit: https://localhost/codex/settings
# Click "Sync from Keycloak"

2. Codex API Not Accessible:
# Check Codex running
cd hivematrix-helm
python cli.py status

# If down, start it:
python cli.py start codex

3. Cache Not Invalidated:
# Theme saved to Codex but Nexus still uses old value

# Force cache invalidation:
curl -X POST https://localhost/api/invalidate-cache \
  -H "Cookie: session=<session-cookie>"

# Or wait 5 minutes for cache to expire

4. JavaScript Errors:
# Check browser console for errors
# Common issues:
# - fetch() missing credentials: 'same-origin'
# - CORS errors (shouldn't happen with same-origin)
# - Authentication errors (401/403)

# Fix: Check theme toggle JavaScript was injected correctly
view-source:https://localhost/codex/
# Search for: function toggleTheme()


Keycloak Proxy Not Working¶
Symptoms:
- /keycloak/* returns 502 error
- External browsers cannot access Keycloak
- OAuth flow fails
Causes & Solutions:
1. Keycloak Backend URL Wrong:
# Check environment variable
grep KEYCLOAK_BACKEND_URL .flaskenv

# Should be: http://localhost:8080
# If wrong, regenerate config

2. Keycloak Not Running:
# Check Keycloak
curl http://localhost:8080/health

# If down, start it:
cd ../keycloak-26.4.0
bin/kc.sh start-dev

3. URL Rewriting Failing:
# Check Nexus logs for rewriting errors
cd ../hivematrix-helm
python logs_cli.py nexus --tail 50 | grep -i rewrite

# Common issue: Encoding errors (non-UTF8 content)
# Solution: Restart Keycloak, clear browser cache

4. Cookie Issues:
# Check browser developer tools → Cookies
# Look for Keycloak cookies:
# - AUTH_SESSION_ID
# - KC_RESTART

# Path should be: /keycloak
# If path is /auth or /, cookies won't be sent

# Solution: Clear cookies, try again


SSL Certificate Errors¶
Symptoms:
- Browser shows "Your connection is not private"
- Certificate expired or invalid
- Cannot connect to Nexus
Causes & Solutions:
1. Self-Signed Certificate (Development):
# Browser warning is normal for self-signed certs

# Solution: Click "Advanced" → "Proceed to localhost"
# Or add exception to browser

2. Certificate Expired:
# Check expiration
openssl x509 -in certs/nexus.crt -noout -enddate

# If expired, regenerate:
cd certs
openssl req -x509 -newkey rsa:4096 \
  -keyout nexus.key \
  -out nexus.crt \
  -days 365 \
  -nodes \
  -subj "/CN=localhost"

# Restart Nexus
cd ../../hivematrix-helm
python cli.py restart nexus

3. Certificate File Not Found:
# Check files exist
ls -la certs/

# Should show:
# -rw-r--r-- nexus.crt
# -rw------- nexus.key

# If missing, generate certificates (see above)

4. Permission Errors:
# Check permissions
ls -la certs/

# nexus.crt should be: 644 (readable by all)
# nexus.key should be: 600 (readable only by owner)

# Fix permissions:
chmod 644 certs/nexus.crt
chmod 600 certs/nexus.key


See Also¶
Related Services¶

Core - Authentication - JWT minting, token validation, session management
Helm - Orchestration - Service management and centralized logging
All Backend Services - Complete service inventory

Architecture & Design¶

Gateway Pattern & Frontend Composition
Authentication Flow
HTML Injection Architecture
Keycloak OAuth2 Integration
Service Routing

Configuration & Setup¶

Installation Guide - Complete installation walkthrough
SSL/TLS Setup - Certificate configuration
Keycloak Configuration - OAuth2 provider setup
Service Routing - Backend service registration

Security¶

Security Guide - Security best practices
SSL Certificate Management
OAuth2 Security
Session Security

External Resources¶

RFC 6749 - OAuth 2.0 - OAuth2 authorization framework
RFC 7519 - JWT - JSON Web Token specification
RFC 7807 - Problem Details - HTTP API error format
BeautifulSoup Documentation - HTML parsing library
Flask Documentation - Web framework
Requests Documentation - HTTP library
Let's Encrypt - Free SSL certificates

Tools¶

Service CLI: hivematrix-helm/cli.py
Log Viewer: hivematrix-helm/logs_cli.py
Config Manager: hivematrix-helm/config_manager.py
Security Audit: hivematrix-helm/security_audit.py


Last Updated: 2025-11-22
Version: 1.0
Maintained By: HiveMatrix Team

Nexus - API Gateway & Frontend Proxy¶

Table of Contents¶

Overview¶

Gateway Responsibilities¶

Architecture¶

Technology Stack¶

Production Features¶

Per-User Rate Limiting¶

Structured Logging¶

RFC 7807 Problem Details¶

OpenAPI/Swagger Documentation¶

Comprehensive Health Checks¶

Graceful Degradation¶

API Reference¶

Authentication Endpoints¶

GET /login¶

GET /keycloak-callback¶

GET /logout¶

Proxy Routes¶

GET /<service>/*¶

GET /keycloak/<path>¶

Utility Endpoints¶

POST /api/invalidate-cache¶

GET /health¶

GET /docs¶

GET /helpdesk¶

GET /professional-services¶

Authentication Flow¶

OAuth2 Login Flow¶

Token Validation¶

Session Management¶

Logout Flow¶

Reverse Proxy¶

Service Routing¶

Request Forwarding¶

Response Handling¶

SSE Streaming¶

HTML Injection¶

Global CSS Injection¶

Side Panel Navigation¶

Theme Application¶

JavaScript Injection¶

Keycloak Proxy¶

URL Rewriting¶

Cookie Handling¶

Content Modification¶

User Preferences¶

Theme Management¶

Home Page Routing¶

Preference Caching¶

Configuration¶

Environment Variables¶

Flask Configuration¶

Service URLs¶

Keycloak Configuration¶

Logging Configuration¶

SSL Certificates¶

Service Registry¶

Security¶

TLS Termination¶

Session Security¶

CSRF Protection¶

Rate Limiting¶

Monitoring & Observability¶

Health Checks¶

Logging¶

Development¶

Running Locally¶

Testing¶

Manual Testing¶

Load Testing¶

SSL Certificate Generation¶

Troubleshooting¶

502 Bad Gateway¶

Authentication Redirect Loop¶

CSS Not Loading¶

Theme Not Applying¶

Keycloak Proxy Not Working¶

SSL Certificate Errors¶

See Also¶

`GET /login`¶

`GET /keycloak-callback`¶

`GET /logout`¶

`GET /<service>/*`¶

`GET /keycloak/<path>`¶

`POST /api/invalidate-cache`¶

`GET /health`¶

`GET /docs`¶

`GET /helpdesk`¶

`GET /professional-services`¶