Architecture

NuFi is a set of small products around one gateway. People use the app, the console, and the two agent products; every model call any of them makes goes through the NuFi AI Gateway, which routes it, checks it, prices it and records it; the gateway calls whichever provider serves the model.

At a glance

   people ──────────────────────────────────────────────────────────────────┐
     │                                                                      │
     ▼                                                                      ▼
  ┌──────────────┐  one cookie  ┌──────────────┐  vouches for   ┌──────────────────────┐
  │ the NuFi app │ ◄──────────► │ NuFi Console │ ─────────────► │ NuFi Studio  (flows) │
  │ chat.nufi.me │              │ console.     │  identity      │ NuFi Works   (agents)│
  └──────┬───────┘              └──────┬───────┘                └──────────┬───────────┘
         │ endpoint key                │ per-user keys                     │ agent key
         ▼                             ▼                                   ▼
  ┌──────────────────────────────────────────────────────────────────────────────────┐
  │                              NuFi AI Gateway                                     │
  │  routing · keys and budgets · five guardrails (G1–G4) · trace · metrics         │
  └───────────┬────────────────────────────────┬──────────────────────┬──────────────┘
              │                                │                      │
              ▼                                ▼                      ▼
     ┌────────────────┐               ┌─────────────────┐    ┌──────────────────┐
     │ model providers│               │ Langfuse        │    │ Prometheus       │
     │ Gemini, OpenAI,│               │ every request:  │    │ Grafana          │
     │ Anthropic, or  │               │ who, model,     │    │ Alertmanager     │
     │ your own server│               │ tokens, cost    │    │                  │
     └────────────────┘               └─────────────────┘    └──────────────────┘

  NuFi Admin Panel (admin.app.nufi.me) ──► the app's admin API: settings, users, roles, groups

What sits where

• The NuFi app is where conversations happen. It holds accounts, conversations, agents and files in its own MongoDB, and sends every completion to the gateway through one configured endpoint, with the signed-in account's id attached to each request.
• NuFi Console is where a developer creates gateway keys and watches their own usage. It reads the app's session cookie, so there is no second sign-in, and it creates the person in the gateway on their first visit with the deployment's default budget and limits. It is also the identity issuer for the agent products: it publishes signing keys, mints a short-lived identity for NuFi Studio, and runs an OpenID Connect flow for NuFi Works, after asking the app who the member is.
• NuFi Studio (a canvas for flows, published as endpoints) and NuFi Works (a team of agents with goals, approvals and budgets) are separate products with their own databases. Their model calls go to the gateway under a model name reserved for agents.
• The gateway is LiteLLM with NuFi's guardrail package inside it. It knows the models, holds the keys, budgets and rate limits, runs the five security controls, and emits one trace and one set of metrics per request. Security explains the controls.
• The providers are whatever the operator registered: NuFi's own gateway ships with Google Gemini behind several names, and any OpenAI-compatible server (vLLM, Ollama, a vendor API) can be added.
• Langfuse keeps a trace per request: the user id the app sent, the model name and the hardware that served it, tokens, latency, cost. Prometheus scrapes the gateway's request and guardrail metrics, Grafana draws them, Alertmanager routes the nine alerts.
• NuFi Admin Panel configures the app: features, users, roles, groups, per-role overrides. It talks to the app's admin API over HTTP and has no database of its own.

Who pays for what

Budgets and rate limits live on gateway keys. A key a developer creates in the console is that person's: its spend and its limits are theirs, and the trace names them. The app's own traffic goes through one endpoint key for the whole deployment (the gateway's master key on the compose stack, a gateway key on Railway), so the gateway's budget on that key covers the app as a whole. Per-person accounting for chat traffic comes from the trace, which carries the account id the app attaches to every request, not from a per-person key. The app can optionally mirror admin-created endpoints into the gateway with a key per endpoint; that is off by default.

Two deployments

NuFi's hosted product is two deployments that meet at the gateway.

A self-hosted NuFi is the compose stack alone: everything in one place, one .env, one script. Deploying NuFi is that path.

Hostnames

On a self-hosted stack the last two rows are yours to name; the compose file binds each of those services to a host port, and the reverse proxy or tunnel in front decides who sees them. Prometheus and Alertmanager have host ports too and are meant for an SSH tunnel, not a hostname.

Sign-in

The app issues the session. The console reads the same cookie, so it needs no sign-in of its own. Studio and Works trust an identity the console issues, which the console builds by asking the app for the member's email and role. The admin panel signs in separately, with the same accounts, and admits only the ADMIN role. Signing out of the app does not revoke a session already issued to Studio or Works; those expire on their own, eight hours by default.

What is next

• Components: every service, in one table.
• Data flow: one message, from the browser to the model and back.