Glossary

The NuFi app The chat product, at chat.nufi.me when hosted by NuFi. It holds accounts, conversations, agents and files, and issues the session the other surfaces reuse. Chat is one feature of it; the manual does not call the product "NuFi Chat".

NuFi Console The self-service developer surface at console.nufi.me: gateway keys, your own usage, per-key limits. Also the identity issuer for the agent products.

NuFi Admin Panel The admin surface at admin.app.nufi.me: settings, users, roles, groups, per-role overrides, the audit log. Talks to the app's admin API; keeps its own session.

NuFi Studio The flow canvas at studio.nufi.me: wire a model, documents and tools into a flow, run it, publish it as an endpoint. A vendored Langflow.

NuFi Works The agent-operations app at works.nufi.me: give a team of agents a goal, approve what matters, watch the spend. A vendored Paperclip.

The chooser agents.nufi.me, the page that sends a member to Studio or Works. A route on the console, not a service.

Identity issuer The console's second job: after asking the app who a member is, it mints a short-lived identity for Studio (a signed cookie checked against the console's published keys) and runs an OpenID Connect flow for Works.

NuFi AI Gateway The one gate every model call passes through: LiteLLM with NuFi's guardrails inside. Routing, keys, budgets, rate limits, the five controls, traces and metrics. Reachable at api.codechi.me for NuFi's own deployments and at litellm-proxy:4000 inside the compose stack.

Endpoint The app's name for a group of models behind one URL and key. NuFi deployments have one, pointing at the gateway: called NPUOps in the compose stack's configuration and NuFi on Railway.

Model name What a caller asks the gateway for. It maps to a provider entry in the gateway's configuration, and the two can differ: NuFi's own gateway answers several names with Google Gemini.

backend_type, hardware_id Two labels every registered model carries: the kind of backend (gpu, npu, cloud) and the specific hardware that serves it (gemini-cloud, mac-local). Traces and cost reports aggregate on hardware_id, which is the only truthful record of what ran.

Virtual key A gateway key issued to a person or a service, with its own budget, rate limits and, optionally, model allow-list. Members issue their own in the console. The master key is the operator's, and the app's own endpoint uses it on the compose stack.

Budget The spend cap, in dollars, on a key or on a user in the gateway; refreshed on the configured interval, 30 days for a new console user.

Guardrails, G1 to G4 The five controls at the gateway. G1 blocks prompt injection before the model; G2a logs personal data in the prompt; G2b hides personal data in the answer; G3 blocks an answer that echoes the system prompt; G4 strips tracking images and scripts from the answer.

Policy file deploy/platform/litellm/guardrails/policy.yaml, the only place a guardrail decision is configured: modes, thresholds per source, failure behaviour, actions. Mounted into the gateway; a restart applies a change.

Canonicalisation The first step before any control: Unicode normalised, invisible characters stripped, lookalike letters folded, encoded spans decoded, so obfuscation cannot hide an attack from the detectors.

Span source Who wrote a piece of the prompt: the person (user), the model's earlier turns (assistant), the system prompt, a tool result, or retrieved content (untrusted). Thresholds and the two-detector rule are set per source.

Trace One request as Langfuse records it: user, model, hardware, tokens, latency, cost, and nested observations for each call the request made.

Audit event One record per guardrail decision: a reference code, the control, the verdict and the evidence, never the triggering text.

Agent (in the app) A saved assistant with instructions, tools and knowledge files, shareable with a team.

Skill A reusable instruction bundle, a SKILL.md with optional resources, that an agent in the app can load.

Preset A saved bundle of model, parameters and system prompt, to start a conversation from.

Override A configuration value scoped to a role, group or user that beats the base value, resolved by priority. Scoped overrides.

Capability The unit of authorisation in the admin panel; roles bundle capabilities, users inherit them through roles.

Bootstrap deploy/platform/scripts/bootstrap.sh: fills the replace-me secrets in .env, pulls the images, starts the stack, registers a model, runs the smoke test. Re-running keeps existing values.

The compose stack deploy/platform: the gateway with its guardrails and sidecars, Langfuse, the monitoring trio, the databases, and a copy of the app and the console, on one host.

The wrapper deploy/railway: the app on its own, with its MongoDB and a RAG service, and the image Railway builds around the app's published image.

Resumable streams The app's ability to reconnect and continue a reply after the connection drops; Redis-backed; needed when the app runs as more than one instance.