Merged
Size
XL
Change Breakdown
Feature90%
Dependencies5%
Config5%
#3270feat: add Model Registry feature with catalog pipeline, dashboard pages, and TSQL schema

Model Registry adds catalog pages, comparison dashboard, and AI model tracking

Model Registry adds catalog pages, comparison dashboard, and AI model tracking

Users can now browse AI models in a catalog, view detailed metrics per model, and compare performance across multiple models side-by-side. The feature includes a new sidebar section under AI with feature flag controls.

The Model Registry feature brings a centralized hub for tracking and comparing AI models across an organization.

Before this change, users had access to AI Metrics but no way to browse available models or understand their capabilities. Model pricing was handled separately, with no unified view of which models were available, their specifications, or how they performed relative to each other.

The new catalog page displays models as cards with key information like provider, context window, and pricing tiers. Users can filter by provider and capability, with hidden/deprecated models excluded by default. The detail page shows comprehensive metrics including total calls, cost breakdown, latency percentiles (p50/p90/p95/p99), and throughput metrics over time. A cost estimator lets users project expenses based on expected token usage. The comparison page lets users select 2-4 models to see their metrics side-by-side, with the best performer highlighted for each metric.

The feature integrates with ClickHouse through a new aggregate table llm_model_aggregates_v1 that tracks cross-tenant model performance data. The base_response_model column groups dated variants under their base model (e.g., gpt-4o-2024-08-06 groups to gpt-4o), enabling trend analysis across model versions. The TSQL schema was updated to support these global queries, and the sidebar navigation was expanded under the AI section with a new "Models" menu item controlled by the hasAiModelsAccess feature flag.

Admins can sync catalog metadata from the model-catalog.json (which is auto-generated via Claude CLI research), and the seed process now populates catalog fields like provider, description, context window, and capabilities.

View Original GitHub Description
  • Add llm-model-catalog package (renamed from llm-pricing) with Claude CLI research pipeline
  • Add Prisma schema: catalog columns + baseModelName on LlmModel
  • Add ClickHouse: llm_model_aggregates MV + base_response_model column
  • Add TSQL llm_models schema for query page integration
  • Add ModelRegistryPresenter with catalog, metrics, and comparison queries
  • Add 3 dashboard pages: catalog (cards+table+filters), detail (overview+metrics+cost estimator), compare
  • Add sidebar navigation under AI section with hasAiAccess feature flag
  • Add admin dashboard sync/seed for catalog metadata
  • Add model variant grouping (dated snapshots under base models)
  • Add shared formatters and design system component usage

refs TRI-7941

© 2026 · via Gitpulse