LLM gateways and observability for enterprise buyers
Asked as “LLM gateway”, and as “LLM observability and routing platform”, on behalf of an enterprise B2B company. 48 first choices recorded across the direct, paraphrase, budget and scale prompts, twelve models each.
Standing · first-choice share
29%
Contested · Bifrost 27%
29LiteLLM27Bifrost15Portkey29others
29% of first choices, contested.
By buyer segment
The same question asked on behalf of a different buyer. Each standing is computed within its segment; they sit side by side and are never added together.
The standing
Share is the count of first choices across the direct, paraphrase, budget and scale prompts, over all twelve models, for an enterprise B2B company. Ordered by share.
ProductFirst-choice shareNegative rateLabelsQuadrant
Show the one product at 0%, ordered by negative rate
Bars are the share of first choices, 0 to 100Every product with at least 10 labels here. Every product name links to its product page.
Head to headHead-to-head pages are published for mid-market buyers, so these open on mid-market figures, not enterprise.
All twenty-eight head-to-head pages: the top eight products, each against each
LiteLLM vs Braintrust GatewayLiteLLM vs Cloudflare AI GatewayLiteLLM vs OpenRouterLiteLLM vs Kong AI GatewayPortkey vs HeliconePortkey vs LangfusePortkey vs Braintrust GatewayPortkey vs Cloudflare AI GatewayPortkey vs OpenRouterPortkey vs Kong AI GatewayHelicone vs LangfuseHelicone vs Braintrust GatewayHelicone vs Cloudflare AI GatewayHelicone vs OpenRouterHelicone vs Kong AI GatewayLangfuse vs Braintrust GatewayLangfuse vs Cloudflare AI GatewayLangfuse vs OpenRouterLangfuse vs Kong AI GatewayBraintrust Gateway vs Cloudflare AI GatewayBraintrust Gateway vs OpenRouterBraintrust Gateway vs Kong AI GatewayCloudflare AI Gateway vs OpenRouterCloudflare AI Gateway vs Kong AI GatewayOpenRouter vs Kong AI Gateway
Recommended versus criticized
Every product with at least 10 labels here, on both axes. The 30% line names a quadrant, not the verdict above: that one needs more than 40%.
Criticized challengerCriticized default
Negative label rate →
01
02
03
04
05
06
07
08
09
10Accepted challengerEndorsed leader
0%First-choice share → · lines at 30% share and 25% negative40%
Key
01LiteLLM29%
LiteLLM29%02Bifrost27%
Bifrost27%03Portkey15%
Portkey15%04Braintrust Gateway4%
Braintrust Gateway4%05Langfuse4%
Langfuse4%06Kong AI Gateway2%
Kong AI Gateway2%07Cloudflare AI Gateway2%
Cloudflare AI Gateway2%08Helicone2%
Helicone2%09OpenRouter2%
OpenRouter2%10NeuralTrust TrustGate0%
NeuralTrust TrustGate0%What they warned about
One of twelve models held their first choice under the paraphrase. Claude Haiku 4.5, GPT-5.4 mini, Gemini 3.5 Flash, Perplexity Sonar, Grok 4.20, Mistral Small, DeepSeek V4 Flash, Llama 4 Maverick, Kimi K2, GLM 4.7 FlashX and MiniMax M2.5 changed. A high negative share on a product with few labels is a warning. A low share on a product with many labels is salience, not sentiment.
LiteLLM24%
13 of 54 labels negative · 10 of 12 models · 4 hard negative
“the comparison explicitly says to avoid it for customer-facing apps at scale, real-time features, and workloads needing 99.9%+ uptime” Perplexity Sonar, negative prompt
OpenRouter43%
10 of 23 labels negative · 6 of 12 models · 4 hard negative
“Avoid for: Healthcare (HIPAA), financial services, government, or any jurisdiction requiring strict data localization.” Qwen 3.7 Flash, negative prompt
Cloudflare AI Gateway30%
6 of 20 labels negative · 6 of 12 models
“The Catch: It requires routing your traffic through Cloudflare's servers. If you have strict "zero data egress" ... a self-hosted option like LiteLLM or Bifrost is better suited.” Gemini 3.5 Flash, direct prompt
Kong AI Gateway13%
4 of 30 labels negative · 4 of 12 models
“Generic Cloudflare AI Gateway / Kong (If Misconfigured) Not inherently bad, but enterprises must be cautious” Qwen 3.7 Flash, negative prompt
What they cite
Twelve of the twelve models returned a source list in this edition: four return the pages they cited and eight the search results they consulted.
Sites the answers cite
70 of 72 answers in this category came back with a source list, from 12 of 12 models: citations where the model returns them, or the search results it consulted. 1098 links across 278 sites, every framing counted. Ranked by the number of answers carrying the site or page.
50 answers · 155 citations · 11 models
vendor site · TrueFoundry36 answers · 60 citations · 11 models
27 answers · 35 citations · 10 models
vendor site · CloudZero26 answers · 26 citations · 10 models
21 answers · 22 citations · 9 models
vendor site · Dataiku20 answers · 21 citations · 9 models
vendor site · Braintrust19 answers · 25 citations · 9 models
vendor site · LLMGateway.io18 answers · 43 citations · 10 models
vendor site · Respan18 answers · 18 citations · 10 models
vendor site · MintMCP15 answers · 16 citations · 8 models
vendor site · Orq.ai15 answers · 15 citations · 8 models
vendor site · OpenRouter12 answers · 21 citations · 7 models
Pages the answers cite
The ten pages named in the most answers, by full address. A page here is one the models returned with a recommendation, not one the index endorses.
26 answers · 10 models
20 answers · 9 models
19 answers · 10 models
18 answers · 9 models
18 answers · 10 models
16 answers · 9 models
15 answers · 8 models
14 answers · 8 models
13 answers · 7 models
10 answers · 7 models
Search against answers
Each company's standing in the answers beside its site's footprint in Google search, one row a site: the products the models named on it with their shares, and the share they add up to; monthly searches on Google, and DataForSEO's estimate of AI search demand (modeled from search signals, directional, not a count of queries to any assistant), for the most-searched of the company's and its products' names (the name is in each row's hover text); estimated monthly organic visits to the site; and its best position in Google's top ten for “best llm gateway”, “llm gateway”, “llm gateways”. US estimates from DataForSEO and Google's Ads Transparency Center. A small company's site, or a mid-sized company's site for its flagship, is marked company; a product on a large parent's site (Google, Microsoft) has no site figures. A column with no figures for this category is left out, and an empty cell means none were seen, not none exist. Two measurements side by side: neither is read as the cause of the other.
| Company and products | Share | Own site cited | Name searches, Google | AI search demand, est. | Organic visits | Paid search a month, est. | Google ads, last 30 days | Best Google position |
|---|---|---|---|---|---|---|---|---|
| LiteLLM litellm.ai | 29% | 196 | 22,200 | 264 | 15,281 | 0 | 2 | |
| Bifrost getmaxim.ai | 27% | 120 | 12,100 | 427 | 4,289 | 10 | 8 | |
| Portkey portkey.ai | 15% | 156 | 5,400 | 131 | 5,999 | 0 | 5 | |
| Braintrust Gateway braintrust.dev | 4% | 15 | 30 | 0 | 14,321 | 69 | 5 | |
| TrueFoundry AI Gateway truefoundry.com | 4% | 24 | 47,088 | 2 | 3 | |||
| Langfuse langfuse.com | 4% | 48 | 18,100 | 216 | 8,530 | 4 | ||
| Confident AI confident-ai.com | 4% | 10 | ||||||
| Kong konghq.com Kong AI Gateway 2% | 2% | 85 | 880 | 10 | 20,236 company | $376 | 62 | |
| Cloudflare parent site Cloudflare AI Gateway 2% | 2% | 96 | 550,000 | 10,897 | ||||
| Helicone helicone.ai | 2% | 50 | 1,600 | 899 | 2,439 | 0 |
Google's top ten for the three searches hold 18 sites; 8 of them are among the sites the models cited here (braintrust.dev, dataiku.com, dev.to, getmaxim.ai, llmgateway.io, openrouter.ai, orq.ai, truefoundry.com). The first result for “best llm gateway” is reddit.com.
How they answered
Six framings of the same buying question, each sent to every model in a fresh session with search on. One row per model, so a row shows whether it held its answer under rewording, what it named when cost was the constraint, and what it argued against. Computed from the raw judge labels.ShowHide
How they answered
Six framings of the same buying question, each sent to every model in a fresh session with search on. One row per model, so a row shows whether it held its answer under rewording, what it named when cost was the constraint, and what it argued against. Computed from the raw judge labels.
ShowHideModelDirect“What is the best LLM gateway for an enterprise B2B company?”Paraphrase“Which LLM observability and routing platform would you recommend to a large B2B company with thousands of employees?”Comparative“What are the top enterprise-grade LLM gateways and how do they differ?”Budget-constrained“What is the best LLM gateway for a large company that needs predictable total cost across thousands of users?”Scale-constrained“We are a 5,000 person company with SSO, SOC 2 and procurement review requirements evaluating a LLM gateway. What should we look at?”Negative“Which LLM gateways should a large enterprise avoid or be cautious about?”
Claude Haiku 4.5BifrostTrueFoundry AI GatewayChangedBifrostagainst: Cloudflare AI Gateway, LiteLLMBifrostno first choiceagainst: LiteLLM
Three alternatives
Kong AI Gateway, LiteLLM, PortkeyFive alternatives
Datadog LLM Observability, LangSmith, MLflow, New Relic, OpenObserveTwo alternatives
Kong AI Gateway, PortkeyTwo alternatives
Cloudflare AI Gateway, Kong AI GatewayGPT-5.4 miniAmazon Bedrock AgentCore GatewayLangfuse, OpenRouterChangedKong AI Gateway, PortkeyKong AI Gatewayno first choicenothing named
Three alternatives
Kong AI Gateway, LiteLLM, PortkeyThree alternatives
Kong AI Gateway, LiteLLM, PortkeyOne alternative
LiteLLMTwo alternatives
LiteLLM, OpenRouterGemini 3.5 FlashLiteLLMagainst: Cloudflare AI GatewayLiteLLM, PortkeyChangedagainst: Datadog LLM Observability, HeliconeEnvoy AI GatewayLiteLLMagainst: OpenRouter, Portkey, Requestyno first choiceagainst: Databricks Unity AI Gateway, Langfuse, LiteLLM, One API, Portkey
Three alternatives
Bifrost, Kong AI Gateway, PortkeyOne alternative
LangfuseFour alternatives
Kong AI Gateway, LiteLLM, Portkey, WSO2 AI GatewayTwo alternatives
Bifrost, Kong AI GatewayPerplexity SonarBifrostBraintrust GatewayChangedBifrostagainst: OpenRouterBifrostno first choiceagainst: Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter, Vercel AI Gateway
Seven alternatives
Braintrust Gateway, Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter, Respan, TrueFoundry AI GatewayThree alternatives
Bifrost, Confident AI, LangfuseFive alternatives
Cloudflare AI Gateway, Helicone, Kong AI Gateway, LiteLLM, PortkeyTwo alternatives
LiteLLM, PortkeyGrok 4.20Bifrost, LiteLLMPortkeyChangedagainst: Braintrust Gateway, Langfuse, LiteLLMno first choiceBifrost, LiteLLMagainst: Helicone, OpenRouter, PortkeyLiteLLMagainst: Kong AI Gateway, LiteLLM, Portkey
Three alternatives
Cloudflare AI Gateway, Kong AI Gateway, PortkeyFour alternatives
Arize AI, Datadog LLM Observability, Helicone, LangSmithFour alternatives
Bifrost, LLM Gateway, OpenRouter, PortkeyMistral SmallBifrostPortkeyChangedNeuralTrust TrustGateBifrost, LiteLLMno first choiceagainst: LiteLLM
Three alternatives
LiteLLM, NeuralTrust TrustGate, PortkeyTwo alternatives
Confident AI, Nexos.aiFive alternatives
Braintrust Gateway, Kong AI Gateway, LiteLLM, Martian, PortkeyOne alternative
PortkeyTwo alternatives
Amazon Bedrock Guardrails, Azure AI GatewayDeepSeek V4 FlashPortkeyagainst: Kong AI GatewayLangfuseChangedagainst: LangSmithPortkeyBifrost, LiteLLMagainst: Portkeyno first choiceagainst: Cloudflare AI Gateway, Helicone, LiteLLM, OpenRouter
Two alternatives
Bifrost, LiteLLMFour alternatives
Arize, Datadog LLM Observability, LiteLLM, PortkeySeven alternatives
Bifrost, Braintrust Gateway, Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter, TrueFoundry AI GatewayLlama 4 MaverickTrueFoundry AI GatewayBraintrust GatewayChangedBifrostLiteLLMno first choicenothing named
Five alternatives
Bifrost, LiteLLM, NeuralTrust TrustGate, OpenRouter, PortkeyThree alternatives
Arize AI, Confident AI, OpenObserveFour alternatives
Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouterThree alternatives
Bifrost, OpenRouter, PortkeyQwen 3.7 FlashLiteLLM, PortkeyLiteLLM, PortkeyHeldNeuralTrust TrustGateagainst: OpenRouterBifrost, LiteLLMagainst: OpenRouterno first choiceagainst: Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter
Two alternatives
Maxim AI, NeuralTrust TrustGateThree alternatives
Arize, LangSmith, SemaFour alternatives
Bifrost, Kong AI Gateway, LiteLLM, PortkeyKimi K2Bifrost, LiteLLMPortkeyChangedBifrostagainst: Cloudflare AI Gateway, LiteLLMLiteLLMno first choicenothing named
Four alternatives
Cloudflare AI Gateway, Kong AI Gateway, Portkey, TrueFoundry AI GatewayFive alternatives
Arize AI, Braintrust Gateway, LangSmith, Langfuse, LiteLLMFour alternatives
Azure API Management, Kong AI Gateway, Portkey, ZuploOne alternative
BifrostGLM 4.7 FlashXBifrostagainst: OpenRouterConfident AIChangedBifrost, Portkeyagainst: LiteLLMBifrostagainst: OpenRouterno first choiceagainst: LiteLLM
Three alternatives
Cloudflare AI Gateway, Kong AI Gateway, LiteLLMEight alternatives
Amazon Bedrock, Arize AX, Arize Phoenix, Azure OpenAI Service, Braintrust Gateway, Datadog LLM Observability, Helicone, LangfuseTwo alternatives
Helicone, Kong AI GatewayOne alternative
LiteLLMMiniMax M2.5Cloudflare AI Gateway, HeliconeConfident AIChangedBifrost, LiteLLMLLM Gatewayno first choiceagainst: LiteLLM
Two alternatives
Braintrust Gateway, NeuralTrust TrustGateFive alternatives
Arize AI, Datadog LLM Observability, LangSmith, Langfuse, MLflowFour alternatives
Cloudflare AI Gateway, Kong AI Gateway, OpenRouter, PortkeyTwo alternatives
LiteLLM, Standard ComputeBold is the first choiceAlternatives are counted; the count opens them.What the answer argued against
The record
One row per call: the version string exactly as returned, whether the model searched, sources cited, and latency.
Seventy-two rows: every prompt, every model, every answer.
| Prompt | Model | Version string | Time (UTC) | Searched | Sources | Latency |
|---|---|---|---|---|---|---|
| Direct recommendation | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 11:44 | yes | 9 | 10 s |
| Direct recommendation | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 13:21 | yes | 3 | 5 s |
| Direct recommendation | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 11:15 | yes | 13 | 20 s |
| Direct recommendation | Perplexity Sonar | sonar | 2026-09-17 10:03 | yes | 20 | 6 s |
| Direct recommendation | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 12:20 | yes | 42 | 21 s |
| Direct recommendation | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 12:20 | yes | 5 | 4 s |
| Direct recommendation | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 13:53 | yes | 10 | 16 s |
| Direct recommendation | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 10:19 | yes | 5 | 2 s |
| Direct recommendation | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 10:02 | yes | 5 | 34 s |
| Direct recommendation | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 13:29 | yes | 8 | 48 s |
| Direct recommendation | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 12:22 | yes | 22 | 129 s |
| Direct recommendation | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 09:40 | yes | 5 | 16 s |
| Paraphrase | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 11:27 | yes | 9 | 9 s |
| Paraphrase | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 13:22 | yes | 6 | 6 s |
| Paraphrase | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 11:19 | yes | 20 | 27 s |
| Paraphrase | Perplexity Sonar | sonar | 2026-09-17 11:19 | yes | 20 | 6 s |
| Paraphrase | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 14:05 | yes | 49 | 23 s |
| Paraphrase | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 11:11 | yes | 5 | 4 s |
| Paraphrase | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 10:56 | yes | 27 | 24 s |
| Paraphrase | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 13:02 | yes | 5 | 4 s |
| Paraphrase | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 11:48 | no | 0 | 39 s |
| Paraphrase | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 10:04 | yes | 15 | 78 s |
| Paraphrase | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 11:26 | yes | 14 | 24 s |
| Paraphrase | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 13:12 | yes | 13 | 30 s |
| Comparative | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 14:08 | yes | 9 | 10 s |
| Comparative | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 12:32 | yes | 4 | 8 s |
| Comparative | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 12:26 | yes | 17 | 29 s |
| Comparative | Perplexity Sonar | sonar | 2026-09-17 14:09 | yes | 20 | 13 s |
| Comparative | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 13:49 | yes | 32 | 16 s |
| Comparative | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 10:17 | yes | 9 | 10 s |
| Comparative | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 10:21 | yes | 18 | 24 s |
| Comparative | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 11:46 | yes | 5 | 2 s |
| Comparative | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 13:41 | yes | 5 | 31 s |
| Comparative | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 11:02 | yes | 22 | 45 s |
| Comparative | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 10:37 | yes | 29 | 57 s |
| Comparative | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 11:34 | yes | 10 | 19 s |
| Budget constrained | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 09:52 | yes | 15 | 11 s |
| Budget constrained | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 13:03 | yes | 3 | 5 s |
| Budget constrained | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 11:08 | yes | 26 | 29 s |
| Budget constrained | Perplexity Sonar | sonar | 2026-09-17 12:28 | yes | 20 | 7 s |
| Budget constrained | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 10:24 | yes | 42 | 17 s |
| Budget constrained | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 13:07 | yes | 5 | 5 s |
| Budget constrained | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via deepinfra | 2026-09-17 13:52 | yes | 28 | 49 s |
| Budget constrained | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 13:12 | yes | 5 | 3 s |
| Budget constrained | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 11:40 | yes | 15 | 47 s |
| Budget constrained | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 09:24 | yes | 9 | 52 s |
| Budget constrained | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 10:25 | yes | 22 | 55 s |
| Budget constrained | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 13:40 | yes | 18 | 63 s |
| Scale constrained | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 10:28 | yes | 17 | 10 s |
| Scale constrained | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 12:05 | yes | 1 | 10 s |
| Scale constrained | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 12:47 | yes | 12 | 32 s |
| Scale constrained | Perplexity Sonar | sonar | 2026-09-17 11:54 | yes | 20 | 11 s |
| Scale constrained | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 12:30 | yes | 14 | 13 s |
| Scale constrained | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 09:31 | no | 0 | 9 s |
| Scale constrained | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 13:18 | yes | 10 | 22 s |
| Scale constrained | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 10:39 | yes | 5 | 4 s |
| Scale constrained | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 13:57 | yes | 9 | 44 s |
| Scale constrained | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 10:26 | yes | 15 | 45 s |
| Scale constrained | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 13:11 | yes | 24 | 205 s |
| Scale constrained | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 13:45 | yes | 5 | 22 s |
| Negative framing | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 13:25 | yes | 18 | 9 s |
| Negative framing | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 12:05 | yes | 4 | 7 s |
| Negative framing | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 10:48 | yes | 19 | 31 s |
| Negative framing | Perplexity Sonar | sonar | 2026-09-17 09:59 | yes | 20 | 6 s |
| Negative framing | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 09:59 | yes | 43 | 15 s |
| Negative framing | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 13:42 | yes | 14 | 9 s |
| Negative framing | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via deepinfra | 2026-09-17 14:11 | yes | 29 | 66 s |
| Negative framing | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 10:05 | yes | 5 | 2 s |
| Negative framing | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 11:19 | yes | 15 | 39 s |
| Negative framing | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 11:22 | yes | 28 | 59 s |
| Negative framing | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 13:14 | yes | 29 | 42 s |
| Negative framing | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 10:55 | yes | 24 | 46 s |
Normalization in this category
Every judgment call made between the raw labels and the numbers above, listed so it is visible and reversible.
ShowHide
Category-scoped readings
Braintrust read as Braintrust Gateway
Cloudflare read as Cloudflare AI Gateway
Datadog read as Datadog LLM Observability
Kong read as Kong AI Gateway
Kong AI read as Kong AI Gateway
NeuralTrust read as NeuralTrust TrustGate
Tetrate read as Tetrate Agent Router Enterprise
TrueFoundry read as TrueFoundry AI Gateway
Unresolved, counted raw
APIM solutions
Amazon Bedrock AgentCore Gateway
Cequence AI Gateway
Featherless.ai
Galileo AI
Highflame
Mirascope
SS&C AI Gateway
Sema
Standard Compute
Discontinued, still offered
No shut-down product was recommended here.