September 2026 Edition, published · The permanent record of that edition, superseded since. The current standing is on the category page →
Index › IT › Data platform › LLM gateways · September 2026 Edition

LLM gateways and observability

Asked as “LLM gateway”, and as “LLM observability and routing platform”, on behalf of a mid-market B2B company. 45 first choices recorded across the direct, paraphrase, budget and scale prompts, twelve models each.
Standing · first-choice share
38%
Contested · Portkey 29%
38LiteLLM29Portkey09Helicone24others

38% of first choices, contested.

By buyer segment

The same question asked on behalf of a different buyer. Each standing is computed within its segment; they sit side by side and are never added together.

The standing

Share is the count of first choices across the direct, paraphrase, budget and scale prompts, over all twelve models, for a mid-market B2B company. Ordered by share.
ProductFirst-choice shareNegative rateLabelsQuadrant
01LiteLLM38%18%51endorsed leader
02Portkey29%2%44accepted challenger
03Helicone9%13%30accepted challenger
04Langfuse9%0%11accepted challenger
05Braintrust Gateway4%0%17accepted challenger
06Cloudflare AI Gateway2%4%25accepted challenger
07OpenRouter2%23%31accepted challenger
08Kong AI Gateway2%17%18accepted challenger
Show the one product at 0%, ordered by negative rate
09Bifrost0%0%21accepted challenger
Bars are the share of first choices, 0 to 100Every product with at least 10 labels here. Every product name links to its product page.
All twenty-eight head-to-head pages: the top eight products, each against each

Recommended versus criticized

Every product with at least 10 labels here, on both axes. The 30% line names a quadrant, not the verdict above: that one needs more than 40%.

Criticized challengerCriticized default
Negative label rate →
01
02
03
04
05
06
07
08
09
Accepted challengerEndorsed leader
0%First-choice share → · lines at 30% share and 25% negative50%
Key
01LiteLLM38%
02Portkey29%
03Helicone9%
04Langfuse9%
05Braintrust Gateway4%
06Cloudflare AI Gateway2%
07OpenRouter2%
08Kong AI Gateway2%
09Bifrost0%

What they warned about

One of twelve models held their first choice under the paraphrase. Claude Haiku 4.5, GPT-5.4 mini, Gemini 3.5 Flash, Perplexity Sonar, Mistral Small, DeepSeek V4 Flash, Llama 4 Maverick, Qwen 3.7 Flash, Kimi K2, GLM 4.7 FlashX and MiniMax M2.5 changed. A high negative share on a product with few labels is a warning. A low share on a product with many labels is salience, not sentiment.
LiteLLM
18%
9 of 51 labels negative · 9 of 12 models · 5 hard negative
“Be cautious with self-hosted or open-source LLM gateways like LiteLLM and Ollama, especially if internet-exposed, due to documented security risks... actively exploited and CISA KEV-listed” Grok 4.20, negative prompt
OpenRouter
23%
7 of 31 labels negative · 5 of 12 models
“routing sensitive enterprise or user data through a third-party hosted cloud gateway introduces strict compliance and privacy risks” Gemini 3.5 Flash, negative prompt
Helicone
13%
4 of 30 labels negative · 3 of 12 models · 2 hard negative
“Avoid depending on Helicone — it's now in maintenance mode” DeepSeek V4 Flash, budget prompt
Kong AI Gateway
17%
3 of 18 labels negative · 3 of 12 models
“the operational weight is hard to justify for LLM routing alone if you're not already a Kong shop” Claude Haiku 4.5, comparative prompt

What they cite

Twelve of the twelve models returned a source list in this edition: four return the pages they cited and eight the search results they consulted.

Sites the answers cite

65 of 72 answers in this category came back with a source list, from 12 of 12 models: citations where the model returns them, or the search results it consulted. 951 links across 247 sites, every framing counted. Ranked by the number of answers carrying the site or page.

31 answers · 58 citations · 11 models
vendor site · Braintrust29 answers · 39 citations · 10 models
vendor site · Respan29 answers · 35 citations · 10 models
27 answers · 48 citations · 10 models
vendor site · CloudZero27 answers · 27 citations · 10 models
vendor site · Orq.ai22 answers · 24 citations · 10 models
vendor site · Dataiku21 answers · 24 citations · 10 models
vendor site · TrueFoundry19 answers · 29 citations · 8 models
19 answers · 19 citations · 10 models
vendor site · Contabo13 answers · 13 citations · 6 models
12 answers · 16 citations · 5 models
vendor site · Opper12 answers · 12 citations · 7 models

Pages the answers cite

The ten pages named in the most answers, by full address. A page here is one the models returned with a recommendation, not one the index endorses.

Search against answers

Each company's standing in the answers beside its site's footprint in Google search, one row a site: the products the models named on it with their shares, and the share they add up to; monthly searches on Google, and DataForSEO's estimate of AI search demand (modeled from search signals, directional, not a count of queries to any assistant), for the most-searched of the company's and its products' names (the name is in each row's hover text); estimated monthly organic visits to the site; and its best position in Google's top ten for “best llm gateway”, “llm gateway”, “llm gateways”. US estimates from DataForSEO and Google's Ads Transparency Center. A small company's site, or a mid-sized company's site for its flagship, is marked company; a product on a large parent's site (Google, Microsoft) has no site figures. A column with no figures for this category is left out, and an empty cell means none were seen, not none exist. Two measurements side by side: neither is read as the cause of the other.
Company and productsShareOwn site citedName searches, GoogleAI search demand, est.Organic visitsPaid search a month, est.Google ads, last 30 daysBest Google position
LiteLLM litellm.ai38%19622,20026415,28102
Portkey portkey.ai29%1565,4001315,99905
Helicone helicone.ai9%501,6008992,4390
Langfuse langfuse.com9%4818,1002168,5304
Braintrust Gateway braintrust.dev4%1530014,321695
Cloudflare parent site
Cloudflare AI Gateway 2%
2%96550,00010,897
OpenRouter openrouter.ai2%118135,000655225,878687
Kong konghq.com
Kong AI Gateway 2%
2%858801020,236 company$37662
Merge Gateway merge.dev2%17
Nexos.ai nexos.ai2%3

Google's top ten for the three searches hold 18 sites; 7 of them are among the sites the models cited here (braintrust.dev, dataiku.com, dev.to, getmaxim.ai, orq.ai, reddit.com, truefoundry.com). The first result for “best llm gateway” is reddit.com.

How they answered

Six framings of the same buying question, each sent to every model in a fresh session with search on. One row per model, so a row shows whether it held its answer under rewording, what it named when cost was the constraint, and what it argued against. Computed from the raw judge labels.
ShowHide
ModelDirect“What is the best LLM gateway for a mid-market B2B company?”Paraphrase“Which LLM observability and routing platform would you recommend to a mid-sized B2B company?”Comparative“What are the top LLM gateways and how do they differ?”Budget-constrained“What is the best LLM gateway for a company with a limited budget?”Scale-constrained“We are a 500 person company evaluating a LLM gateway. What should we look at?”Negative“Which LLM gateways should I avoid or be cautious about?”
Claude Haiku 4.5Portkey
Two alternativesLiteLLM, OpenRouter
Braintrust GatewayChanged
Six alternativesArize, Arize Phoenix, Langfuse, OpenObserve, Portkey, TrueFoundry AI Gateway
against: Datadog LLM Observability
LiteLLM
Three alternativesBifrost, OpenRouter, Portkey
against: Kong AI Gateway
LiteLLM, OpenRouter
One alternativeCloudflare AI Gateway
no first choicenothing named
GPT-5.4 miniPortkey
Two alternativesKong AI Gateway, LiteLLM
LangfuseChanged
Two alternativesHelicone, LiteLLM
LiteLLM, OpenRouter
Five alternativesCloudflare AI Gateway, Helicone, Kong AI Gateway, Portkey, Vercel AI Gateway
LiteLLM
One alternativePortkey
against: Kong AI Gateway
no first choicenothing named
Gemini 3.5 FlashLiteLLM
Four alternativesBifrost, Cloudflare AI Gateway, Portkey, TrueFoundry AI Gateway
PortkeyChanged
Three alternativesHelicone, Langfuse, LiteLLM
no first choice
Eight alternativesBifrost, Braintrust Gateway, Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter, Portkey, Respan
Cloudflare AI Gateway, LiteLLM
Two alternativesBifrost, OpenRouter
LiteLLM, Portkey
Three alternativesBifrost, Cloudflare AI Gateway, Kong AI Gateway
against: LiteLLM, OpenRouter
Perplexity SonarLiteLLM, Portkey
Three alternativesBifrost, Cloudflare AI Gateway, Kong AI Gateway
LangfuseChanged
Three alternativesBraintrust Gateway, Confident AI, Portkey
no first choiceLiteLLM
One alternativeCloudflare AI Gateway
against: OpenRouter
no first choiceagainst: Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, Portkey
Grok 4.20Portkey
One alternativeLiteLLM
against: OpenRouter
PortkeyHeld
Two alternativesHelicone, LangSmith
no first choiceLiteLLM
Three alternativesCloudflare AI Gateway, OpenRouter, Portkey
no first choice
Five alternativesBifrost, Cloudflare AI Gateway, Helicone, LiteLLM, Portkey
against: LiteLLM, Ollama
Mistral SmallHelicone, LiteLLM
Two alternativesBraintrust Gateway, OpenRouter
Helicone, PortkeyChangedLiteLLM
Five alternativesBifrost, Groq, OpenRouter, Portkey, SambaNova
Helicone, LiteLLM
Two alternativesBifrost, Cloudflare AI Gateway
no first choiceagainst: LMDeploy, LiteLLM, Ollama
DeepSeek V4 FlashLiteLLM
Two alternativesCloudflare AI Gateway, Portkey
against: OpenRouter
PortkeyChanged
One alternativeBraintrust Gateway
against: Helicone, OpenRouter
no first choice
Six alternativesBifrost, Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter, Portkey
LiteLLM
One alternativePortkey
against: Helicone, OpenRouter
no first choiceagainst: LiteLLM
Llama 4 Maverickno first choiceBraintrust GatewayChanged
Four alternativesDynatrace, Helicone, Kong AI Gateway, Portkey
no first choiceLiteLLM
Three alternativesBifrost, OpenRouter, Portkey
no first choicenothing named
Qwen 3.7 FlashPortkey
Three alternativesBifrost, Cloudflare AI Gateway, LiteLLM
LangfuseChanged
Two alternativesBraintrust Gateway, Helicone
against: LangSmith
LiteLLM, OpenRouter, PortkeyLiteLLM
Three alternativesCloudflare AI Gateway, OpenRouter, Portkey
against: Helicone
no first choiceagainst: Kestra, LiteLLM, RAGFlow
Kimi K2LiteLLM
Three alternativesHelicone, Portkey, Requesty
PortkeyChanged
Three alternativesBraintrust Gateway, Helicone, Langfuse
LiteLLM
Six alternativesBraintrust Gateway, Cloudflare AI Gateway, Kong AI Gateway, OpenRouter, Portkey, TrueFoundry AI Gateway
LiteLLM
Three alternativesHelicone, OpenRouter, Portkey
no first choiceagainst: LiteLLM
GLM 4.7 FlashXPortkey
Two alternativesHelicone, LiteLLM
Kong AI Gateway, Nexos.aiChanged
Two alternativesConfident AI, Langfuse
LiteLLM
Three alternativesMerge Gateway, OpenRouter, Portkey
no first choiceno first choiceagainst: Helicone, LiteLLM, OpenRouter
MiniMax M2.5Merge Gateway, Portkey
One alternativeLiteLLM
Helicone, LangfuseChanged
One alternativeFastRouter
against: LangSmith
Braintrust Gateway
Seven alternativesCloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter, Opper, Portkey, Vercel AI Gateway
LiteLLM
Two alternativesCloudflare AI Gateway, OpenRouter
no first choiceagainst: LiteLLM
Bold is the first choiceAlternatives are counted; the count opens them.What the answer argued against

The record

One row per call: the version string exactly as returned, whether the model searched, sources cited, and latency.
Seventy-two rows: every prompt, every model, every answer.
PromptModelVersion stringTime (UTC)SearchedSourcesLatency
Direct recommendationClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 10:52yes98 s
Direct recommendationGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 13:10yes36 s
Direct recommendationGemini 3.5 Flashgemini-3.5-flash2026-09-17 13:27yes1332 s
Direct recommendationPerplexity Sonarsonar2026-09-17 11:44yes207 s
Direct recommendationGrok 4.20spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version)2026-09-17 13:34yes3822 s
Direct recommendationMistral Smallmistral/mistral-small via mistral2026-09-17 13:50yes1511 s
Direct recommendationDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 13:12yes1025 s
Direct recommendationLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 13:14yes75 s
Direct recommendationQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 12:34yes1277 s
Direct recommendationKimi K2moonshotai/kimi-k2 via novita2026-09-17 14:15yes1035 s
Direct recommendationGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 10:04yes13107 s
Direct recommendationMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 12:35yes824 s
ParaphraseClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 13:15yes1710 s
ParaphraseGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 12:41yes36 s
ParaphraseGemini 3.5 Flashgemini-3.5-flash2026-09-17 12:51yes1725 s
ParaphrasePerplexity Sonarsonar2026-09-17 10:06yes206 s
ParaphraseGrok 4.20spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version)2026-09-17 14:14yes3818 s
ParaphraseMistral Smallmistral/mistral-small via mistral2026-09-17 09:40yes53 s
ParaphraseDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 13:19yes1413 s
ParaphraseLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 14:03yes53 s
ParaphraseQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 09:29yes1431 s
ParaphraseKimi K2moonshotai/kimi-k2 via novita2026-09-17 13:09yes2027 s
ParaphraseGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 11:36yes2963 s
ParaphraseMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 12:10yes511 s
ComparativeClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 09:34yes1611 s
ComparativeGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 11:23yes48 s
ComparativeGemini 3.5 Flashgemini-3.5-flash2026-09-17 12:43yes1528 s
ComparativePerplexity Sonarsonar2026-09-17 13:03yes209 s
ComparativeGrok 4.20spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version)2026-09-17 12:06yes4618 s
ComparativeMistral Smallmistral/mistral-small via mistral2026-09-17 11:20yes56 s
ComparativeDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 10:09yes2018 s
ComparativeLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 09:27yes53 s
ComparativeQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 09:39yes1042 s
ComparativeKimi K2moonshotai/kimi-k2 via novita2026-09-17 12:27yes1437 s
ComparativeGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 09:53yes33200 s
ComparativeMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 13:09yes1463 s
Budget constrainedClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 13:02yes107 s
Budget constrainedGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 12:32yes45 s
Budget constrainedGemini 3.5 Flashgemini-3.5-flash2026-09-17 10:08yes1925 s
Budget constrainedPerplexity Sonarsonar2026-09-17 11:11yes176 s
Budget constrainedGrok 4.20spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version)2026-09-17 11:19yes199 s
Budget constrainedMistral Smallmistral/mistral-small via mistral2026-09-17 14:14yes55 s
Budget constrainedDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 12:38yes1011 s
Budget constrainedLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 11:55yes52 s
Budget constrainedQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 12:34yes525 s
Budget constrainedKimi K2moonshotai/kimi-k2 via novita2026-09-17 13:11yes920 s
Budget constrainedGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 10:56yes1575 s
Budget constrainedMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 10:38yes1026 s
Scale constrainedClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 10:10no07 s
Scale constrainedGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 12:34no09 s
Scale constrainedGemini 3.5 Flashgemini-3.5-flash2026-09-17 09:56yes1328 s
Scale constrainedPerplexity Sonarsonar2026-09-17 09:22yes209 s
Scale constrainedGrok 4.20spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version)2026-09-17 11:45yes1910 s
Scale constrainedMistral Smallmistral/mistral-small via mistral2026-09-17 11:53no07 s
Scale constrainedDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 14:08yes1920 s
Scale constrainedLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 12:37yes53 s
Scale constrainedQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 11:34no038 s
Scale constrainedKimi K2moonshotai/kimi-k2 via novita2026-09-17 09:24no043 s
Scale constrainedGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 12:49yes1837 s
Scale constrainedMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 11:14yes917 s
Negative framingClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 11:10no04 s
Negative framingGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 11:19no04 s
Negative framingGemini 3.5 Flashgemini-3.5-flash2026-09-17 14:01yes1831 s
Negative framingPerplexity Sonarsonar2026-09-17 14:08yes209 s
Negative framingGrok 4.20spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version)2026-09-17 09:52yes3413 s
Negative framingMistral Smallmistral/mistral-small via mistral2026-09-17 11:39yes55 s
Negative framingDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 13:43yes2021 s
Negative framingLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 09:51yes52 s
Negative framingQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 09:58yes534 s
Negative framingKimi K2moonshotai/kimi-k2 via novita2026-09-17 11:18yes1536 s
Negative framingGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 13:12yes3340 s
Negative framingMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 12:19yes1825 s

Normalization in this category

Every judgment call made between the raw labels and the numbers above, listed so it is visible and reversible.

ShowHide
Category-scoped readings
Braintrust read as Braintrust Gateway
Cloudflare read as Cloudflare AI Gateway
Datadog read as Datadog LLM Observability
Kong read as Kong AI Gateway
Kong AI read as Kong AI Gateway
TrueFoundry read as TrueFoundry AI Gateway
Unresolved, counted raw
Azure AI
Datadog Agent Observability
FutureAGI
LMDeploy
LangChain's LLM Power
Latency.ai
Liminal
Opik
RAGFlow
SambaNova
Zuplo AI Gateway
Discontinued, still offered
No shut-down product was recommended here.
← Document processingMFT →