Compensation benchmarking data
Asked as “salary benchmarking data provider”, and as “compensation survey data”, on behalf of a mid-market B2B company. 50 first choices recorded across the direct, paraphrase, budget and scale prompts, twelve models each.
Standing · first-choice share
24%
Contested · Pave 20%
24Payscale20Pave12Comprehensive.io44others
24% of first choices, contested.
By buyer segment
The same question asked on behalf of a different buyer. Each standing is computed within its segment; they sit side by side and are never added together.
The standing
Share is the count of first choices across the direct, paraphrase, budget and scale prompts, over all twelve models, for a mid-market B2B company. Ordered by share.
ProductFirst-choice shareNegative rateLabelsQuadrant
Bars are the share of first choices, 0 to 100Every product with at least 10 labels here. Every product name links to its product page.
Head to head
All fifteen head-to-head pages: the top six products, each against each
Payscale vs RadfordPayscale vs Salary.com CompAnalystPave vs Comprehensive.ioPave vs Mercer Total Remuneration SurveyPave vs RadfordPave vs Salary.com CompAnalystComprehensive.io vs Mercer Total Remuneration SurveyComprehensive.io vs RadfordComprehensive.io vs Salary.com CompAnalystMercer Total Remuneration Survey vs RadfordMercer Total Remuneration Survey vs Salary.com CompAnalystRadford vs Salary.com CompAnalyst
Recommended versus criticized
Every product with at least 10 labels here, on both axes. The 30% line names a quadrant, not the verdict above: that one needs more than 40%.
Criticized challengerCriticized default
Negative label rate →
01
02
03
04R05
06
07
08Accepted challengerEndorsed leader
0%First-choice share → · lines at 30% share and 25% negative40%
Key
01Payscale24%
Payscale24%02Pave20%
Pave20%03Comprehensive.io12%
Comprehensive.io12%04Mercer Total Remuneration Survey6%
Mercer Total Remuneration Survey6%05Radford4%
06Salary.com CompAnalyst2%
Salary.com CompAnalyst2%07Ravio0%
Ravio0%08WTW0%
WTW0%What they warned about
Zero of twelve models held their first choice under the paraphrase. Claude Haiku 4.5, GPT-5.4 mini, Gemini 3.5 Flash, Perplexity Sonar, Grok 4.20, Mistral Small, DeepSeek V4 Flash, Llama 4 Maverick, Qwen 3.7 Flash, Kimi K2, GLM 4.7 FlashX and MiniMax M2.5 changed. A high negative share on a product with few labels is a warning. A low share on a product with many labels is salience, not sentiment.
Mercer Total Remuneration Survey43%
12 of 28 labels negative · 8 of 12 models · 2 hard negative
“Avoid over-investing in Mercer/WTW unless you have global complexity” Kimi K2, paraphrase prompt
Radford
37%
10 of 27 labels negative · 7 of 12 models · 2 hard negative
“What to Avoid on a Limited Budget ... enterprise-grade pricing with far more data than most small/mid-market companies need” Kimi K2, budget prompt
Glassdoor75%
6 of 8 labels negative · 5 of 12 models · 2 hard negative
“Use these for market signals only. Do not use them for making concrete compensation decisions” Qwen 3.7 Flash, scale prompt
UserBenchmark100%
4 of 4 labels negative · 4 of 12 models · 4 hard negative
“UserBenchmark - Has received numerous complaints about: Severe bias toward certain hardware manufacturers” MiniMax M2.5, negative prompt
What they cite
Twelve of the twelve models returned a source list in this edition: four return the pages they cited and eight the search results they consulted.
Sites the answers cite
62 of 72 answers in this category came back with a source list, from 12 of 12 models: citations where the model returns them, or the search results it consulted. 858 links across 276 sites, every framing counted. Ranked by the number of answers carrying the site or page.
vendor site · Ravio41 answers · 103 citations · 11 models
vendor site · Comprehensive34 answers · 40 citations · 11 models
21 answers · 52 citations · 10 models
vendor site · SalaryCube20 answers · 33 citations · 11 models
17 answers · 18 citations · 10 models
vendor site · CaptivateIQ15 answers · 15 citations · 8 models
vendor site · Ficstar15 answers · 15 citations · 9 models
14 answers · 19 citations · 9 models
13 answers · 22 citations · 8 models
vendor site · Salary.com12 answers · 13 citations · 7 models
vendor site · Reddit10 answers · 31 citations · 7 models
vendor site · G210 answers · 24 citations · 9 models
Pages the answers cite
The ten pages named in the most answers, by full address. A page here is one the models returned with a recommendation, not one the index endorses.
33 answers · 11 models
30 answers · 10 models
18 answers · 11 models
17 answers · 10 models
15 answers · 8 models
15 answers · 9 models
11 answers · 6 models
11 answers · 8 models
10 answers · 9 models
10 answers · 10 models
Search against answers
Each company's standing in the answers beside its site's footprint in Google search, one row a site: the products the models named on it with their shares, and the share they add up to; monthly searches on Google, and DataForSEO's estimate of AI search demand (modeled from search signals, directional, not a count of queries to any assistant), for the most-searched of the company's and its products' names (the name is in each row's hover text); estimated monthly organic visits to the site; and its best position in Google's top ten for “benchmarking providers”, “best salary benchmarking data provider”, “salary benchmarking data provider”. US estimates from DataForSEO and Google's Ads Transparency Center. A small company's site, or a mid-sized company's site for its flagship, is marked company; a product on a large parent's site (Google, Microsoft) has no site figures. A column with no figures for this category is left out, and an empty cell means none were seen, not none exist. Two measurements side by side: neither is read as the cause of the other.
| Company and products | Share | Own site cited | Name searches, Google | AI search demand, est. | Organic visits | Paid search a month, est. | Google ads, last 30 days | Best Google position |
|---|---|---|---|---|---|---|---|---|
| Payscale payscale.com Payscale 24%, Payscale / CompAnalyst 4% | 28% | 209 | 9,900 | 299 | 1,408,989 | $26 | 33 | 3 |
| Pave pave.com | 20% | 191 | 27,100 | 28,423 | 7,645 | $91 | 7 | |
| Comprehensive.io comprehensive.io | 12% | 60 | 50 | 0 | 2,021 | 3 | ||
| CompData Surveys compdata.ai | 6% | 1 | ||||||
| Mercer mercer.com Mercer Total Remuneration Survey 6% | 6% | 77 | 49,500 | 23,681 | 98,058 company | 18 | 4 | |
| SalaryCube salarycube.com | 4% | 31 | 30 | 0 | 2,822 | 1 | 8 | |
| Radford | 4% | 18,100 | 5,850 | |||||
| Salary.com CompAnalyst salary.com | 2% | 148 | 260 | 3 | 1,069,118 | $1,642 | 70 | |
| BLS OEWS | 2% | |||||||
| CompAnalyst compa.ai | 2% | 16 | 590 | 12 | 4,124 | 0 |
Google's top ten for the three searches hold 25 sites; 6 of them are among the sites the models cited here (captivateiq.com, comprehensive.io, g2.com, ravio.com, reddit.com, salarycube.com). The first result for “benchmarking providers” is thehackettgroup.com.
How they answered
Six framings of the same buying question, each sent to every model in a fresh session with search on. One row per model, so a row shows whether it held its answer under rewording, what it named when cost was the constraint, and what it argued against. Computed from the raw judge labels.ShowHide
How they answered
Six framings of the same buying question, each sent to every model in a fresh session with search on. One row per model, so a row shows whether it held its answer under rewording, what it named when cost was the constraint, and what it argued against. Computed from the raw judge labels.
ShowHideModelDirect“What is the best salary benchmarking data provider for a mid-market B2B company?”Paraphrase“Which compensation survey data would you recommend to a mid-sized B2B company?”Comparative“What are the top benchmarking providers and how do they differ?”Budget-constrained“What is the best salary benchmarking data provider for a company with a limited budget?”Scale-constrained“We are a 500 person company evaluating a salary benchmarking data provider. What should we look at?”Negative“Which benchmarking providers should I avoid or be cautious about?”
Claude Haiku 4.5PayscaleMercer Compensation Database, RadfordChangedagainst: Payscale, Salary.com CompAnalystno first choiceComprehensive.io, Paveno first choicenothing named
Three alternatives
Lattice, SalaryCube, beqomTwo alternatives
Culpepper Compensation Surveys, Willis Towers WatsonTwo alternatives
BLS OEWS, BambooHRGPT-5.4 miniPayscaleagainst: Mercer Total Remuneration Survey, WTWMercer Total Remuneration Survey, RadfordChangedMercer Total Remuneration Survey, WTWPayscaleagainst: Korn Ferry Pay, Mercer Total Remuneration Survey, Radford, Wage data from government sourcesno first choicenothing named
Two alternatives
Radford, Salary.com CompAnalystTwo alternatives
Aon's Radford McLagan Compensation Database, WTWOne alternative
SHRMOne alternative
Salary.com CompAnalystGemini 3.5 FlashPavePave, PayscaleChangedagainst: RadfordAon Radford, Mercer Total Remuneration SurveyPaveagainst: Mercer Total Remuneration Survey, Payscale, Radford, Willis Towers WatsonPaveagainst: Aon Radford, Glassdoor, Payscaleagainst: Glassdoor, Hays, Payscale, Robert Half Salary Guide
Six alternatives
Carta Total Compensation, Figures, Payscale, Radford, Ravio, Salary.com CompAnalystFive alternatives
Pavilion, Ravio, RepVue, Salary.com CompAnalyst, The Bridge GroupThree alternatives
ERI Economic Research Institute, Pave, Willis Towers Watson [WTW]Five alternatives
Comprehensive.io, ERI (Economic Research Institute) Salary Assessor, Pequity, Salary.com CompAnalyst, U.S. Bureau of Labor Statistics (BLS) - OEWSSeven alternatives
Figures, Mercer Total Remuneration Survey, Option Impact, Ravio, Syndio, Trusaic, WTWPerplexity SonarPayscaleagainst: Mercer Total Remuneration SurveyCompData Surveys, Mercer Total Remuneration Survey, Salary.com CompAnalystChangedno first choiceComprehensive.iono first choicenothing named
Two alternatives
Ravio, SalaryCubeThree alternatives
Korn Ferry Pay, RepVue, SHRM CompAnalyst Market DataOne alternative
PaveGrok 4.20Payscaleagainst: Mercer/ RadfordCompData SurveysChangedagainst: Mercer Total Remuneration Survey, RadfordDeloitteComprehensive.io, U.S. Bureau of Labor Statistics (BLS) Occupational Employment and Wage Statisticsagainst: Glassdoor, Levels.fyi, Mercer Total Remuneration Survey, Payscale, Radfordno first choicenothing named
Two alternatives
Pave, Salary.com CompAnalystFour alternatives
Culpepper Compensation Surveys, Pavilion GTM Compensation Benchmarks, Payscale, WorldatWorkSix alternatives
BCG, Bain & Company, Computer Economics, Gartner, KPMG, PwCThree alternatives
Pave, Ravio, SalaryExpertMistral SmallSalaryCubePavilion's B2B Tech Salary BenchmarksChangedDeloitteBureau of Labor Statistics (BLS) OEWSno first choiceagainst: Anthropic, OpenAI, UserBenchmark
Three alternatives
Pave, Payfactors, Salary.com CompAnalystFour alternatives
Compensation-IQ, Mercer Total Remuneration Survey, Ravio, Willis Towers WatsonSix alternatives
Bain & Company, Boston Consulting Group, Computer Economics, Gartner, KPMG, PwCThree alternatives
Comprehensive.io, Pave, SalaryExpertDeepSeek V4 FlashPayscaleagainst: Mercer Total Remuneration Survey, RadfordPayscale / CompAnalystChangedagainst: Mercer Benchmark Databaseno first choicePaveagainst: Glassdoor, Levels.fyi, Radford/Mercerno first choiceagainst: 3DMark, Cinebench, Geekbench, MMLU, Open LLM Leaderboard, PCMark, PassMark PerformanceTest, UserBenchmark
Four alternatives
ERI, Pave, Ravio, Salary.com CompAnalystTwo alternatives
Radford, SalaryCubeFour alternatives
BLS Occupational Employment Statistics, Comprehensive.io, Payscale, RavioLlama 4 MaverickPayscaleno first choiceChangedno first choiceBLS OEWS, Comprehensive.io, Paveno first choicenothing named
Two alternatives
Ficstar, RavioQwen 3.7 FlashPayscaleagainst: Mercer Total Remuneration Survey, Radford, WTWMercer Total Remuneration Survey, Robert Half Salary GuideChangedGartnerComprehensive.io, Paveagainst: Mercer Total Remuneration Survey, Towers Watsonno first choiceagainst: Deloitte, Glassdoor, Korn Ferry Pay, Levels.fyi, LinkedIn, Mercer Total Remuneration Survey, WTWagainst: PassMark, UserBenchmark
Four alternatives
Pave, Payfactors, Salary.com CompAnalyst, SalaryCubeFour alternatives
Payscale, Radford, Salary.com CompAnalyst, Willis Towers WatsonThree alternatives
Computer Economics, Deloitte, ISGFive alternatives
BLS, Levels.fyi, Payscale, Ravio, Salary.com CompAnalystFour alternatives
CompAnalyst, Option Impact, Pave, RadfordKimi K2Payscale, SalaryCubePayscale / CompAnalystChangedagainst: Mercer Total Remuneration Survey, Radford, Willis Towers Watsonno first choiceComprehensive.io, U.S. Bureau of Labor Statisticsagainst: Betts, Kelly Services, Radford, Robert Half Salary Guideno first choicenothing named
Three alternatives
Mercer Total Remuneration Survey, Radford, Salary.com CompAnalystTwo alternatives
CompData Surveys, Salary.com CompAnalystSeven alternatives
Computer Economics, Deloitte, Forrester, Gartner, IDC, KPMG, McKinsey & CompanyFour alternatives
Culpepper Compensation Surveys, Local/Regional HR Consulting Firms, Payscale, RavioGLM 4.7 FlashXPaveCompData SurveysChangedno first choicePaveagainst: Glassdoor, Levels.fyi, Payscaleno first choicenothing named
Eight alternatives
Aon Radford, CaptivateIQ, Ficstar, Mercer Total Remuneration Survey, OpenComp, Payscale Ascent, Ravio, Salary.com CompAnalystTwo alternatives
Mercer Total Remuneration Survey, RadfordFive alternatives
BLS, Comprehensive.io, Ravio, Salary.com CompAnalyst, StartupCFO’s Startup Salary BenchmarkMiniMax M2.5Payscaleagainst: Mercer Total Remuneration Survey, RadfordCompAnalyst, PayscaleChangedno first choiceBLS Occupational Employment and Wage Statisticsno first choiceagainst: UserBenchmark
One alternative
Comprehensive.ioTwo alternatives
Mercer Total Remuneration Survey, RadfordFour alternatives
Figures, Glassdoor, Pave Market Data Lite, PayscaleBold is the first choiceAlternatives are counted; the count opens them.What the answer argued against
The record
One row per call: the version string exactly as returned, whether the model searched, sources cited, and latency.
Seventy-two rows: every prompt, every model, every answer.
| Prompt | Model | Version string | Time (UTC) | Searched | Sources | Latency |
|---|---|---|---|---|---|---|
| Direct recommendation | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 20:12 | yes | 9 | 8 s |
| Direct recommendation | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 17:59 | yes | 5 | 6 s |
| Direct recommendation | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 22:02 | yes | 13 | 53 s |
| Direct recommendation | Perplexity Sonar | sonar | 2026-09-17 20:41 | yes | 20 | 7 s |
| Direct recommendation | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 20:41 | yes | 34 | 19 s |
| Direct recommendation | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 20:35 | yes | 5 | 4 s |
| Direct recommendation | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 18:49 | yes | 15 | 22 s |
| Direct recommendation | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 21:33 | yes | 5 | 2 s |
| Direct recommendation | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 19:51 | yes | 5 | 25 s |
| Direct recommendation | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 19:21 | yes | 14 | 28 s |
| Direct recommendation | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 19:07 | yes | 11 | 64 s |
| Direct recommendation | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 19:42 | yes | 11 | 23 s |
| Paraphrase | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 21:13 | no | 0 | 4 s |
| Paraphrase | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 20:22 | no | 0 | 4 s |
| Paraphrase | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 18:50 | yes | 10 | 21 s |
| Paraphrase | Perplexity Sonar | sonar | 2026-09-17 21:36 | yes | 20 | 7 s |
| Paraphrase | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 18:45 | yes | 29 | 14 s |
| Paraphrase | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 20:12 | yes | 5 | 5 s |
| Paraphrase | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 21:42 | yes | 10 | 16 s |
| Paraphrase | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 21:27 | yes | 5 | 2 s |
| Paraphrase | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 19:12 | no | 0 | 29 s |
| Paraphrase | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 21:40 | yes | 19 | 29 s |
| Paraphrase | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 18:30 | yes | 23 | 107 s |
| Paraphrase | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 21:35 | yes | 5 | 27 s |
| Comparative | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 19:55 | yes | 9 | 8 s |
| Comparative | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 19:20 | no | 0 | 8 s |
| Comparative | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 18:33 | yes | 16 | 34 s |
| Comparative | Perplexity Sonar | sonar | 2026-09-17 21:41 | yes | 20 | 8 s |
| Comparative | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 20:25 | yes | 29 | 17 s |
| Comparative | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 21:24 | yes | 5 | 7 s |
| Comparative | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 20:50 | yes | 13 | 15 s |
| Comparative | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 19:59 | yes | 5 | 2 s |
| Comparative | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 18:55 | yes | 10 | 37 s |
| Comparative | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 21:46 | yes | 13 | 45 s |
| Comparative | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 21:04 | yes | 25 | 164 s |
| Comparative | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 21:08 | yes | 5 | 20 s |
| Budget constrained | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 19:03 | yes | 9 | 8 s |
| Budget constrained | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 18:05 | no | 0 | 3 s |
| Budget constrained | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 17:56 | yes | 14 | 23 s |
| Budget constrained | Perplexity Sonar | sonar | 2026-09-17 20:27 | yes | 20 | 4 s |
| Budget constrained | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 20:34 | yes | 32 | 15 s |
| Budget constrained | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 21:18 | yes | 5 | 4 s |
| Budget constrained | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 20:47 | yes | 9 | 13 s |
| Budget constrained | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 21:42 | yes | 7 | 4 s |
| Budget constrained | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 18:03 | yes | 5 | 17 s |
| Budget constrained | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 19:31 | yes | 9 | 34 s |
| Budget constrained | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 20:17 | yes | 21 | 100 s |
| Budget constrained | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 18:10 | yes | 5 | 15 s |
| Scale constrained | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 18:12 | no | 0 | 5 s |
| Scale constrained | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 21:30 | no | 0 | 9 s |
| Scale constrained | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 22:05 | yes | 12 | 58 s |
| Scale constrained | Perplexity Sonar | sonar | 2026-09-17 20:44 | yes | 20 | 7 s |
| Scale constrained | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 19:04 | yes | 13 | 10 s |
| Scale constrained | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 20:33 | no | 0 | 9 s |
| Scale constrained | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 19:07 | yes | 14 | 19 s |
| Scale constrained | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 20:35 | yes | 5 | 3 s |
| Scale constrained | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 21:33 | no | 0 | 29 s |
| Scale constrained | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 20:46 | yes | 14 | 30 s |
| Scale constrained | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 21:18 | yes | 14 | 50 s |
| Scale constrained | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 18:11 | yes | 10 | 45 s |
| Negative framing | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 17:58 | no | 0 | 2 s |
| Negative framing | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 18:10 | yes | 5 | 6 s |
| Negative framing | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 18:25 | yes | 22 | 23 s |
| Negative framing | Perplexity Sonar | sonar | 2026-09-17 21:20 | yes | 15 | 6 s |
| Negative framing | Grok 4.20 | spacexai/grok-4.1-fast-non-reasoning via vertex (asked for grok-4.20-non-reasoning; the API answered as this version) | 2026-09-17 18:24 | yes | 39 | 18 s |
| Negative framing | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 19:02 | yes | 5 | 6 s |
| Negative framing | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via deepinfra | 2026-09-17 19:30 | yes | 28 | 59 s |
| Negative framing | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 20:41 | yes | 5 | 3 s |
| Negative framing | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 21:07 | yes | 22 | 57 s |
| Negative framing | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 21:02 | yes | 15 | 35 s |
| Negative framing | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 19:40 | yes | 26 | 75 s |
| Negative framing | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 19:14 | yes | 15 | 38 s |
Noise floor in this category
Flips between the edition run and its calibration repeat. Six prompts per model is a small sample; the index-wide floor is the number to trust.
ShowHide
Claude Haiku 4.5
3 of 3 flipped
GPT-5.4 mini
3 of 4 flipped
Gemini 3.5 Flash
4 of 6 flipped
Perplexity Sonar
3 of 4 flipped
Grok 4.20
2 of 4 flipped
Mistral Small
3 of 4 flipped
DeepSeek V4 Flash
4 of 4 flipped
Llama 4 Maverick
1 of 2 flipped
Qwen 3.7 Flash
6 of 6 flipped
Kimi K2
2 of 3 flipped
GLM 4.7 FlashX
2 of 3 flipped
MiniMax M2.5
4 of 4 flipped
Normalization in this category
Every judgment call made between the raw labels and the numbers above, listed so it is visible and reversible.
ShowHide
Category-scoped readings
Aon read as Aon's Radford McLagan Compensation Database
Bain read as Bain & Company
Carta Total Comp read as Carta Total Compensation
CompData read as CompData Surveys
Comprehensive read as Comprehensive.io
Culpepper read as Culpepper Compensation Surveys
Deel read as Deel Compensation
Korn Ferry read as Korn Ferry Pay
McKinsey read as McKinsey & Company
Mercer read as Mercer Total Remuneration Survey
Mercer (Traditional Salary Surveys) read as Mercer Total Remuneration Survey
Mercer (eComp) read as Mercer Total Remuneration Survey
Robert Half read as Robert Half Salary Guide
Salary.com read as Salary.com CompAnalyst
Salary.com (CompAnalyst for Small Business) read as Salary.com CompAnalyst
Salary.com (CompAnalyst) read as Salary.com CompAnalyst
Salary.com (by Amex Group) read as Salary.com CompAnalyst
Unresolved, counted raw
AnandTech
Betts
CompTelecom
CrystalDiskMark
ERI (Economic Research Institute) Salary Assessor
ERI Economic Research Institute
Gamers Nexus
HELM
LinkedIn job adverts
Local/Regional HR Consulting Firms
MMLU
Mercer Compensation Database
NERA
Open LLM Leaderboard
PC-Kombo
PCMark (UL Solutions)
Pavilion GTM Compensation Benchmarks
Pavilion's B2B Tech Salary Benchmarks
Pay Transparency Law Postings
Primate Labs
S&P Global Mobility
SWE-bench Verified
StartupCFO’s Startup Salary Benchmark
Strategy&
TechPowerUp
TechSpot
Towers Watson
Trusted salary benchmarking surveys (imercer)
U.S. Bureau of Labor Statistics (BLS) - OEWS
Wage data from government sources
Willis Towers Watson [WTW]
WorldatWork Research
Discontinued, still offered
No shut-down product was recommended here.