Back to browse
GitHub Repository

Context-driven valuation bias and halo effects across six multimodal LLMs (companion study to Lee, 2026)

1 starsPython

I put a $2.43 necklace on 3 outfits. VLMs priced it at $19 to $104

by BrianneLee011·Jul 28, 2026·17 points·23 comments

AI Analysis

●●●BangerDark HorseBig Brain

Claude prices same $2.43 necklace at $62 or $19 depending entirely on the outfit context.

Strengths
  • 1,500 stateless API sessions across six frontier models with rigorous counterbalancing.
  • Text-only descriptions reproduce halo effect, eliminating photographic quality as confound.
  • Sequential arm tests whether models recognize identical objects when shown side-by-side.
Weaknesses
  • Study design is thorough but findings confirm known LLM susceptibility to contextual framing.
  • No proposed mitigation technique beyond documenting the bias exists in current implementation.
Category
Target Audience

AI researchers studying model bias and vision-language hallucination

Similar To

HELM · BigBench

Similar Projects