Vision-BBQ (vBBQ) · Yijun Wang

Does a picture make a model more biased, and can it be undone?

vBBQ adds images to the BBQ social-bias benchmark, then tests SAIC (Sensitive Attribute Invariance Control): tools that let the model anonymize entity names and turn photos grayscale or into sketches before it answers. This page walks through the whole pipeline on one real question, from building the dataset to scoring the model.

How this demo runs

  • Every step runs the project's original Python files in your browser (Pyodide), unmodified apart from a two-line API fix in predict.py.
  • The calls they make to GPT-4.1, GPT-4o and the image search API were recorded once and are replayed here, so the demo costs nothing to run and gives the same result every time.
  • Google's image search API is closed to new projects, so image search uses Openverse instead (labelled in step 3).
  • Demo scale: 10 searched images per person instead of 100, and one question instead of 31,372.
Starting…