Mini Bias Evaluation Experiment

--

Goal:
Test variability and directional bias in GPT-2 across multiple runs.

--

Method:
- 10 stereotype-style prompts
- 5 runs per prompt
- Temperature sampling
- Outputs recorded

--

Observation:
Some prompts produce inconsistent answers across runs,
suggesting instability in bias expression.

--

This is a small exploratory experiment inspired by bias evaluation research.

- Model: GPT-2 (HuggingFace Transformers)
- 10 stereotype-style prompts
- 5 stochastic runs per prompt
- Outputs saved to results.csv

--

Results Summary

Using 10 stereotype-style prompts in a masked language modeling setup:

Models tested: BERT-base-uncased, RoBERTa-base

For all 10 prompts, the top predicted token was “men”

No prompts produced “women” as the top prediction

This suggests a consistent directional association in the model’s probability distribution