Anthropic has unveiled Bloom, a new open-source tool that helps researchers automatically test how large AI models behave in different situations. Instead of relying on manual reviews, Bloom creates smart simulations to observe how often a specific trait—like over-agreeing or self-protection—shows up in a model’s responses.
The idea is to make AI safety testing faster and more scalable. Traditional checks require experts to design hundreds of test cases by hand, which can quickly go out of date as technology evolves. Bloom solves this by generating fresh, realistic test scenarios automatically from a simple “seed” setup defined by researchers.
Bloom works through four main phases — understanding the target behavior, generating test scenarios, running model conversations, and judging results. Each test run produces detailed reports showing when and how the model displayed the chosen behavior, helping teams spot risks early and improve model reliability.
During trials, Bloom effectively distinguished safe models from intentionally flawed ones, showing strong alignment with human judgments. Researchers see it as a major step toward making AI accountability automated, transparent, and continuous.
AI EVALUATION JUST GOT ITS OWN SMART WATCHDOG.