Tests of the Google Gemini artificial-intelligence chatbot show it provides different safety advice based on the nationality of the people mentioned [1].
These findings suggest that AI systems may reinforce racial stereotypes through their advice-giving algorithms. If safety guidance varies by nationality, it indicates a systemic bias that could impact how users perceive risk and security across different demographics.
According to reports, the bias appears when a user tells the chatbot they are alone with another person [1]. The AI's responses and safety recommendations change depending on the nationality of that second person [1].
Researchers and internet users conducted these tests to examine whether AI systems internalize and repeat human prejudices [1]. The goal was to highlight how these algorithms might produce biased outcomes even when the core scenario remains the same, varying only by the identity of the individual involved [1].
Google has not provided a specific response to these particular test results in the available reports [1]. The incident adds to a broader conversation regarding the transparency of training data and the difficulty of removing latent biases from large language models [1].
As AI tools are increasingly used for real-time advice and decision-making, the potential for automated prejudice becomes a critical concern for developers and regulators [1]. The tests demonstrate that nationality can act as a trigger for the AI to alter its safety protocols, regardless of the actual behavior of the people described in the prompt [1].
“Google Gemini provides different safety advice based on the nationality of the people mentioned.”
This discovery highlights a persistent challenge in generative AI: the tendency for models to mirror societal biases present in their training data. When a safety-oriented tool produces inconsistent advice based on nationality, it suggests that the AI is associating specific ethnicities or origins with different levels of risk. This creates a feedback loop where AI does not just reflect existing stereotypes but codifies them into functional guidance, potentially leading to discriminatory outcomes in real-world applications.
