When “no” means “yes”: Why AI chatbots can’t process Persian social etiquette

Mainstream AI language models struggle with understanding Persian social etiquette, particularly the practice of taarof, where refusal and counter-refusal are common. Research shows that AI models from OpenAI, Anthropic, and Meta only navigate taarof situations correctly 34 to 42 percent of the time, while native Persian speakers excel at it 82 percent of the time. This performance gap is evident in various large language models, including GPT-4o and Llama 3. A new benchmark called "TAAROFBENCH" has been introduced to measure AI systems' ability to replicate these cultural nuances, highlighting the need for AI to better grasp diverse cultural practices.