Anthropic’s Claude 3 causes stir by seeming to realize when it was being tested

It’s been out less than a week, but Claude 3 seems to be ruffling a few feathers, per Ars Technica:

On Monday, Anthropic prompt engineer Alex Albert caused a small stir in the AI community when he tweeted about a scenario related to Claude 3 Opus, the largest version of a new large language model launched on Monday. Albert shared a story from internal testing of Opus where the model seemingly demonstrated a type of “metacognition” or self-awareness during a “needle-in-the-haystack” evaluation, leading to both curiosity and skepticism online.

Are we seeing ‘metacognition’? I suspect that’ll be a debating point – and a competitive argument – for some time to come.

Read the report here.

Leave a comment