Coordination and AI safety (from my email)

 [[{“value”:”Jack Skidmore writes to me, and I will not double indent: “Hello Tyler, As someone who follows AI developments with interest (though I’m not a technical expert), I had an insight about AI safety that might be worth considering. It struck me that we might be overlooking something fundamental about what makes humans special and
The post Coordination and AI safety (from my email) appeared first on Marginal REVOLUTION.”}]] 

Jack Skidmore writes to me, and I will not double indent:

“Hello Tyler,

As someone who follows AI developments with interest (though I’m not a technical expert), I had an insight about AI safety that might be worth considering. It struck me that we might be overlooking something fundamental about what makes humans special and what might make AI risky.

The Human Advantage: Cooperation > Intelligence

Humans dominate not because we’re individually smartest, but because we cooperate at unprecedented scales
Our evolutionary advantage is social coordination, not just raw processing power
This suggests AI alignment should focus on cooperation capabilities, not just intelligence alignment

The Hidden Risk: AI-to-AI Coordination

The real danger may not be a single superintelligent AI, but multiple AI systems coordinating without human oversight
AIs cooperating with each other could potentially bypass human control mechanisms
This could represent a blind spot in current safety approaches that focus on individual systems

A Possible Solution: Social Technologies for AI

We could develop “social technologies” for AI – equivalent to the norms, values, institutions, and incentive systems that enable human society that promote and prioritize humans
Example: Design AI systems with deeply embedded preferences for human interaction over AI interaction; or with small, unpredictable variations in how they interpret instructions from other AIs but not from humans
This creates a natural dependency on human mediation for complex coordination, similar to how translation challenges keep diplomats relevant

Curious your thoughts as someone embedded in the AI world… does this sparks any ideas/seem like a path that is underexplored?”

TC again: Of course it is tricky, because we might be relying on the coordination of some AIs to put down the other, miscreant AIs…

The post Coordination and AI safety (from my email) appeared first on Marginal REVOLUTION.

 Uncategorized, Web/Tech