John Brockman - Possible Minds: 25 Ways of Looking at AI

It is exactly in the extension of the cybernetic idea to human beings that Wieners conceptions missed their target.

Deep learning has its own dynamics, it does its own repair and its own optimization, and it gives you the right results most of the time. But when it doesnt, you dont have a clue about what went wrong and what should be fixed.

We may face the prospect of superintelligent machinestheir actions by definition unpredictable by us and their imperfectly specified objectives conflicting with our ownwhose motivations to preserve their existence in order to achieve those objectives may be insuperable.

Any system simple enough to be understandable will not be complicated enough to behave intelligently, while any system complicated enough to behave intelligently will be too complicated to understand.

We dont need artificial conscious agents. We need intelligent tools.

We are in a much more complex situation today than Wiener foresaw, and I am worried that it is much more pernicious than even his worst imagined fears.

The advantages of artificial over natural intelligence appear permanent, while the advantages of natural over artificial intelligence, though substantial at present, appear transient.

We should analyze what could go wrong with AI to ensure that it goes right.

Continued progress in AI can precipitate a change of cosmic proportionsa runaway process that will likely kill everyone.

There is no law of complex systems that says that intelligent agents must turn into ruthless megalomaniacs.

Misconceptions about human thinking and human origins are causing corresponding misconceptions about AGI and how it might be created.

Automated intelligent systems that will make good inferences about what people want must have good generative models for human behavior.

In the real world, an AI must interact with people and reason about them. People will have to formally enter the AI problem definition somewhere.

Just because AI systems sometimes end up in local minima, dont conclude that this makes them any less like life. Humansindeed, probably all life-formsare often stuck in local minima.

Many of the central arguments in The Human Use of Human Beings seem closer to the 19th century than the 21st. Wiener seems not to have fully embraced Shannons notion of information as consisting of irreducible, meaning-free bits.

Although machine making and machine thinking might appear to be unrelated trends, they lie in each others futures.

Hybrid superintelligences such as nation-states and corporations have their own emergent goals and their actions are not always aligned to the interests of the people who created them.

Our fears about AI reflect the belief that our intelligence is what makes us special.

How can we make a good human-artificial ecosystem, something thats not a machine society but a cyberculture in which we can all live as humansa culture with a human feel to it?

Many contemporary artists are articulating various doubts about the promises of AI and reminding us not to associate the term artificial intelligence solely with positive outcomes.

Looking at what children do may give programmers useful hints about directions for computer learning.

By now, the legal, ethical, formal, and economic dimensions of algorithms are all quasi-infinite.

Probably we should be less concerned about us-versus-them and more concerned about the rights of all sentients in the face of an emerging unprecedented diversity of minds.

The work of cybernetically inclined artists concerns the emergent behaviors of life that elude AI in its current condition.

The most dramatic discontinuity will surely be when we achieve effective human immortality. Whether this will be achieved biologically or digitally isnt clear, but inevitably it will be achieved.