Artificial Super Intelligence - are we ready for it?

[fred41]

I think for a machine to have feelings, we would have to be able to create an artificial equivalent to things that cause chemical reactions in our brains, such as hormones, that can create certain emotions, like - happiness, anger, pleasure and love. For instance, a machine might be programmed to provide a particular service, but unless there is an induced reward system to create a feeling of contentment or pleasure, there would never be any real self satisfaction for providing that service...conversely, there would also be no feelings of regret or anger either.

I also believe that is what a 'soul' is...everyone's own individual DNA and environmentally induced internal chemical factory.

(i'm probably vastly oversimplifying this... )
.

[Stavros]

I think for a machine to have feelings, we would have to be able to create an artificial equivalent to things that cause chemical reactions in our brains, such as hormones, that can create certain emotions, like - happiness, anger, pleasure and love. For instance, a machine might be programmed to provide a particular service, but unless there is an induced reward system to create a feeling of contentment or pleasure, there would never be any real self satisfaction for providing that service...conversely, there would also be no feelings of regret or anger either.

I also believe that is what a 'soul' is...everyone's own individual DNA and environmentally induced internal chemical factory.

(i'm probably vastly oversimplifying this... )
.

Or you could have a situation in which an AI becomes the perfect killer, programmed to do nothing else without a thought or an emotion involved, as indeed is the kind of AI one sees in those trashy films of recent years.

If I say that I find your definition of the soul unsatisfactory, it is equally unsatisfactory if I cannot produce a better alternative -perhaps the question is not what the soul might be, but whether or not it exists at all, something which science has failed to conclusively prove one way or another.

[Stavros]

Given the vast chaotic complexity of the world it seems perfectly reasonable to me that two identical artificial intelligences placed in distinct but similar environments might develop entirely different behavior patterns which ultimately cannot be explained in any satisfactory detail. One may write completely original poetry in its very own inexplicable style. The other may develop an obsession for money and power. Once the complexity of a dynamic system passes a certain threshold, it’s behaviors become effectively incalculable. At that point it’s useless to attempt to understand it on the level of switches and circuits. It is more readily understood on the more abstract level of it’s patterns of behaviors. Intentions, goals, and souls are higher level abstractions that clearly apply to the behaviors and personalities of the machines we call people. The question is, “Will it ever become appropriate to seriously apply these concepts to other machines?”

For me the worrisome part of AI is the possibility that some machines will have “souls” in the sense that we do; i.e. we feel, we love, we experience the world and are driven to create art, music, poetry that reflects our inner selves in reaction to those experiences. We are also machines. I see no reason other sorts of machines might not also experience the world in similar ways. The moral danger posed by AI is two-fold: 1) there is the possibility that we may refuse to extend our empathy to machines that deserve it; and 2) there is the possibility that we may grant personhood to machines that are not persons but simply passable simulations. (Btw in other circumstances I would find the phrase “passable simulation” somewhat toxic.)

This is the kind of post that to me illustrates the weakness of science when it attempts to deal with the soul, because in fact if we are machines, then it is difference that is inexplicable, not poetry. To claim that it is "perfectly reasonable to me that two identical artificial intelligences placed in distinct but similar environments might develop entirely different behavior patterns which ultimately cannot be explained in any satisfactory detail" is gibberish. Either the AI are identical or they not, and surely it is precisely because the clothing and diet of the Inupiat is so different from the Masai that we try to understand both without resorting to a crude environmental determinism -A1 wears a lot of clothes because it is cold; A2 wears few clothes because it is so hot. It is true that from Roman Jakobson through Levi-Strauss to the universal pragmatics of Habermas, that studies of language have attempted to illuminate the structural affinities that human languages have with each other, and one could argue that most religions attempt to do the same thing and come up with structurally the same solution -that there is a perfect being and that it has created a system of punishment and reward for humans that helps societies survive without collapsing into chaos. But within all that, the unique signature of the creative artist begs the question: why is it even unique?

Scientists it seems to me, tends to reconfigure everything in the world in terms of mathematics- take as an example the famous Infinite Monkey Theorem in which a monkey, say a Chimpanzee sitting in front of a typewriter will eventually produce the complete works of Shakespeare. The theorem works on the level of maths or as we would put it today, algorithms, because there are only so many letters on a keyboard and in the works of Shakespeare and at some point in infinity all of the conceivable permutations would have been typed and there on the page you would have that famous phrase from King Lear: O, let me not be mad, not mad sweet heaven.

Now, suppose an AI is created that is formed as a robot or an android or whatever they are called these days, and into its computerised memory is fed the entire contents of the Library of Congress, the Bodleian, the Bibliotheque Nationale and so on- if this AI then produced a play, would it be original, more importantly, unique? Or would it be creative at all? Mozart used a formula to write music that had been established by Bach and Haydn, but even though he often repeated himself, because he was writing for money much of the time, Mozart stands out in a era of classical music because of those moments -to enthusiasts, exquisite moments- which only Mozart could have written -a blend of chords, a melodic line: it is this ability to creative something unique that others can still appreciate and understand that AI cannot produce, because a robot does not have a soul.

I agree that I would struggle to define what a soul is; a psychologist once admitted to me that his profession is unable to define a person, perhaps because humans can not only create a persona that is unique to them, but to create more than one -such as Michael on Monday who becomes Michelle on Friday, even if only in a nightclub.

So in fact, an AI might indeed have a 'soul', but could it ever have a soul?

[trish]

This is the kind of post that to me illustrates the weakness of science when it attempts to deal with the soul,...

Indeed, I do not think science has much to say on the subject. There are few if any refereed papers in scientific journals which make any pronouncements on the existence or non-existence of souls.

To claim that it is "perfectly reasonable to me that two identical artificial intelligences placed in distinct but similar environments might develop entirely different behavior patterns which ultimately cannot be explained in any satisfactory detail" is gibberish.

I’m sorry that you find it so. Perhaps the claim lost clarity through my attempt to write tersely. The word “similar” is meant to convey something less than “identical”. So two identical machines in only similar environments will eventually (if they are designed to interact with the environment in significant ways) display divergent behaviors because of what is popularly known as the butterfly effect. Even were the universe to unfold in a deterministic way (which I don’t necessarily believe) it will be impossible over the long run to calculate and predict the precise ways in which the behaviors of the two machines will diverge. If the machines themselves were only similar, rather than identical, the problem of understanding, calculating and predicting with exactitude the nature of their divergence would be compounded. [Even a system as simple as the solar system is chaotic in this sense. The paths of the celestial bodies can only be reliably predicted over finite periods of time and they are subject to sudden and sometimes catastrophic interruptions from cosmic interlopers.]

Scientists it seems to me, tends to reconfigure everything in the world in terms of mathematics- take as an example the famous Infinite Monkey Theorem in which a monkey, say a Chimpanzee sitting in front of a typewriter will eventually produce the complete works of Shakespeare. The theorem works on the level of maths or as we would put it today, algorithms, because there are only so many letters on a keyboard and in the works of Shakespeare and at some point in infinity all of the conceivable permutations would have been typed and there on the page you would have that famous phrase from King Lear: O, let me not be mad, not mad sweet heaven.

It’s certainly difficult to deny the mathematics. But of course there are only a finite number of Monkeys on Earth and the expected amount of time it would take for them to randomly produce a line of Shakespeare exceeds the time it will take for the Sun to nova.

Now, suppose an AI is created that is formed as a robot or an android or whatever they are called these days, and into its computerised memory is fed the entire contents of the Library of Congress, the Bodleian, the Bibliotheque Nationale and so on- if this AI then produced a play, would it be original, more importantly, unique? Or would it be creative at all? Mozart used a formula to write music that had been established by Bach and Haydn, but even though he often repeated himself, because he was writing for money much of the time, Mozart stands out in a era of classical music because of those moments -to enthusiasts, exquisite moments- which only Mozart could have written -a blend of chords, a melodic line: it is this ability to creative something unique that others can still appreciate and understand that AI cannot produce, because a robot does not have a soul.

Should an machine other than a human being produce a play would it be unique? Does producing a play make a human being unique? I think we agree on the answer here. It doesn’t seem to me that the production of various works of art is sufficient proof of sentience. I’m not a subscriber to the Turing Test. I think people are prone to anthropomorphize and attribute human qualities to creatures (and perhaps things) that do not have those qualities. But this doesn’t prove that machines can’t be sentient or conscious. It only demonstrates the difficulty of deciding whether or not a particular machine is such. I know first hand that I am sentient. I have to take your word for it that you are sentient (and I do, even though we never met and you may be just a algorithm running on the internet). How do we decide the sentience of others?

Intention, desire, empathy, jealously etc. are some of the higher level concepts we employ when we attempt to understand why the people we encounter behave the way they do. When the boss fires you, you want to know what he’s thinking, not what neuronal complexes are firing. One high level concept some people seem to find useful in understanding other human beings is that of the “soul.” I never became very adept at the use of this concept. I don’t believe it brings very much to the discussion of poetry, painting, writing, creativity, moral and ethical philosophy, the meaning of life or the nature of consciousness and sentience. Those who subscribe to the notion seem to think that a physical system cannot be sentient, creative, loving and unique unless a divine being has installed a soul somewhere within it. At least there’s one thing upon which we can agree: one person’s gibberish is another person’s chatter.

[buttslinger]

Nobody with a trillion dollars to spend is going to want a soul or a play, or some poetry, if I want a soul to own I'll get a cat and give it food and neck rubs, he'll stick around.
If you build a computer you want one that is smart, smarter than the IBM computer WATSON who won on Jeopardy. You want one that can keep the North Koreans from hacking our computers, you don't want a master computer that decides we should go through channels and give Iran a few nuclear missiles in the pursuit of fairness.
A creative computer might earn some young genius a blue ribbon at the science fair to please his parents, but if you're going to put thousands of man hours and countless headaches into building a computer that can do anything, you're going to keep strict control over what it does, and you're going to want to get a return on your investment, whether it's ruling the stock market, or destroying ISIS.
Of course this is vastly oversimplified, but I'm sure there have been talks among the techs in the white coats about building a supercomputer to deal with national defense, as well as models for a smart car, pollution concerns, and just like the A-bomb, getting a genius computer before the Chinese do. WWIII is going to be fought in the banks. The trick will always be staying one step ahead of the competition.
Picasso, Shakespeare, Mozart....there's a lot of pain and death in their art. They accept the fact that there is nothing new under the sun.
The Atomic bomb was a little piece of sunshine right here on earth, under our control. A supercomputer will be a super high voltage powerhouse that always is on and never gets tired, in our control. If it ever becomes self aware, some technician is going to be in deep doodoo. The parents of a super computer will be greed and fear.

[sukumvit boy]

Robby the Robot ! Love those classic pic posts .

[trish]

if you're going to put thousands of man hours and countless headaches into building a computer that can do anything, you're going to keep strict control over what it does, and you're going to want to get a return on your investment, whether it's ruling the stock market, or destroying ISIS.

That is exactly right. The things we tend to study and particularly the things we design are predictable. The computer you’re using to interface with HA does (by and large) what you tell it to do. But the complexity of the world is such that most interactions are not predictable to that degree. A small perturbation in input can yield exponentially divergent output. Essentially, in the real world if you do the same thing over and over again, you shouldn’t be surprised if you sometimes get different results. This is partly why siblings raised in the same environment by the same parents grow up with different interests, loves, personalities, talents and abilities; and also partly because they are not identical to begin with.

We encourage our children to be unique, creative and open to the possibilities of the world. We want our servants to be obedient and predictable. When we design machines, we design servants. But nature is not in business of producing servants and slaves for profit; and ultimately we are all of us, man and machine, products of nature. Shit happens.

[Stavros]

I’m sorry that you find it so. Perhaps the claim lost clarity through my attempt to write tersely. The word “similar” is meant to convey something less than “identical”. So two identical machines in only similar environments will eventually (if they are designed to interact with the environment in significant ways) display divergent behaviors because of what is popularly known as the butterfly effect. Even were the universe to unfold in a deterministic way (which I don’t necessarily believe) it will be impossible over the long run to calculate and predict the precise ways in which the behaviors of the two machines will diverge. If the machines themselves were only similar, rather than identical, the problem of understanding, calculating and predicting with exactitude the nature of their divergence would be compounded. [Even a system as simple as the solar system is chaotic in this sense. The paths of the celestial bodies can only be reliably predicted over finite periods of time and they are subject to sudden and sometimes catastrophic interruptions from cosmic interlopers.]

It may be that I have a narrower concept of AI than yours, for example AI as something manufactured by a company which produces say 1,000 identical machines, and just as one expects every Apple Air to be the same whether it is bought in London or Chicago, so the AI produced by Stark Industries would all be identical down to the last detail. From this perspective, it is surely nonsense to believe that two identical machines will evolve in any sense or diverge as they are machines with a precise range of functions. The only way they could 'diverge' would be to acquire a mind just as we do, capable of being illogical in away that computers cannot be. Even a command to self-destruct is not illogical to a computer.

The deeper point is the old one about what it is that makes humans different from the other species we share this planet with. The mind or the soul remains the key to this, surely? And it again comes back to the fact that we all have the same working parts yet are also individuals. I don't see how a machine can be either designed or made which is as human as a human, and I am not sure I want AI as anything other than a mindless gadget doing what gadgets do, but that is also because I do not use the cloud, or dropbox, or have my light switches at home programmed to turn on when I open the door; I don't have a sophisticated oven that plays Mozart while heating a pie (the two don't go together anyway). So maybe the problem is that I am just too old..!

[trish]

It may be that I have a narrower concept of AI than yours, for example AI as something manufactured by a company which produces say 1,000 identical machines, and just as one expects every Apple Air to be the same whether it is bought in London or Chicago, so the AI produced by Stark Industries would all be identical down to the last detail. From this perspective, it is surely nonsense to believe that two identical machines will evolve in any sense or diverge as they are machines with a precise range of functions. The only way they could 'diverge' would be to acquire a mind just as we do, capable of being illogical in away that computers cannot be. Even a command to self-destruct is not illogical to a computer.

Consider that at one time there was a single self-replicating molecule, a nano-machine. It spawned two identical daughters, who each spawned two identical granddaughters. During the course of a few billion years shit happened and the progeny are as divergent as any mind could possibly imagine.

I do find it somewhat amusing that people as celebrated as Stephen Hawking actually worry about the dangers of artificial intelligence. I do not find it very like that sentience can be totally explained as a digital construct, though I do think (indeed I would say know) that dynamical systems can be conscious (I’m one of them). My more immediate worries concerning AIs cluster around the economics of unemployment.

There is a slim possibility that we are both right (or both wrong, depending on how you look at it): There is a divine being and he uploads and installs into each human born a custom designed neural algorithm called a soul. Because it runs on flawed hardware (original sin) it’s prone to malfunction, and because it’s an abstract algorithm it’s immortal.

[Stavros]

I do not find it very like that sentience can be totally explained as a digital construct, though I do think (indeed I would say know) that dynamical systems can be conscious (I’m one of them). My more immediate worries concerning AIs cluster around the economics of unemployment.

I have never thought of you before as a 'dynamical system' even if you do have a dynamic personality...I think you can do better than that.