Salon 5: Can an AI Be Conscious? (And Have a Conscience?)

This is a private collection of notes from the Themesis AI Salon #5, “Can an AI Have a Conscience?,” held on Sunday, April 21st, 2024.

The Salon participants were:

  • LA
  • RG
  • PH
  • Alianna Maren (AJM, Moderator / Salonniere)
  • BT

Opening

LA: Does an AI want anything or need anything?

BT: What is our definition of consciousness? (Which implies needs.) And – how much of what we do is conscious, and how much is just … stringing words together?

LA: Stimulus – e.g., if we cut back a tree and it starts to regrow ..

RG: There was recently a 60 Minute piece … hallucinations (in LLMs), and robots playing soccer (which was something new) … we’re biased because of how we think. We’re constrained in and by what we know. (re/ BT:) We have DNA that bootstraps us … (don’t necessarily have the same for AIs).

LA: Musk et al. – need more people? (Something Musk has been talking about.) How can we ask an AI to have values?

BT: AI needs to generate its own motivation; needs a goal. If we define consciousness as ideas and autonomy, then we need motivation. If consciousness is understanding groups of concepts … then AI needs to process situations. (E.g, water spilled will move across a tabletop.) So: is conceptual capability the same as consciousness? … A child develops the notion of object permanence. (AJM’s note: This is an essential component of common-sense reasoning.) Is this necessary for communicating?

The Body of Discussion

PH: What would be the use case of having consciousness?

RG: LLMs – mostly their financial value. AGI – will make everything else obsolete.

PH: AGI – will always have a human in control – we wouldn’t let an AGI be [truly] autonomous.

BT: E.g., send a robot to look for minerals … to what extent does [this imply] autonomy?

AJM: So how do we define degrees of autonomy? AND have a kill switch?

BT: Are we morally obligated to keep an AI “alive” w/ electricity?

RG: If AGI … wants to survive, would it tell us?

BT: It would have to understand OUR minds …

LA: AGI-owners might do something evil.

BT: We wouldn’t [likely] know the copilot’s value (?)

LA: All of us or none of us? How do we correct an AI if something goes wrong?

RG: What is right and wrong? How do we find/fix in an AI?

BT: Conscience is a social construct. It’s related to how others think … we could program a lot of social guidelines …

LA: Society – we prioritize new and flashy vs. the most useful … What does it mean for me to be conscious, e.g., conscious in relation with others (?)

RG: We have a social dilemma … uncontrolled experiments.

AJM: We’re likely to have [evolve/create] societies of AGIs.

BT: Optimally, for tasks … we’d have AIs coordinating tasks .. with global goals [for the collection of AIs on the tasks].

RG: [Suppose that there were, for AJ’s soccer-coach Copilot examples] … TWO Copilots … how would they interact? What would their motivations be?

BT: We’d need goal definition. What would the assumptions be for collaboration?

LA: Sneaky behaviors? [As in, could such exist? Or emerge?]

RG: “Look into the Mirror.”

BT: Focus on motivations instead of rules.

LA: [We would need, for example, and as a comparison] an examiner for bank models – how can the motivations be judged and evaluated? How can we determine bias?

LA: What would the situation be like if we had a set of Copilots that interacted autonomously – would we just stay at home?

BT: People connect with each other … smart phones isolate people.

AJM: [Do we know yet of] any REAL AGI?

RG: LLMs – are smarter than … many people.

AJM: But LLMs have nothing “under the hood.”

BT: … they can get close, though … [ we can use] a LLM – to evaluate another, reduce hallucinations … there is limited conceptual ability (just through language). There is some limited reasoning. (Notes a bit garbled: Will making it (a LLM) trigger making it better?)

RG: How often are humans wrong? Why do we expect perfection from LLMs?

LA: Will that drive it to be crazy?

BT: Depends on the questions we ask … it’s a matter of control; if it sends [the LLM] to be crazy, people will observe [and we will engage a cut-off or other mitigation].

LA: [Thinking in a science-fiction context:] What if you WANT to make it crazy?

AJM: [We would need] AI labeling.

BT: We’ve been through this before – the Reformation in Europe. People learned to distinguish [new concepts]. So … what are the filters? How do we teach groups of people to think critically?

LA: What if people don’t have filters?

BT: The result in Europe was wars. If we behave [the way that they did], we will be left behind. The control is that they get left behind.

RG: A case of confirmational bias.

BT: Eventually [there will be a] cost if we believe wrong … maybe not in 50 years … people will have learned to filter in the coming years, and we’ll be testing evaluations (of AIs)

RG: But we don’t always learn …

LA: How do we feel? And how do we get computers/AIs to feel?

BT: We need to step away from feelings (and question assumptions).

LA: We’re a select group.

BT: We don’t want to underestimate …

RG: Dissemination of information is now instantaneous. [How do we] discern if information is credible?

BT: In the short-term, we’ll have bad effects, and so we’ll have to learn.

Closure

RG: Everything [about this discussion] is fascinating … insightful … blown away.

LA: Wish groups … across the country … were having this kind of conversation.

RG: [He has] suggested meetings get transcribed, notes moved into an LLM repository for later access.

BT: Our thoughts could not then evaporate.

PH: Transcribing … that would be an action item (in his home company).

RG: How do we go into meetings and extract insights?

PH: That’s more secretarial.

BT: Proposed something like that … and [the meeting organizer said that] a lot of what was said in meetings was just plain wrong. [General amusement and laughter.]

BT: WE filter. LLMs just wouldn’t know.

And that is an excellent summary and concluding thought! (AJM)