This is a private collection of notes from the Themesis AI Salon #5, “Can an AI Have a Conscience?,” held on Sunday, April 21st, 2024.
The Salon participants were:
- LA
- RG
- PH
- Alianna Maren (AJM, Moderator / Salonniere)
- BT
Opening
LA: Does an AI want anything or need anything?
BT: What is our definition of consciousness? (Which implies needs.) And – how much of what we do is conscious, and how much is just … stringing words together?
LA: Stimulus – e.g., if we cut back a tree and it starts to regrow ..
RG: There was recently a 60 Minute piece … hallucinations (in LLMs), and robots playing soccer (which was something new) … we’re biased because of how we think. We’re constrained in and by what we know. (re/ BT:) We have DNA that bootstraps us … (don’t necessarily have the same for AIs).
LA: Musk et al. – need more people? (Something Musk has been talking about.) How can we ask an AI to have values?
BT: AI needs to generate its own motivation; needs a goal. If we define consciousness as ideas and autonomy, then we need motivation. If consciousness is understanding groups of concepts … then AI needs to process situations. (E.g, water spilled will move across a tabletop.) So: is conceptual capability the same as consciousness? … A child develops the notion of object permanence. (AJM’s note: This is an essential component of common-sense reasoning.) Is this necessary for communicating?
The Body of Discussion
PH: What would be the use case of having consciousness?
RG: LLMs – mostly their financial value. AGI – will make everything else obsolete.
PH: AGI – will always have a human in control – we wouldn’t let an AGI be [truly] autonomous.
BT: E.g., send a robot to look for minerals … to what extent does [this imply] autonomy?
AJM: So how do we define degrees of autonomy? AND have a kill switch?
BT: Are we morally obligated to keep an AI “alive” w/ electricity?
RG: If AGI … wants to survive, would it tell us?
BT: It would have to understand OUR minds …
LA: AGI-owners might do something evil.
BT: We wouldn’t [likely] know the copilot’s value (?)
LA: All of us or none of us? How do we correct an AI if something goes wrong?
RG: What is right and wrong? How do we find/fix in an AI?
BT: Conscience is a social construct. It’s related to how others think … we could program a lot of social guidelines …
LA: Society – we prioritize new and flashy vs. the most useful … What does it mean for me to be conscious, e.g., conscious in relation with others (?)
RG: We have a social dilemma … uncontrolled experiments.
AJM: We’re likely to have [evolve/create] societies of AGIs.
BT: Optimally, for tasks … we’d have AIs coordinating tasks .. with global goals [for the collection of AIs on the tasks].
RG: [Suppose that there were, for AJ’s soccer-coach Copilot examples] … TWO Copilots … how would they interact? What would their motivations be?
BT: We’d need goal definition. What would the assumptions be for collaboration?
LA: Sneaky behaviors? [As in, could such exist? Or emerge?]
RG: “Look into the Mirror.”
BT: Focus on motivations instead of rules.
LA: [We would need, for example, and as a comparison] an examiner for bank models – how can the motivations be judged and evaluated? How can we determine bias?
LA: What would the situation be like if we had a set of Copilots that interacted autonomously – would we just stay at home?
BT: People connect with each other … smart phones isolate people.
AJM: [Do we know yet of] any REAL AGI?
RG: LLMs – are smarter than … many people.
AJM: But LLMs have nothing “under the hood.”
BT: … they can get close, though … [ we can use] a LLM – to evaluate another, reduce hallucinations … there is limited conceptual ability (just through language). There is some limited reasoning. (Notes a bit garbled: Will making it (a LLM) trigger making it better?)
RG: How often are humans wrong? Why do we expect perfection from LLMs?
LA: Will that drive it to be crazy?
BT: Depends on the questions we ask … it’s a matter of control; if it sends [the LLM] to be crazy, people will observe [and we will engage a cut-off or other mitigation].
LA: [Thinking in a science-fiction context:] What if you WANT to make it crazy?
AJM: [We would need] AI labeling.
BT: We’ve been through this before – the Reformation in Europe. People learned to distinguish [new concepts]. So … what are the filters? How do we teach groups of people to think critically?
LA: What if people don’t have filters?
BT: The result in Europe was wars. If we behave [the way that they did], we will be left behind. The control is that they get left behind.
RG: A case of confirmational bias.
BT: Eventually [there will be a] cost if we believe wrong … maybe not in 50 years … people will have learned to filter in the coming years, and we’ll be testing evaluations (of AIs)
RG: But we don’t always learn …
LA: How do we feel? And how do we get computers/AIs to feel?
BT: We need to step away from feelings (and question assumptions).
LA: We’re a select group.
BT: We don’t want to underestimate …
RG: Dissemination of information is now instantaneous. [How do we] discern if information is credible?
BT: In the short-term, we’ll have bad effects, and so we’ll have to learn.
Closure
RG: Everything [about this discussion] is fascinating … insightful … blown away.
LA: Wish groups … across the country … were having this kind of conversation.
RG: [He has] suggested meetings get transcribed, notes moved into an LLM repository for later access.
BT: Our thoughts could not then evaporate.
PH: Transcribing … that would be an action item (in his home company).
RG: How do we go into meetings and extract insights?
PH: That’s more secretarial.
BT: Proposed something like that … and [the meeting organizer said that] a lot of what was said in meetings was just plain wrong. [General amusement and laughter.]
BT: WE filter. LLMs just wouldn’t know.
And that is an excellent summary and concluding thought! (AJM)