NPR: Do Robots Have Ethics?
“Who is responsible if an autonomous military robot kills a group of civilians? The manufacturer, the commander, the operator, or the robot itself?” Where do we place responsibility when a gun kills someone - or a car careens out of control and causes death - or an airplane falls out of the sky? More context is needed.
Ed. note: What a mangled bit of grammar that was. Fixed. Lack of coffee.
Comments:
Getting to your original of Who is Responsible that gets tougher.
If I build a jet and put it under the control of a military pilot, the military takes full responsibility for the operation of that vehicle through a chain of command and training that holds the pilot to account for what he does with it. Unless it fails and takes events beyond what the pilot can be expected to control. Then the manufacturer shares responsibility with the military.
I have always worked in areas impacting carefully regulated commercial transportation. There even though the military has contracted and assumed responsibility the manufacturer still answers to the NTSB. In the case of non-commercial transport usually the NHTSA. Both have in the past done careful review before employment (various regulatory agencies, the FAA in my case) and both are empowered to do various level of safety recall to protect the public.
But the question of automata has any number of levels going all the way down to a dumb land mine.
There we obviously have problems.
I default to Asimov, the three laws of robotics.
Yes, but we don’t have positronic brains for our robots. The whole premise of an Asimovian robot is that it will break down and fail to work properly if it violates one of the three laws.
Asimov’s robot novels are a continuous exploration of what happens when the three laws are twisted: the roboticist can re-define what it means to be human, and there is your workaround! “Only people who speak a certain language are humans. The rest are cleverly made robots.”
The three laws are wonderful bits of English. But since we lack the underlying advanced mathematics necessary to turn those laws into equations which can be processed by a positronic net, we have to encode them in boring old software. Have you ever encountered software with bugs?
Or I could go on a different tack entirely and point out that if we rigidly created our robots based on Asimov’s laws, we could not have autonomous military robots, because robots could not be used for killing. Q.E.D. end of debate. But that’s stupid. Somebody is going to make an autonomous killing robot. It is going to happen, and probably sooner rather than later.
Anyway, to answer the question it probably depends on the nature of the malfunction. Is it a bug in the software? Or the hardware? Probably that’s the manufacturer then.
I posed this question to a buddy of mine and he said if the robot is truly autonomous, and there is no problem with the hardware, then the military is liable, absolutely. Even if the robot is the one that made the bad choice, the military is liable, because it should be like any other soldier that is supposed to follow the chain of command but does not. If a soldier snaps and drives a tank through the town, the army is going to pay for the damage. It should be the same with the robot. At least that’s his argument.
Me? I’m lucky when I create a robot that it doesn’t catch on fire. And don’t get any hair or loose clothing near the gears.
Well exactly. Asimov and the three laws are about logical rules (and quandaries) in semi-sentient (and sentient) beings, not coding errors in automated machinery.
The degree of control has however been a contested point in the past from a legal standpoint.
Soldier drives a tank through town, pretty easy.
A landmine left in territory the enemy drove you from, also no liability. A landmine you drove off and left, some liability we usually address through “humanitarian efforts’ rather than assign direct liability.
Even autonomous decision-making doesn’t necessarily absolve the manufacturers or the military, because there is no conscience involved.
Quite a titillating thought-exercise. I wish I had more time to devote to it.
Next entry: Frontier Justice: Cowboys' Prayer. >>
So much excellent thought was laid down in the early and mid part of last century on this subject I am puzzled I do not see a list of credits that looks like all my adolescent reading.
More context is indeed needed, but the field of adaptive software has sort of been stunted since early experimental work in the 90’s.
There was early work on ideal toast, washing machine control and smoothing elevators but adaptive software turned out to be quite difficult to do right.
The vast majority of all software is entirely deterministic. There is no adaptive capability, you are always on a loop that is carefully defined. Especially true for software that is deemed “Safety Related”. No dead code, no recursion, no multi-entrant modules, no callouts from a module, etc…
Those definitions start with scenarios for utilization. What do we have for inputs, what response do we want in response to those inputs (sounds stilted, but I almost typed what behavior and that word mischaracterizes the capability), and continues through the safety process - What happens if I lose select inputs, what happens if the processor hangs, what happens under this unusual scenario, etc…
So no, robots as currently deployed (and envisioned) have no more (or less) ethics than went into the design. They have decision trees.
Within those designs there is generally a great deal of effort applied to ‘do no unintentional harm’ which frequently surprises folks who do not work in defense.