[…] machines already strike back at us even when they are stupid. How much more will they strike back when they have become smarter?
[…]
This is a problem of design: What should machines be like if their striking back is not to cause us pain? Or, better still: if it is to do us some good? What should the stone jackals be like if they are not to tear us apart and if we ourselves are not to behave like jackals? Naturally, we can design them in such a way that they lick us instead of biting us. But do we really want to be licked? These are difficult questions because nobody really knows what they want to be like. However, these issues need to be addressed before one can start to design stone jackals. (or mollusc clones or bacterial chimeras for that matter).
Or Spimes, or networked bags or …
And these issues are more interesting than stone jackals and supermen. Are designers ready to address them?
[Quotes from “The Lever Strikes Back” in The Shape of Things, Vilem Flusser, 1999]