Quote Originally Posted by Architect View Post
The belief I tend to have is that because humans discovered and use the Golden Rule to a great degree that GAI's will necessarily do so also, just because it works better. Or perhaps better said 'it's a more optimal winning strategy'. Democracy works better because it allows competing agents all succeed to the best of their circumstances and abilities, and when others succeed in a democracy that helps all. With AlphaGo zero we might see an indication of this idea, consider the following analysis by Michael Redmond and Chris Garlock

AlphaGo Zero self trained in the absence of any human data to 9d level. In Go this is a remarkable situation, it's an alien professional player, what moves would it come up with? For example the opening play is usually to the four star points but that's human convention, we don't know if there are other openings that are better. Turns out Zero's play is much like human play, to Mr Redmond's disappointment. It appears that playing the star points might actually be optimal opening play (Redmond isn't willing to say that but Garlach goes in that direction).

The point? Given a performance metric to succeed the computer necessarily learns good play much like humans. The discussion should then turn to politics or philosophy, what is the winning course of action, the destruction of human kind or to help human kind? The idea that humans are irrelevant I think is immature, my believe is GAI's will necessarily see the mutual benefit that comes with cooperation.
Doesn't it seem the Golden Rule is an incomplete strategy? It doesn't say what to do when the other entity doesn't treat you as you want to be treated.

If you are treated badly and know it and the bad treatment continues, you might get a wee bit resentful or see the relationship as...unfulfilling. So even if the AI is golden-rule centric, it's clients are apt to not be as such from time to time for whatever reason. This doesn't seem stable if the AI is more than a virtual dog.

A more stable situation, as long as we are near equals, is tit for tat, where both sides train each other to be nice.

Which could make for interesting times.

To recap, AIs may be better than us, but we still have us to deal with.