A few days ago someone posted this and I forgot whom, but now I read it and it's good
https://dustycloud.org/blog/sussman-on-ai/
"AI should be "accountable", in the sense that it should be able to express its symbolic reasoning, and be held up to whether or not its assumptions held up to that."
i.e. an AI should be able to tell us why it made some decision, because
"if you can't inspect a system, you're held prisoner by it"