Propaganda Games: Sesame Credit - The True Danger of Gamification - Extra Credits

I believe much of this can be dealt with, if we start RIGHT NOW to address a serious issue in our systems - the lack of a way to represent Morals and Ethics (the When and the Why) in the systems we are building. This needs to provide important input to, and thus shape the DIKW(D) pyramid. I've been doing some work with the IEEE on this - and I'm looking here on ycombinator to get some real-world feeedback on what people are thinking and concerned about. I have some (personal) ideas that might work to address the concerns I'm seeing. {NOTE Some of this is taken from a previous post I wrote (but kinda missed the thread developing, I was late I don't think anyone read it). It is useful for this thread, so a modification of that post follows.} First, I think you need a way to define 'ethics' and 'morals', with a 'ontology' and a 'epistemology' to derive a metaphysic for the system (and for my $.02, aesthetics arises from here). Until we can have a bit of rigor surrounding this, it's a challenge to discuss ethics in the context of an AI, and AI in the context of the metaphysical stance it takes towards the world. This is vital, as we need to define what 'malicious use' IS. This is still an area (as the thread demonstrates) of serious contention. Take sesame credit (a great primer, and even if you know all about it, it is still great to watch: ▶ this video ). Now here is a question for you: Is it wrong for the government to create a system that uses social pressure, rather than retributive justice or the reactive use of force, to promote social order and a 'better way of life'? Now, I'm not arguing for this (nor against for the purposes of this missive), but using it as a way to illustrate that different cultures, governmental systems, and societies, may have vastly different perspectives on the idea of a persons relationship viz a viz the state when it comes to something like privacy. I would suggest that transperancy in these decisions is a good idea. But right now we have no way to do that. I think the current way the industry is working - seemingly hell-bent on developing better, faster, more eficient, et al ways to engineer Epistemic engines and Ontologic frameworks in isolation is the root cause of the problem of malicious use. Even the analysis of potential threats (from the article referenced 'The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation' - I just skimmed it so I can keep up with this thread, please enlighten me if I'm missing something important) only pays lip service to this idea. In the executive Summary, it says: 'Promoting a culture of responsibility. AI researchers and the organisations that employ them are in a unique position to shape the security landscape of the AI-enabled world. We highlight the importance of education, ethical statements and standards, framings, norms, and expectations.' However in the 'Areas for Further research' section, I would point out that the questions are at a higher level of abstraction than the other areas, or discuss the narrative and not the problem. This might be due to the authors not having exposure to this area of research and development (such as the IEEE) - but I will concede that the fact that the note about the narrative shows that very few are aware of the work we are doing... This isn't pie-in-the-sky stuff, it has real-world use in areas other than life or death scenarios. To quickly illustrate - let's take race or gender bias (for example the Google '3 white kids' vs. '3 black kids' issue a while back in 2016). I think this is a metaphysical problem (application of Wisdom to determine correct action) that we mistake for an epistemic issue (it came from 'bad' encoding). This is another spin on kypro's concern about the consequences of AI deployment to enable the construction of a panopticon. This is about WISDOM - making wise choices - not about coding a faster epistemic engine

Beyond that, consider what Tencent are doing with a "social credit score" to compete with Alibaba's Sesame Credit. It will be scary when these systems eventually migrate to or evolve in the US. Tencent social credit: * link Sesame Credit video from Extra Credits (almost a couple years old now, but most easily-engaged piece I've seen on implications of social credit): * ▶ this video

This hasn't appeared very high, probably due to the fact that HN hates links to vids with no context, so here it is, China is implementing a social credit system that will be mandatory in 2020, that gives you a social score based on what the state deems are "good" and "bad" actions. What you share on social media, your friends, your purchases etc. This is scarily Orwellian. ▶ this video

China has been working on this for a while, and barring people from transportation is not the scary bit. That's last-century scary. No, it's the emergence of a new sort of ideological warfare making use of the recent knowledge developed in gaming industry that is the scary bit. This is a new kind of threat, something we as humans haven't learned how to deal with. Here's some analysis from a game designer from 2015: ▶ this video

Extra Credit's episode about Sesame Credit: ▶ this video So, where else is there a widespread system of social media scores used to enforce conformity? Right here! The only difference is that there is a greater diversity factions influencing this in the west. From what I've seen, we are all being very effectively conditioned to hate "the other." link