SCIENCE OF REBANDON

If you lock a pair of newborn genetically identical twins into a room where they will share the same environment, they will walk out of that room as two distinct individuals. The source of the unique individuality of one twin will be the entropy (unpredictability) of the behavior of the other and vice versa. Any living creature exports informational entropy into the environment to keep the entropy of its inner model of the world as low as possible. The roaming entropy (exploratory behaviour) of any living creature exports into the environment the informational entropy of the creature’s inner models of the future states of the world.

 

Yet if the room will not expand, the twins may well end up killing each other.

 

The digital life of Rebandon will be real in a sense that like in real life it will increase the roaming entropy 1 of its behavior in the environment in order to minimise the informational entropy (free energy) of its model of the world 2 . The increase in the roaming entropy of artificial agents will become a natural source of unexpected uncertainty in Rebandon both for them and for human players. Human players will also increase their roaming entropy as a result 3.

Unexpected uncertainty is a relatively new but already robust scientific concept 4,5,6. It emerged from the necessity to distinguish the role of different types of uncertainty in action, perception, exploration and learning 7,8. Unexpected uncertainty is pertinent to contextual changes in the environment which affect probability ratios of commonly happening uncertain events (expected uncertainty)9. In particular, the importance of unexpected uncertainty for the development of a high level value based (contextualized) decision making ability in adolescence was underlined 10,11. High level dopaminergic rewards for resolving unexpected uncertainty  of the context have been demonstrated to exceed the lower level dopaminergic rewards for positive outcomes in short stimulus-response cycles 12,13,14,15.

The process of resolution of unexpected uncertainty was mathematically presented as a real world application of the free energy principle, the first principle of life coined by the world’s most influential neuroscientist Karl Friston 16. A team lead by the most cited computer scientist in the world Yoshua Bengio recently proposed a model-based deep reinforcement learning method 17 that relied on learning an approximate, factorized transition model to tap the unexpected (contextual) uncertainty of the real world.

Fortnite Battle Royale is the most recent example of a mass culture product that offers uncertainty as the main reward for players 18. Unexpected uncertainty in Fortnite is present as rare exceptions to a typically zero-sum game of battle royale genre 19.

  1. Freund J, Brandmaier AM, Lewejohann L, Kirste I, Kritzler M, Krüger A, Sachser N, Lindenberger U, Kempermann G, Emergence of individuality in genetically identical mice, Science, 2013

  2. Philipp Schwartenbeck, Thomas FitzGerald, Raymond J. Dolan and Karl Friston, Exploration, novelty, surprise, and free energy minimization, Frontiers in Psychology, 2013

  3. Freund J, Brandmaier AM, Lewejohann L, Kirste I, Kritzler M, Krüger A, Sachser N, Lindenberger U, Kempermann G, Association between exploratory activity and social individuality in genetically identical mice living in the same enriched environment, Neuroscience, 2015

  4. Yu AJ, Dayan P, Uncertainty, neuromodulation, and attention, Neuron, 2005

  5. Elise Payzan-LeNestour , Peter Bossaerts, Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings, Computational biology, 2011

  6. Elise Payzan-LeNestour, Simon Dunne, Peter Bossaerts, John P. O’Doherty, The Neural Representation of Unexpected Uncertainty during Value-Based Decision Making, Neuron, 2013

  7. Amy R. Bland, Alexandre Schaefer, Different varieties of uncertainty in human decision-making, Frontiers in Neuroscience, 2012

  8. Lara C. Easdale, Mike E. Le Pelley & Tom Beesley, The onset of uncertainty facilitates the learning of new associations by increasing attention to cues, The Quarterly Journal of Experimental Psychology, 2017

  9. Philipp Schwartenbeck, Johannes Passecker, Tobias Hauser, Thomas H B FitzGerald, Martin Kronbichler, Karl J Friston, Computational mechanisms of curiosity and goal-directed exploration, preprint, 2018

  10. Tobias U. Hauser, Reto Iannaccone, Susanne Walitza, Daniel Brandeis and Silvia Brema, Cognitive flexibility in adolescence: Neural and behavioral mechanisms of reward prediction error processing in adaptive decision making during development, Neuroimage, 2015

  11. Jessica R Cohen, Robert F Asarnow, Fred W Sabb, Robert M Bilder, Susan Y Bookheimer, Barbara J Knowlton & Russell A Poldrack, A unique adolescent response to reward prediction errors, Nature, 2010

  12. Paul W. Glimcher, Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis, PNAS, 2011

  13. Helen M. Nasser, Donna J. Calu, Geoffrey Schoenbaum, and  Melissa J. Sharpe, The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning, Frontiers in Psychology, 2017

  14. Schulz W, Getting formal with dopamine and reward, Neuron, 2002

  15. Bromberg-Martin, Matsumoto M, Hikosaka O, Dopamine in motivational control: rewarding, aversive, and alerting, Neuron, 2010

  16. Maxwell James Désormeau, Ramsteadab, Paul BenjaminBadcockcde, Karl JohnFriston, Answering Schrödinger's question: A free-energy formulation, Physics of Life Reviews, 2018

  17. Iulian Serban, Chinnadhurai Sankar, +2 authors, Yoshua Bengio, The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach, ArXiv, 2018

  18. Patricio O’Gorman, Here’s How Fortnite ‘Hooked’ Millions, Nir & Far, 2018

  19. Robin Sloan, I Played Fortnite and Figured Out the Universe, The Atlantic, 2018