After college I had the opportunity to work with Dr. Ilya Monosov to study brain function when learning risks and rewards in an uncertain environment. My thought was that studying how biological neurons encoded risks and rewards could inform the design of artificial neural networks. At the time, there was a breakthrough in using neural networks to learn how to play video games without human supervision by using "only the pixels and the game score as inputs". However, when watching videos of the neural network's performance, I noticed it failed to learn how to play a game focused on exploration. I thought one way to remedy this was to add an intermediate "reward" signal for exploring the game environment.
With this in mind, I ran experiments to measure a monkey's neuronal activity while it watched visual cues that gave information about the probability it would receive a reward in the next few seconds. I ended up characterizing two neuronal signals: one was a pre-reward ramping signal that was associated with anticipating a probablistic reward, and the other was a post-reward spiking signal that was associated with receiving a reward with a low probability of occuring. These findings suggest that ramping and spiking can be signals that encode levels of uncertainty, which expands on previous work that focused on neuronal ramping and spiking strictly in the context of more deterministic tasks.