In college I wanted to study brain function, but never got the chance. After college I found an opportunity to work with Dr. Ilya Monosov to study brain function in the context of learning risks and rewards in an uncertain environment. I thought that studying how biological neurons encoded risks and rewards could inform how to design artificial neural networks for reinforcement learning. At the time, there was a breakthrough in using neural networks to learn how to play Atari games without human supervision by using "only the pixels and the game score as inputs". However, when watching videos of how the neural network learned to play, I noticed it failed to score any points on the game "Montezuma's Revenge" and thus failed to learn anything at all. I thought that one way to remedy this was to add a "reward" signal for exploring the game environment instead of relying only on the final game score as the sole reward.
With this in mind, I ran experiments to measure a monkey's neuronal activity while it watched visual cues that were associated with different probabilities of future rewards. I ended up characterizing two relevant neuronal signals: one was associated with anticipating an uncertain reward (pre-reward ramping) and the other was associated with receiving a low-probability reward (post-reward spiking). These findings suggest that ramping and spiking can be signals that encode levels of uncertainty, which expands on previous work that focused on neuronal ramping and spiking strictly in the context of more deterministic tasks.