Learning to Learn

One interesting development in my research interests is the unexpected focus on ‘learning to learn’ that many of my recent papers and projects have been focused on.

During my PhD, I had shyed away from the more complex topics in cognitive modeling, such as meta learning. This decision was in part motivated by my interests in understanding the most basic and fundamental aspects of human and machine learning. Another aspect of this decision was the increased difficulty and potential problems that are associated with designing more complex tasks. The overall motivation behind much of my research is in bridging the gap between the understanding of how the human mind functions in reality, and how we choose to design machine learning methods. The hope is that through an increased understanding of how the human mind achieve behavioral goals, we can design more human-like machine learning methods that better reflect cognative realities. However, this motivation presupposes the usefulness of machine learning methods that are similar to the human mind.

The simplest justification for designing human-like ML is that humans excell at a variety of tasks, such as learning to learn, and if we can build ML models that reflect human abilities in these ways, we can improve them. However, much of the research in AI is not interested in doing things in a human-like manner, rather preferring to describe some loss function and optimize model behavior relative to it.

While many recent methods in machine learning, as well as classical approaches, do draw heavily from motivations related to human learning, much of the recent success in areas like Large Language Models (LLMs) appear to go against what we know about human learning and behavior. One specific example is that models that are designed for conversation and text generation typically do so by iteratively predicting the next word, and maximizing the probability of that word relative to the dataset that it is trained on. One major factor of these models is that they are stateless, meaning they aren’t basing their pedictions based on some existing model of the world or an environment they exist in. This obviously seems very disconnected from the way that humans generate language, which is intrisically tied to the environment that they are in which defines their goals, preferences, and desires, and in turn the language that they generate.

One of the most relevant aspects of recent ML methods to the question of the difference between human and machine learning is model overparameterization, which can be simply described as the case where the number of parameters in a ML model exceeds the number of data instances that the model is trained on. Part of the reason why LLMs do not need the same type of highly specialized language generation structure that the human mind has is that they are overparameterized. This means that hypothetically, the same type of world model based behavior that humans rely on could be an emergent phenomenon resulting from the overparameterization of LLMs and their training regiem in predicting the next word in a sentence.

But there is a quesiton of whether or not the human mind is overparameterized in a similar way as LLMs. Traditionally we think of human cognition as highly limited in its capacity, which can be assumed to exist due to the highly biased behavior of humans, and the apparent biological realities of the limitations of information processing in the brain. However, in his speech to NeurIPS 2022, Geoffrey Hinton noted that ‘Perhaps… we are data limited rather than capacity limited’, due to the disparity between the experience of humans (as measured in seconds) and the relatively larger number of neurons in the brain. This raised the question of whether information capacities are sufficiently impactful in determining the behavior of humans.

Related to this issue of whether the human brain is overparameterized is the question of the source of meta-cognitive abilities in the human brain and ML methods. LLMs have been suggested to have meta-cognitive abilities even though they were not trained explicitly to produce meta-cognition. This raises the question of whether faculties like transfer of learning in humans emerge as a natural result of our brains being trained and optimized through evolution (though thinking of evolution as a direct optimizing force or process is not correct) and experience to achieve some goal distinct from meta-cognition. This is related to the issue of how brains vs. LLMs are trained. Predicting the next word in a sentence seems counterintuitive to assumptions of human language generation, but it seems highly connected to theories of human cognition like the free energy principle, which suggests that the brain may at its base function to predict future states.

Overall, I don’t think that the differences in training and structure of humans and LLMs can be swept under the rug. There may be many emergent phenomenon in both humans and LLMs that are not explicitly reflected in the ways that these models and brains are ‘trained’, but the existence of two emergent phenomenon that produce similar behavior does not entail a structural similarity in the base models themselves. To me, there is still a value in designing ML models with a human-like justification.