Applying LLMs in Cognitive Models

Two submissions our lab has been working on were recently accepted to the AAAI 2023 Fall Symposium Series on the Integration of Cognitive Architectures and Generative Models. This was an exciting series for me to see since I have written multiple blog posts on this site about LLMs, but hadn’t thought deeply on how to apply them to cognitive models. LLMs definitely have been an interest of mine, but I hadn’t had a good reason to push for their integration in the models that we work with.

The model used for my thesis does employ a type of generative model, but not a LLM, as I thought of text and computational linguistics as too far from my own interests in visual perception and utility-based learning and decision-making. I think that in general most people tend to be a bit hesitant about the use and applicability of LLMs in their own areas of work, while also believing that LLMs and other AI models will replace many other jobs that they are not intimately familiar with. For this reason I believe that it is fortunate that the symposium will look into methods for incorporating LLMs and other generative models into cognitive architectures.

The submission that I worked on was titled: “Generative Environment-Representation Instance-Based Learning: A :hamster: Cognitive Model” by Tyler Malloy, Yinuo Du, Fei Fang, and Cleotilde Gonzalez. This work focused on the integration of generative models with Instance-Based Learning models of human utility-based learning and dynamic decision making. We looked at applications of combining the major groups of models that take in as input visual information, autoencoders, generative adversarial networks, and visual transformers.

All these generative models were integrated with an Instance-Based Learning model, which serves as the foundation of the DDM labs work in modelling human learning and dynamic decision making for the past 20 years. We applied these generative models onto the following task:

In this task, both human participants and cognitive models were presented with a trinary choice task where they had to make a selection between 3 options based on the features of that option. They made their selection and then observed a reward feedback, which told them how good their choice was. Initially this was based on the shape, then color was introduced as a new feature, and finally the texture was added as well. The models and the participants needed to learn what the relation between the previous experience was, and how it impacted new experience. Specifically, one of the three shapes was initially high-value, and associated with roughly a reward of 6 vs 4 from the other two shapes. Then, participants needed to learn how those initially high-value features impacted the value of future options they experienced. We applied the GERIBL model using 2 of each type of generative model, and compared their performance against humans (green line) and a standard IBL model (purple line).

In addition to comparing the model’s performance with humans, we also compared how well the generative models could transfer their visual representation formation strategies from one task to another. To do this, we trained 3 different versions of each type of cognitive model using only the subset of stimuli related to a single task (i.e. only shape-color images, or only shape images). The difference between these modes training is shown in the shading of the above figure. The main result is that models that have small representation sizes (green, right) are not significantly impacted by training method, while models with much larger representations (red, left) are impacted.

I am excited to see where this model can next be applied, such as in domains with both textual and visual information. The DDM lab has many broad interests and there are many ways we can think of applying LLMs in different contexts, such as cybersecurity research, human-autonomous teaming with complex communication, or other more fundamental investigations of human learning such as transfer of learning.