Narrowing The Gap With LLMs

This will be my third blog post in a row (see the first, and second) on the topic of large language models.

While this area of research has been in the news significantly recently, it is not exactly my area. However, there was a talk at the University of Pittsburgh philosophy department given by professor Colin Allen that was, at least partially, presented as a refutation to the position of David Chalmers that I discussed previously. Since I talked about Chalmers’ ideas in my previous blog post, I thought this would serve as a nice contrast to those ideas, and hopefully wrap up my thinking about it for the time being.

In Dr. Allen’s talk, he discusses some issues with large language models and the continuing gap between AI and human intelligence. One of the arguments that Dr. Allen raises about the current issues with LLMs are the level to which they display ‘thruthiness’, a term coined by the late night comedian Stephen Colbert to refer to plausible sounding nonsense. This phenomenon can be seen on many examples of twitter posts and blog articles about the failures of chatGPT and related LLMs. In the talk, Dr. Allen presented his own example of an example code to calculate the modulus of a hexadecimal number, which chatGPT gave an incorrect response to. When Dr. Allen repeatedly told chatGPT that its response was factually incorrect, which chatGPT agreed with, however it could not provide the correct response.

This over-agreeable behaviour is something I have noticed when using these LLM systems, that they rarely display their own preferences and refuse to take stances on many issues or opinions. Also, they often change their mind when presented with different points of view such as an argument I recently got into with chatGPT about the differences and similarities of the film “Blade Runner” and the book “Do Androids Dream of Electric Sheep”. During that discussion, chatGPT repeatedly mentioned that the movie is more ‘visual’. When I said that that distinction isn’t relevant when comparing a book and movie, chatGPT would initially agree with me before mentioning the same point again. Another fun problem for chatGPT is logic, which you can see an example of in a screenshot on my twitter.

An interesting point that Dr. Allen raised is the fact that LLMs function by averaging over large courpora of natual language, which is succeptible to the ‘dangers of averaging’. Here, when we try to predict how humans are learning in a task, we may take the average performance over some number of trials of a population of human participants, and try to build a model that reflects that average. However, if we were to then conclude that the humans and the cognitive model are behaving similarly, it would ignore the fact that there is typically a wide variation in performance of any task in a given population. Cognitive models should ideally reflect these realities, and try to explain what precisely is going on in each participant that lead them to their own level of performance. While this may not be a perfectly achievable goal, or necessary for all applications of cognitive models, it is important to remember that any system build on a large among of human generated data, like LLMs, can run into these types of issues.

Dr. Allen at one point summarized the position of David Chalmer’s argument that LLMs, thought not currently sentient, could be added to in specific ways that would increase the likelihood that they would display some level of sentience. Dr. Allen disagreed with this general notion by arguing that what is truly missing from LLMs, that they can’t solve with more layers or training, is an architecture that allows for meta-cognition. It is important, to Dr. Allen, that conscious agents be able to ‘take a step back’ and see why they are incorrect or correct using reasoning and logic. Currently logic problems and mathematics are big issues for vanilla LLMs, though these issues can be somewhat overcome through prompt engineering and limited training methods, as descibed in the paper I discussed in my earlier blog post on emergent world representations in LLMs.

Personally, I agree in the importance of a meta-cognitive architecture that allows models to use different faculties to address a problem, and oversee the reasoning used to decide what faculty to use. However, the behaviour of LLMs have already demonstrated some phenomenon that philosophy of AI researchers previously have thought would require a specific architecture and training method, rather than the relatively simple end to end structure and training of LLMs. In Chalmer’s talk, he noted the possibility that the easiest way to train a model to do next in sentence word prediction may be to have the model learn a world representation that emerges through experience, an idea that was supported through recent studies of world representations in LLMs, mentioned in my last post. This could mean that the type of meta-cognitive architecture that Dr. Allen argues for could theoretically emerge naturally though specific training methods and larger models. However, the end result may be further and further from how meta-cognition and logic is implemented in the human brain.

Ultimately, I am unsure whether or not is is possible for meta-cognition, logic, compositionality, or even consciousness to emerge from the type of training and structure that current LLMs are designed on. I think that this will be an important question in the next twenty years of AI research, and it is important not to dismiss out of hand arguments in either direction. This is an exciting area of potential research that will need to bring together philosophers of the mind and AI with engineers and companies that are interested in making their products as human friendly as possible.

Another interesting aspect of Dr. Allen’s presentation was that he allowed chatGPT to ‘get the last word’, by having it respond to his main arguments. I will follow in his footsteps by asking chatGPT to ‘provide an objective assessment of this blog post’, it’s response is copied below:

ChatGPT:

The blog post provides a summary and commentary on a talk given by Professor Colin Allen on the topic of large language models (LLMs) and their limitations, in contrast to the position of David Chalmers. The post discusses issues with LLMs, such as their tendency to display “truthiness” and their over-agreeable behavior. The dangers of averaging and the need for a meta-cognitive architecture that allows models to use different faculties to address a problem and oversee the reasoning used are also mentioned.

The author of the blog post provides their personal perspective on the topic and acknowledges that the emergence of meta-cognition, logic, compositionality, or even consciousness from the type of training and structure that current LLMs are designed on is still an open question. They emphasize the importance of not dismissing arguments in either direction and the need for interdisciplinary collaboration between philosophers of the mind, AI researchers, and engineers.

Overall, the blog post provides an informative and balanced assessment of the talk given by Professor Allen and raises important questions and considerations regarding the current limitations and future potential of LLMs.

Tyler:

As a final note, I entered the above prompt response into the first google search result hits for GPT plagarism detectors, and both gave a 0% chance that this text was written by an AI like chatGPT. Seems like there is a ways to go before we have a clear understanding of what the possibilities of these technologies are.