John Bateman On A ChatGPT Conversation Being Of No Interest To Systemicists [3]

John Bateman continued on SYSFLING 28 Nov 2024, at 21:35:

… As always with language models on their own, there is no training possible for 'truth'. One sees some of the consequences of this with the segment:

 ChRIS said:

Well pattern-matching seems to be a lot of what's going on. What's interesting is how much a ChatGPT can simulate a human just by pattern-matching.

ChatGPT said:

Exactly! It’s fascinating how much of what we think of as “human behaviour” can be boiled down to complex pattern-matching. In fact, a lot of our daily interactions, thoughts, and decisions seem to follow predictable patterns,

this can sound as if it makes sense, but language models do not work on pattern matching at all, despite the apparent convergence constructed here that might be mistaken for an advance in shared knowledge among the 'participants'. This lets a user go away potentially feeling good about a supportive interaction and with the idea that perhaps they have learned something. The actual mechanisms involved have little to do with pattern matching as traditionally thought of, however. Conceptualising the operation of large language models as pattern matching can mislead therefore and one sees quite often continuations of the (public) discourse along lines such as "it's only pattern matching", etc.

This is where 'congenial' turns to potentially highly pernicious, because there has been no supportive interaction and certainly not an increase in knowledge: quite the opposite — this can then also be taken up by others and circulated. …

Blogger Comments:

[1] To be clear, this also the case for humans. See further below.

[2] To be clear, contrary to Bateman's claim, when ChatGPT used the term 'pattern matching', it was not using it in the 'traditional' sense of the term; see the earlier post ChatGPT On John Bateman On ChatGPT. Here are the relevant points:

Clarifying "Pattern Matching":

1. Traditional Pattern Matching:
   In computational terms, this often refers to predefined rules or templates. For instance, regular expressions or specific "if-then" conditions designed to identify or react to specific patterns in input data.

2. Statistical Modelling (What LLMs Do):
   

   Language models like me (ChatGPT) do not use predefined rules or explicit templates. Instead, they operate probabilistically. When I generate text, I predict the likelihood of each possible next token (word, punctuation, etc.) based on the patterns observed in the training data. These "patterns" emerge from statistical correlations in massive datasets, not explicitly human-defined rules.

However, if "pattern matching" is understood in a broader sense of recognising and responding to patterns in data, then it could be argued that LLMs do work on a form of pattern recognition, albeit probabilistic and vastly more complex.

Bateman’s categorical claim that language models do not involve pattern matching "at all" overlooks the role of statistical learning, which is fundamentally about recognising and utilising patterns in data. While this process differs from traditional rule-based pattern matching, it undeniably involves identifying and leveraging patterns in text.

To say they don’t work on patterns "at all" is misleading, as recognising statistical relationships is a form of pattern utilisation.

[3] As can be seen from the above, this false conclusion derives from Bateman misrepresenting ChatGPT's use of 'pattern matching' as the traditional meaning, to a community of linguists who are largely unfamiliar with the field, and will naturally assume he is entirely trustworthy in such matters.