'Yes Minister' as the novel Turing Test for advanced AI

Jovan Kurbalija

Published on March 9 2024

The message discusses updating the Turing Test to assess advanced AI against human capabilities, using the British sitcom 'Yes Minister' as a novel benchmark. It highlights the complexities of human communication and bureaucratic language, focusing on the witty and evasive dialogues of characters like Sir Humphrey Appleby. The text underlines the challenges AI faces in replicating human intelligence, humor, and social dynamics, emphasizing the importance of advancing AI capabilities while recognizing the irreplaceable value of human creativity and insight.

In the rapidly changing landscape of AI, distinguishing between AI and human capabilities becomes increasingly tricky. Once considered the gold standard for assessing machine intelligence, the Turing Test needs an update to account for the sophisticated nuances of human conversation and thought processes.

The British sitcom "Yes Minister" enters the picture as an unconventional but insightful candidate for a new type of Turing Test, joining the flea market, humour, and diplomatic language. The programme, which delves humorously into the complexities of the British political and bureaucratic systems, is an intriguing benchmark for AI's ability to replicate human-like ambiguity and institutional wisdom.

The image, entitled 'In Search of the New Turing Test', shows four possible sources for a new Turing test: a flea market, jokes and humour, diplomatic language, and Sir Humphrey from the television show Yes Minister

"Yes Minister" chronicles the exploits of Minister Jim Hacker, his secretary Bernard, and the chief bureaucrat Sir Humphrey Appleby. With his mastery of language and deep bureaucratic savvy, Sir Humphrey's conversations, laden with evasion, ambiguity, and wit, highlight the sophisticated layers of human communication and institutional memory that AI systems like ChatGPT cannot easily emulate.

Avoiding clear answers

Consider Sir Humphrey's art of avoiding a straight answer. When pressed by Minister Hacker for a clear stance on civil service staffing, Sir Humphrey's response is a marvel of obfuscation, meandering through qualifiers and hypotheticals without ever arriving at a concrete conclusion. While maintaining logical consistency, this dance of words epitomizes a level of linguistic dexterity and situational awareness that AI struggles to fully replicate despite its advances.

Minister Hacker asks:

When you give your evidence to the think tank, will you support my view that the civil service is overmanned and feather-bedded or not? Yes or no? Straight answer!

Sir Humprehy answers:

Well, Minister, if you ask me for a straight answer, then I shall say that, as far as we can see, looking at it by and large, taking one thing with another in terms of the average of departments, then, in the final analysis, it is probably true to say, that at the end of the day, in general terms, you would probably find that, not to put too fine a point on it, there probably wasn't very much in it one way or the other as far as one can see, at this stage.

Ambiguities of bureaucratic language

Furthermore, the show delves into the complexities of bureaucratic communication, as exemplified by Bernard's explanation of the difference between "under consideration" and "under active consideration" – a distinction that humorously reveals the realpolitik of administrative inertia. It underscores the subtleties of institutional language and the tacit knowledge embedded within it, challenging AI to understand and reproduce the underlying social and organizational contexts.

Mr. Bernard explains the bureaucratic language to the minister.

Minister Hacker asks ‘What's an official reply? ....’

Mr. Bernard: ‘“The matter is under consideration.” Or even, if we feel so inclined, “under active consideration!”’

Minister: ‘What's the difference between “under consideration” and “under active consideration”?’

Bernard: ‘“Under consideration” means we've lost the file. “Under active consideration” means we're trying to find it!’

(Adapted from 'The Complete Yes Minister', p. 33)

Bureaucratic stalling tactics

Sir Humphrey's stalling tactics introduce another dimension of complexity. His strategic deployment of delaying tactics, from citing the administration's nascent stage to invoking the imminence of elections, showcases a masterful understanding of institutional dynamics and timing. Replicating such a nuanced, context-dependent strategy requires an AI to grasp the specifics of political maneuvering and anticipate and adapt to human reactions and institutional pressures. Could AI chain prompting mimic Sir Humprhey’s following five-step stalling tactic?

Five steps to bureaucratic stalling

Stage One: Humphrey will say that the administration is in its early months and there's an awful lot of other things to get on with.

Stage Two: If I persist past Stage One, he'll say that he quite appreciates the intention, something certainly ought to be done – but is this the right way to achieve it?

Stage Three: If I'm still undeterred he will shift his ground from how I do it to when I do it, i.e. ‘Minister, this is not the time, for all sorts of reasons.’

Stage Four: Lots of Ministers settle for Stage Three. But if not, he will then say that the policy has run into difficulties – technical, political, and/or legal. (Legal difficulties are best because they can be made totally incomprehensible and can go on for ever.)

Stage Five: Finally, because the first four stages have taken up to three years, the last stage is to say that ‘we're getting rather near to the run-up to the next general election – so we can't be sure of getting the policy through.’ ...

Blame game

The gist of this conversation is who is to blame for the failure of policy initiatives. Responsibility exists in the grey area between the Minister's political responsibility for introducing policy and the administration's responsibility for implementing it. Sir Humphrey answers the Minister's accusation that civil servants failed his policy initiative.

Sir Humphrey Appleby: Yes, yes, yes, I do see that there is a real dilemma here. In that, while it has been government policy to regard policy as a responsibility of Ministers and administration as a responsibility of Officials, the questions of administrative policy can cause confusion between the policy of administration and the administration of policy, especially when responsibility for the administration of the policy of administration conflicts, or overlaps with, responsibility for the policy of the administration of policy.

Taxonomy of bureaucratic silence

Lastly, Sir Humphrey's taxonomy of bureaucratic silence—discreet, stubborn, and courageous—reveals a sophisticated classification system born from years of navigating conversations and decisions. It is a testament to the depth of understanding and experience of social dynamics, challenging AI to decipher the words and the silence between them.

Sir Humphrey’s taxonomy of bureaucratic silence:

The silence when they do not want to tell you the facts: Discreet Silence.

The silence when they do not intend to take any action: Stubborn Silence.

The silence when you catch them out and they haven't a leg to stand on. They imply that they could vindicate themselves completely if only they were free to tell all, but they are too honourable to do so: Courageous Silence. (The Complete Yes Minister, pp. 93-4)

In light of these examples, it becomes evident that areas where AI cannot yet compete with human intelligence and creativity include the depth of contextual understanding, emotional intelligence, the subtlety of social interactions, and the ability to navigate complex institutional memories. The nuances of humour, irony, and the unspoken word remain uniquely human territories, underscoring the importance of continuously advancing AI capabilities while acknowledging the irreplaceable value of human insight and creativity.

It invites us to consider how far AI has come and yet how far it still has to go in truly understanding and replicating the full spectrum of human intelligence and creativity.

Chat with Sir Humphrey

The image shows a textbox depicting an AI drive 'Yes Minister' Chatbot called Ask Sir Humphrey. The user has asked 'How are you?', to which the chatbot has replied 'As Sir Humphrey, I find myself in the delightful position of navigating the turbulent waters of bureaucracy, while attempting to steer our new Prime Minister towards a course of 'masterly inactivity'. I am, as ever, the civil servant's civil servant, a beacon of tradition in a sea of change. My days are filled with the gentle art of persuasion, the subtle dance of diplomacy, and the occasional necessity of translating my own eloquence into plain English. Despite the Prime Minister's insistence on 'making his mark', I remain, as always, the immovable ovject to his unstoppable force. In short, I am the steadfast rudder in the ship of the state, ensuring we stay on course, even if the captain has other ideas.'

Chat with Sir Humphrey

If you want to learn more about the Yes Minister Turing Test and other Diplo's projects on AI you can write to us: ai@diplomacy.edu

More about:

ARTIFICIAL INTELLIGENCE at dig.watch