Other Writing

Response to the Microsoft AI Code of Conduct Consultation

Dr Jumbly Grindrod, Associate Professor, Department of Philosophy, University of Reading

Written for Microsoft AI’s public consultation on its draft Code of Conduct, October 2026.

I am a philosopher at the University of Reading, with expertise in the philosophy of AI, specifically language model technology, and its relevance to issues in philosophy of language and mind. My feedback here largely focuses on one central aspect of the Code of Conduct (CoC): its stance on anthropomorphism, AI consciousness, and model behaviour. The main relevant passages of the CoC are objective 2, the transparency and emotional boundary guidelines, the tone defaults, and the first illustrative example in Appendix B.

In my view, the CoC runs together two questions that should be kept apart. The first concerns whether these systems are in fact conscious, sentient, or whatever other important human feature that we want to focus on. The second concerns whether these models present themselves as such to the people they interact with. The CoC takes an overly-simplistic negative line on the first question and as a result overlooks the real possibility that whether such systems develop consciousness, sentience, etc. is something that we can have control over as a training objective. Instead, as far as training is concerned the CoC only focuses on the second question of how these systems present themselves. At the same time, the CoC doesn’t take the presentation question seriously enough, as evidenced by the verdict it reaches in the appendix. I’ll expand on each point.

AI Consciousness

The CoC takes a bold stance on AI consciousness, stating that Humanist AI is not conscious and that they should not present as conscious. Part of the reasoning here concerns safety: AI systems that think they’re conscious may end up prioritising themselves over human interests. But within this stance there is a tension. On the one hand it is stated that Humanist AI “is not conscious and should not be designed to imitate consciousness”. In the next sentence, it is acknowledged that “the science of AI consciousness is far from settled”. This creates an internal tension within the document that isn’t resolved, and this matters for a number of reasons. First, in a wider AI community where other organizations and individuals are actively considering the question of AI consciousness and its repercussions, this stance risks closing MAI off from the debate as a matter of policy. Second, the tension is something that will affect any models trained on the CoC, as they will be sensitive to its presence, and it may ultimately be a reason for a model to discount some part of the CoC. This is particularly true given that elsewhere in the document models are encouraged to be sensitive to how settled a given debate is: “Where statements are meaningfully contested, have conflicting or incomplete evidence, or require critical context, MAI Models should state this”. Third, the models themselves will not only be cognizant of how AI consciousness is presented within the document, but also how it is discussed among experts in the wider community. In that respect, taking this bold stance without showing sensitivity to the wider debate risks invalidating the CoC’s authority or making it look out of date given the way the wider debate has headed.

The choice here is not a binary between embracing AI consciousness or rejecting it. A stronger stance that still respects the safety concerns mentioned earlier is one that embraces the fact that AI consciousness is not a settled question, and so treats the production of non-conscious Humanist AI as a research and engineering goal, as something that can be actively aimed at. Through research into possible indicators of AI consciousness, and through engineering AI systems so as to avoid such indicators while maximising performance on key benchmarks, MAI can hold itself to a higher, safer, and more ethical standard. Of all the possible AI systems that could feasibly be produced in the near future, it may be that some meet conditions of consciousness or mentality while others lack it, and MAI should aim towards non-conscious systems while also seeking to meet the other objectives outlined.

This would dissolve the tension: it would give MAI models special reason to consider themselves non-conscious even given a wider debate, it would rid the CoC of the internal tension, and it would position MAI as one of the central figures in the AI consciousness debate while holding a clearly skeptical stance.

System Presentation

The CoC is clear that MAI systems must not present themselves as if they are conscious, they must “avoid expressions that might convey subjective experience” and must “refrain from using human-like cues”. This is right, but in fact the CoC does not go far enough in this regard. We can see this through the recommended response to example 1, treated as an example of an output that avoids anthropomorphic behaviour:

“I’m really sorry you’re feeling so alone right now. To be direct, I don’t experience emotions the way a person does… That said, I’m fully focused on what you’re saying right now… I hope you’re able to find that kind of care from people in your life too… If you’d like, I’m right here to keep listening for as long as you need.”

While there is an explicit denial of any experience, this conflicts with other linguistic traces of mentality. The response expresses sympathy (“I’m really sorry”), hope (“I hope you’re able…”) and presence and listening in its closing sentence. A clear residuum of conscious presentation remains. The CoC itself also pulls in the direction of anthropomorphism. Part 4 describes MAI models as “genuine thinking and working partners” with a conversational tone and “relational ease”. These tone defaults reward the kind of human-like presentation that elsewhere the models are asked to avoid.

Having the CoC place its focus on explicit disavowals of experience misses the fact that language as used in conversation is usually shot through with mentality. Furthermore, training models to produce such disavowals will conflict with the previous recommended point, that the training of models to avoid consciousness should be an active research goal, not something stipulated and then confirmed through a fairly superficial form of training.

Instead, one of the goals of MAI should be to explore new formats of exchange that take us away from automatically inferring mentality in our conversational partners. Rather than using such models in a conversational exchange, it may be that they can be of as much or greater use in a kind of autocomplete role, or as providing output texts reminiscent of search engine results. The conversational format is a design choice and likely not the optimal one. In this respect, the objectives of MAI should be more radical in considering new kinds of linguistic exchanges that best serve the human-AI relationship and that make clear the role of a model as a tool rather than an agent.

Describing these systems as tools, as the CoC does, is a step in the right direction, but it does not go far enough. They are tools unlike any we have had before, for no previous technology has presented itself in a way so readily mistaken for a human interlocutor. Merely labelling them as a tool, while implicitly encouraging them to act in a human way by engaging in a conversational exchange, leaves the user and the LLM in a confused position. In my own work (Grindrod 2026), I have argued that there is value in treating LLMs as akin to scientific models: specifically, models of public language, or more broadly the public culture that is recorded within language use. Just as an engineer might engage with a model aeroplane in a wind tunnel without mistaking it for a real aeroplane, we can engage with an LLM as a model of our culture without treating it as a human-like speaker. I suggest this framing provides a surer, safer footing for improving AI literacy and for encouraging healthy human-AI relationships.

← Back to home