I have built AI products. I have shipped them to real users. I have spent years thinking carefully about what large language models can and cannot do. And I am telling you: what a DOGE staffer reportedly did with federal research grants is one of the most reckless applications of AI I have ever heard of from anyone with institutional authority.

According to Techdirt's reporting, a staffer working with the Department of Government Efficiency reviewed federal grants for so-called 'DEI content' by doing the following: pasting grant descriptions into ChatGPT and asking whether they qualified as DEI-related. That output - from a consumer AI chatbot, with no domain expertise, no audit trail, and no way to verify accuracy - was then used to make decisions about actual research funding.

Let me be extremely precise about what that means technically.

ChatGPT is a large language model. It predicts the next most plausible token in a sequence based on patterns in its training data. It does not have a factual understanding of government grant regulations. It does not know the legal definition of any category of spending. It will confidently generate a response whether or not that response is accurate. It can and does hallucinate - produce confident-sounding answers that are simply fabricated. It has no memory of previous queries, cannot be audited in any meaningful sense, and its outputs vary even when given identical inputs.

None of this makes ChatGPT a bad tool for the things it is actually good at. But 'adjudicating whether federally funded research meets a politically defined content criterion with real consequences for researchers, institutions, and the public programs those grants support' is emphatically not one of those things.

This is a governance catastrophe wearing the clothes of an efficiency measure.

The stakes deserve to be spelled out. Federal research grants fund universities, nonprofits, medical researchers, public health programs, and scientific work that takes years or decades to produce results. Cutting them is not like canceling a software subscription. When a research program loses funding mid-cycle, graduate students lose stipends, ongoing studies are discontinued, and years of work can become unrecoverable.