search = fragile federal data systems and safeguards while doge examines
There is also some reader commentary at this emptywheel post, if you word search "cobol" you will reach it, and one post seems noteworthy:
The Crabgrass guess is the DOGE young very smart computer science recent grads are aware of danger, and surely do not want to be jumped if having broken anything, so the thread of system integrity breakage is minimized. These are not dumb or careless young people. So, what would you expect them to be doing?
The Crabgrass guess to that is, make a database backup sandboxed copy of what is encountered in visiting each data processing center, one or several per agency, do some plain text extraction, and feed that into an AI LLM to tune its giant network to the new input, to be able to query, "Summarize instances within the new data about activity of ENEMY 1A122" where that would be some actual person against whom they'd want to tell a damaging story. For that, the integrity of a cobol legacy system would not be in peril. And this is political, not information gathering for a neutral effort, but to tell a story consistent with good guys, bad guys, and "Here's all we learned, and it is still the tip of the iceberg," narrative.
Back to that opening query: three things returned as an example of problems:
https://www.yahoo.com/news/doge-access-federal-data-absolute-011818394.html
titled:
DOGE’s access to federal data is ‘an absolute nightmare,’ legal experts warn
and:
https://federalnewsnetwork.com/management/2025/02/doges-unimpeded-access-to-classified-data-poses-national-economic-security-risks/
titled:
DOGE’s ‘unimpeded’ access to classified data poses national, economic security risks
and finally:
titled, "Five former Treasury secretaries issue stark warning about DOGE takeover," in which a warning letter from past Treasury illuminaries is noted:
Five former U.S. Treasury secretaries have warned that Elon Musk’s new Department of Government Efficiency may pose a threat to democracy, with efforts being made to “unlawfully undermine the nation’s financial commitments.”
Robert E. Rubin, Lawrence H. Summers, Timothy F. Geithner, Jacob J. Lew, and Janet L. Yellen said they had taken the “extraordinary step” of writing the New York Times op-ed due to concern over the “arbitrary and capricious political control of federal payments.”
This follows a federal judge temporarily restricting Musk and DOGE from accessing a critical Treasury Department payment system that distributes tax returns, social security benefits, and disability payments, among other things. The judge cited a risk of “irreparable harm.”
Now consider the cost of this effort seemingly able to "get the goods" on enemies, if indeed used that way, where LLM hallucination would be a major evidence problem if thing against enemies were to reach litigation. The bright boys of DOGE, if in litigation, would have to go back to agency IT experts, "We think it is in there, but we need quality evidence, so go dig it out and be expert witnesses," which, lacking any better term, would be extreme waste.
It might have other uses to know what others have been shown in data to be doing, but that litigation example, and the LLM halucination problem, are real.
DOGE might not shut down many programs, or find quick and easy admissible evidence, but it could pull some good stories into daylight, friends and enemies in actions, which might have political capital value. Or it might in the court of public opinion be enough to say, "Our state of the art AI discovered XYZ about person q, and we did it all PDQ."
One can hypothesize Elon Musk having the bright boys query things, "Summarize and list all payments to Blue Origin?" Since that is something we can imagine would be of interest to the SpaceX guy. And who'd know if he did such querying?
It represents a big knot of stuff, and we all know what is best there. Alexander and the Gordian Knot.
___________UPDATE__________
Presuming the best of DOGE and not the worse, what would a good LLM query prompt be if finding waste or fraud of great magnitude? Something like:
For the new payment data provided, list the hundred-and-twenty highest amount of grants or other payments, by XYZ Department, over the last five years, and write three summary paragraphs for each including who got the check, what guardrails were imposed, what followup was in place, which politicians got campaign contributions from the payee, and who within government initiated the disbursement, as well as any other information having a high probability of relevance.
That might be having to be broken into promp, response, reprompt sequences, and there might be better ways to skin a rat, but the idea is if waste and fraud are really the targets, DOGE could be compelled to show its efforts to not just sniff around data, but to actually seek and effectively find - objectively and not per any enemy lists - what actually happened. I.e., impose accountability constraints on Elon's crew, and not just let them monkey around at will. Trump and JD could do that, were it a priority to them.
___________FURTHER UPDATE____________
For those with technical curiosity, code translation has been an area of development.
For example: Presuming part of the DOGE database inquries involves access to source code and not merely compiled COBOL, there is IBM ready to be paid by firms when staff IT people are perplexed.
IBM trains its LLM to read, rewrite COBOL apps
The new watsonx Code Assistant for Z eases mainframe modernization, using generative AI to analyze, refactor, transform and validate legacy applications.
That would allow a fast cut to a language the DOGE braintrust might better understand, and readers can search llm access of structured data. The Internet does have resources to be found by search which would yield structured data >> llm few shot learning where detail error can be tolerated with the big picture aim to be to get something into llm prompt-ready form which is not too wholly off base.
For generic sniffing out fraud or waste, perfection of loading info into llm form is not needed. If some waste or fraud is missed, much waste or fraud will be apparent.
The aim is to pin down major waste or fraud, not to pinpoint all waste and fraud.
FURTHER: In terms of basic internet search, such a search uncovered this stackoverflow item. A part of what search will find. A few hours and you'll know more.
FURTHER:
How well can LLMs write COBOL?
LLMs are fast-changing the way that we write software. Over a million developers now pay for GitHub Copilot and recent breakthroughs in LLM reasoning have brought the dream of a fully AI Software Engineer closer to reality. But while it’s not hard to find a demo of an LLM coding a website or a clone of Flappy Bird, not much is known about their ability to write code in older ‘legacy’ languages like COBOL.
The opportunity for LLM COBOL generation is huge. Although the language was first released in 1959, it continues to power critical systems - 95% of US ATM transactions are processed in COBOL. But it's not taught in computer science courses or bootcamps, and the engineers who write it professionally are steadily retiring. If LLMs could understand and write COBOL they could help maintain the 800 billion lines still in production today.
So, how well can LLMs write COBOL? As far as we know, nobody has publicly tried to answer this question. Until now…
Introducing COBOLEval
Today we’re releasing COBOLEval, the first evaluation benchmark for LLM code completions in COBOL. It consists of 146 challenging coding problems that have been converted into COBOL from the widely-used HumanEval Python generation benchmark. Each problem is paired with an average of 6 test cases. An LLM-generated solution has to pass all of them to be correct. We’re also releasing a test harness that you can use to evaluate your own models, as well as mAInframer-1 - a series of open-source models based on CodeLlama that we’ve fine-tuned specifically to write COBOL - which outperform GPT-4.
And, trust me, there is more online about structured data < or > llm use.
As a X IBM mainframe COBOL programmer some 45 years ago though not on systems as complex as these. The answer is very probably yes.[that DOGE agents could break data systems]
They might try to glean the logic in the programs – and that assume source code is available and reflective of the compiled code being run.
On the other hand – and sort of a separate issue no less important – is being able to extract and understand the schema of the “database” even if they raw copied it.
I used a variety to top tier IBM and non IMB databases – DB2, Total, just plan old VSAM files, etc. Just Google IBM mainframe data structures.
Having done that I do now see IBM Z/OS related to legacy mainframe updates so it may be that there is something that “modernized” the old circa 60-70s stuff. It’s still likely very fragile.