Boot
Computing Infrastructure for Clinical AI
Needs and challenges -- NSDF Summit on Federated Autonomous Science
Rudolph Pienaar -- Boston Children's Hospital / Harvard Medical School
Whoami
Rudolph Pienaar
Computational Scientist
Systems Architect
Brain Image Researcher
Assistant Professor of Radiology, Harvard Medical School
Technical Director, Fetal-Neonatal Neuroimaging & Developmental Science Center (FNNDSC), Boston Children's
PhD Biomedical Engineering (bio-control-model reinforcement learning) -- BEng / MEng
Whatido
What I Do
Build the infrastructure that runs research computation inside a hospital.
ChRIS -- open-source platform: containerised analyses, run next to clinical data.
HARBOR -- a registry of processable data. Find the cohort, send the compute to it.
Hospital on one side, research cloud on the other. I live on the boundary.
THE HARD PART WAS NEVER THE MODEL. IT IS EVERYTHING AROUND IT.
Outline
Current State of AI in Hospitals
Current State of "AI" in Hospitals
Honestly? Not much -- not in the Northeast, not at Boston Children's.
Our lab and team deployed the only semi-production AI workflows in clinical radiology at BCH.
They are predictive: images in, labels and segmentations out.
Generative AI is in use -- on text: notes, documentation, productivity.
Where it touches diagnosis it is advisory: a human reads every output.
Generative AI at BCH: Becker's Hospital Review 2024 · OpenAI customer story, May 2026 (vendor-published)
What the Numbers Say
What the Numbers Say
How much AI is in US healthcare in 2026 -- and doing what?
use any "AI" -- surveys don't say which kind; top uses are generative
health systems███████████████75%
physicians████████████████81%
top use in health systems: ambient note-taking (generative)
clinical notes█████████████▌68%
FDA-cleared AI devices (predictive) that read images
radiology███████████████76%
1,614 FDA-cleared AI devices -- tagged as generative / LLM-based: none.
GENERATIVE AI WRITES THE NOTES. PREDICTIVE AI READS THE IMAGES.
Eliciting Insights via Fierce Healthcare, Mar 2026 (n=120 systems) · AMA physician survey, Mar 2026 (n=1,692) · FDA AI-enabled device list, counted Oct 2026 (1,230 of 1,614)
Current State of AI in Hospitals
Current State of "AI" in Hospitals
What AI can do in a lab vs what hospitals actually run
Deployed clinical AI: predictive, mostly imaging.
Generative AI: notes and documentation, a human in the loop.
Agents now beat specialists in simulation (88% vs 78%) -- yet finish only 46% of real EHR tasks.
"AI IN HOSPITALS" IS MUCH SMALLER THAN THE HEADLINES.
Ferber et al., Nature 2026 (MIRA, simulated patients) · Liu et al., PhysicianBench, arXiv 2605.02240 (2026, preprint)
What Exactly Is AI?
What Exactly Is "AI"?
Today, "AI" means LLMs.
But the AI delivering clinical value is pattern matching: tumour detection, tissue labelling, segmentation.
Predictive: a deterministic forward pass. Rich input, bounded output.
Generative: samples one plausible answer from many. The probabilism is the task.
Can LLMs Read Images?
Can LLMs Read Images?
Frontier AI vs radiologists on 200 hard image cases (RadLE 2.0, Jul 2026)
score out of 2000: right answers earn points, confident wrong answers lose them
radiologists█████████▌989
best AI model███████▌758
16 frontier models tested, incl. Gemini 3 Pro, GPT-5.6, Claude Fable 5. None reached the radiologists.
Diffusion models make and clean images -- they don't read them.
BEST MODEL: 23% BELOW RADIOLOGISTS -- AND OFTEN CONFIDENTLY WRONG.
CRASH Lab, RadLE 2.0 technical report, Jul 2026; scores via The Decoder, 19 Jul 2026 · Kazerouni et al., Med Image Anal 2023 (diffusion survey)
What Exactly Is AI?
What Exactly Is "AI"?
Same word, two machines -- they warrant different trust.
This session's example of the first kind: Alter's small-cohort multiomics predictors -- linear algebra, not an LLM.
On "trustworthy autonomy": in the clinic the trust problem is not autonomy -- it is telling the two apart.
CALL IT ALL "AI" AND YOU GOVERN IT ALL LIKE A CHATBOT.
What Is Computing in a Hospital?
What Is Computing in a Hospital?
For clinicians, until recently: Outlook and scheduling.
Behind that: enterprise databases for billing.
Computing as in actual computing? Barely existed. Still barely does.
Forget AI -- most hospitals can't do any advanced computing.
Resources, Not Regions
Resources, Not Regions
Predictive vs generative AI, by hospital type
% of US hospitals using each kind, built into the EHR (2024, same national survey)
█predictive█generative
all hospitals███████71%███31.5%
in a health system████████▌86%███▌38.5%
independent███▌37%█▌16.3%
urban████████81%███▌35.5%
rural█████▌56%██▌25.4%
predictive = ML or statistical risk models · generative = LLMs (ChatGPT, Gemini, DAX)
THE HYPE IS GENERATIVE. THE USE IS PREDICTIVE. BOTH FOLLOW THE MONEY.
Predictive: ASTP Data Brief 80, Sep 2025 · Generative: Everson et al., JAMA Netw Open 2025, Table 1 · both from the 2024 AHA IT Supplement; imaging AI outside the EHR not counted
What Is Computing in a Hospital?
What Is Computing in a Hospital?
Isn't the cloud the answer? Not yet.
"Move to the cloud" mostly means a hosted EHR and some workloads -- not research computing.
43% of healthcare IT leaders name the skills gap as the main cloud barrier.
Planned 2026 spend on AI governance: large systems 6.8% of IT and quality budget, small hospitals 2.3%.
THE GAP FOLLOWS MONEY AND SIZE.
KLAS via Becker's, Jul 2025 (n=36) · Rackspace State of Cloud 2025 (vendor, n=150) · Black Book via Becker's, Oct--Nov 2025 (n=182)
Governance
Governance
Well-meaning people who can't define AI, writing policy to govern it.
Every policy assumes "AI" means generative AI.
So it stalls on anything else.
The 100 MB Model
The 100 MB Model
100 MB · images in · images out · runs on a tablet
Q: hallucination guardrails? -- It does not sample. Deterministic forward pass.
Q: PHI leaking to big model companies? -- There is no text. Nothing leaves.
The process stalls anyway.
Governance
Governance
How ready are hospitals to govern AI? (health-system surveys, late 2025)
Only 18% of health systems have mature AI governance and a full strategy.
AI submissions outpace committee decisions ~3:2 -- a growing backlog.
Only 22% could produce an AI audit trail within 30 days.
SCRUTINY IS HEAVIEST ON THE SAFEST MODELS.
HFMA/Eliciting Insights, Aug 2025 (n=233) · Elion, The AI Governance Playbook, Nov 2025 (vendor) · Black Book via Becker's, Oct--Nov 2025 (n=182)
It's All About the Infrastructure
It's All About the Infrastructure
The first real need.
Places that didn't know computers could compute now need sophisticated computing.
Cloud. Schedulers. Containers. Orchestration.
Alien to IT departments built to install locked-down Windows desktops.
This Room Already Knows
This Room Already Knows
Federated data access. Provenance. Workflow portability. Governance.
Medicine needs exactly these -- under stricter rules, and the data cannot move.
Alter's small, interpretable predictor still needs >50 TB and HPC for one trial.
The NNSA labs built shared, credentialed infrastructure across their sites (Bremer). Each hospital is on its own.
THE MODEL IS SMALL. THE DATA AND THE COMPUTE ARE NOT.
Alter, NSDF Summit 2026, slide 37 (WGS alignment files, one trial) · Bremer (LLNL), NSDF Summit 2026
It's All About the Infrastructure
It's All About the Infrastructure
The value is in small predictive models. Hospitals can't run them, and can't approve them.
Infrastructure is the common fix.
THE HARD PART WAS NEVER THE MODEL.
What Can Be Done?
What Can Be Done?
Triage governance by what a model is: images or text? local or remote? deterministic or sampled?
A fast lane for small, local, image-in/image-out models.
Shared infrastructure, not a data centre per hospital: validated containers run next to the data (ChRIS).
Move the Computation
Move the Computation
Do not move the data. Move the computation.
Find the cohort, send the compute to it. Only results come back -- no images, no PHI.
A small hospital joins with a key, not a data centre.
Xu: the bottleneck moved upstream of training. Here it's the site itself -- FedReady puts an LLM agent and a GPU inside every site.
Bremer weighs data at the source against data near compute. Medicine has no choice: compute goes to the data.
DO NOT MOVE THE DATA. MOVE THE QUESTION.
Xu (NVIDIA), Bremer (LLNL), NSDF Summit 2026
What Can Be Done?
What Can Be Done?
Treat hospitals as first-class sites in national data fabrics.
AURORA (Kirby) offers industry encrypted containers, full access logs, "your data stays yours." Offer it to hospitals.
Trust for people and agents (Hart) -- for data that stays sensitive for a lifetime.
The hospital side brings the data and the questions.
THE HOSPITALS LEAST ABLE TO COMPUTE NEED THIS MOST.
Kirby/Sparks (AURORA, Utah), Hart (Internet2), NSDF Summit 2026
The AI Bubble
The AI Bubble
What tech conferences rarely say: the numbers don't add up
Gartner, 14 Oct: AI data centres need $8.5T of revenue by 2030 to pay back; forecast is $3.6T. Over 80% of neoclouds gone by 2030.
IMF, 13 Oct: 14 "circular" AI deals -- ~$950B of equity and debt in exchange for $1.4T of promised future revenue.
Oracle: credit insurance at a record high (15 Oct); rated one notch above junk.
OpenAI projects ~$278B of cash burn 2026--2030. Anthropic: cash-flow breakeven not before ~2028 (reported).
Ed Zitron has made this case for two years. This week Gartner caught up.
Gartner via The Register, 14 Oct 2026 · IMF GFSR via TNW, 13 Oct 2026 · Oracle CDS: S&P/LSEG data via Securitas, 15 Oct 2026; S&P rating Jul 2026 · OpenAI: FT via Reuters, Sep 2026 · Anthropic: secondary reporting · Ed Zitron, Where's Your Ed At
What the Money Is Betting On
What the Money Is Betting On
The trillion-dollar valuations rest on three claims
Claim: AGI is close -- "we know how to build AGI" (Altman); "a country of geniuses in a datacenter" (Amodei).
Evidence: frontier models trail radiologists on hard images and finish 46% of real EHR tasks.
Claim: the machines are, or will become, conscious. Evidence: none -- consciousness science leans cautiously towards no for current AI.
Claim: they solve what humans can't. Sep 2026: OpenAI says its AI proved Navier-Stokes blow-up -- a $1M Millennium Prize problem.
Look deeper: two mathematicians had worked the same gap for a year, LLMs as tools. Formal proof and paper disagree. Clay still lists it active.
A GOOD NEW TOOL, PRICED AS IF IT WERE A GOD.
Altman, "Reflections", Jan 2025 · Amodei, "Machines of Loving Grace", Oct 2024 · CRASH Lab RadLE 2.0 · Liu et al., PhysicianBench 2026 · arXiv 2605.06965 (2026) · Navier-Stokes: OpenAI, Quanta, Axios, 8 Sep 2026; arXiv discrepancy paper, 6 Oct 2026; Clay Institute status Oct 2026
Earned vs Lost
Earned vs Lost
OpenAI and Anthropic: revenue vs operating loss, 2025
2025, $ billions -- what each company took in vs what it lost running the business
OpenAI earned████████████▌$13.1B
OpenAI lost████████████████████$20.9B
Anthropic earned████$4.6B
Anthropic lost███████▌$8.1B
Headline net losses were bigger ($38.5B, $42B) -- mostly one-off accounting charges, left out here.
BOTH LOSE MORE THAN THEY EARN.
Leaked draft IPO filings: OpenAI via Reuters/Yahoo Finance; Anthropic via Reuters/Fortune, 29 Sep 2026 · not yet public filings
The Subsidy
The Subsidy
What the flat monthly plans hide: per-token vs flat, my own bills
my own bills, similar coding workload
per token, 1 week████████████████████$3,500
flat plan, 1 month▌$100
Per month that's roughly $15,000 vs $100 -- about 150x.
Altman, Jan 2025: losing money on the $200 ChatGPT Pro plan -- "people use it much more than we expected."
Anthropic, Jul 2025: weekly caps on Claude Code for users running it "continuously in the background, 24/7."
TODAY'S FLAT PRICES ARE A SUBSIDY. BUDGETS BUILT ON THEM WON'T LAST.
Speaker's own billing: BCH enterprise Codex (per token) vs personal Claude Max (flat) -- different vendors, illustrative · Altman on X, 6 Jan 2025 (via Fortune) · Anthropic via TechCrunch, 28 Jul 2025
The Gap, in Two Charts
The Gap, in Two Charts
Revenue needed vs forecast -- and data centres delivered vs scheduled
cumulative AI data-centre revenue to 2030, $ trillions
needed to pay back████████████████████$8.5T
forecast████████$3.6T
US data-centre capacity added per year, gigawatts
2024 delivered█████████6.4 GW
2025 delivered████████████▌8.5 GW
2026 scheduled████████████████████13.6 GW
2026 delivered, Q1███2.2 GW
THE REVENUE ISN'T THERE. THE BUILD-OUT RUNS BEHIND ITS OWN PLAN.
Gartner, 14 Oct 2026 (via The Register) · Goldman Sachs Research, 2026 (via secondary coverage): realized additions 2024-25, 2026 schedule, Q1 2026 actual; no full-year 2026 figure published yet
If It Pops
If It Pops
What a crash would mean -- and why hospitals are, ironically, buffered
Economy: a burst could cut UK GDP by 2.2% (Bank of England); a correction would ripple through a "highly leveraged, circular" sector (IMF).
Hospitals, ironically, are buffered: they barely use generative AI yet.
Predictive AI just keeps running: 100 MB, on-prem, no frontier lab's balance sheet behind it.
Who gets hurt: whoever just bought into generative AI -- repricing, failed vendors, stranded integrations.
Science: a crash leaves cheap GPUs behind -- dark fibre after 2001, for federated compute.
BUILD ON WHAT SURVIVES THE CRASH.
Bank of England Financial Stability Report, Jul 2026 (via Crypto Briefing) · IMF GFSR, Oct 2026 · adoption: Everson et al. 2025; ASTP 2025 · implications are the speaker's argument
The AI Bubble
The AI Bubble
After the crash: models you can run yourself are catching up -- with a catch
my own study: median time to fully fix a 7-bug Python package (every setup below solved it)
Opus 5.5, hosted██74 s
GPT-6.1, hosted██▌94 s
local 27B, A6000████████████████9 min
local 27B, Mac████████████████████11 min
Local matched hosted on a bounded task -- 30 of 30 -- just 6-7x slower.
The "4 months behind" figure is for open models on data-centre GPUs. On a workstation: smaller models, smaller context windows.
Closing that gap takes enterprise GPUs -- infrastructure most hospitals don't have.
GOOD ENOUGH FOR BOUNDED WORK. THE REST IS AN INFRASTRUCTURE PROBLEM.
Pienaar, "Local Language Models for Terminal Coding Agents in a Medium-Size Laboratory", unpublished (arXiv, in prep.), Oct 2026: 234 trials; qwen3.6:27b, 4-bit, 128K context · Epoch AI, 29 May 2026
Conclusions
Conclusions
The AI that works in hospitals is small, predictive, and local.
Hospitals were built to run care, not computation.
Governance written for LLMs blocks the models that already work.
The fix: shared infrastructure, compute to the data -- and governance that can tell a segmentation model from a chatbot.