Why Physician Oversight Will Define the Success of Agentic AI

Share:
Dr Panchal explains how healthcare organisations can deploy AI responsibly without diminishing physician oversight

As artificial intelligence advances beyond automation into autonomous, goal-oriented systems, healthcare is entering a new era where AI can actively support clinical decision-making rather than simply assist with administrative tasks. Known as agentic AI, these systems are capable of synthesising longitudinal patient data, identifying emerging health risks, generating clinical documentation, and delivering evidence-based recommendations in real time. While the technology holds immense promise for preventive care, precision medicine, and physician productivity, it also raises critical questions around governance, accountability, transparency, and the evolving role of clinicians.

In this exclusive interview with MedTech Spectrum, Dr Neil Panchal, Chief Medical Officer and Co-founder of Longevitix, discusses why agentic AI should function as a trusted clinical "resident" rather than an unquestioned oracle. Drawing on his experience in emergency medicine and digital health innovation, Dr Panchal explains how healthcare organisations can deploy AI responsibly without diminishing physician oversight. He shares insights on preventing passive reliance on AI recommendations, navigating the emerging medico-legal landscape, leveraging longitudinal health data to transform preventive medicine, and preparing clinicians for an AI-enabled future. He also outlines why the greatest opportunity for agentic AI lies not only in improving clinical efficiency but in democratizing access to high-quality preventive healthcare for millions of patients.

You describe agentic AI as a “resident” rather than an oracle. What specific safeguards and governance frameworks do healthcare organisations need to establish to ensure physicians remain effective supervisors rather than passive approvers of AI-generated recommendations?

The resident analogy works because every physician already knows how that relationship runs. A resident presents a case and proposes a diagnostic and treatment plan. The attending decides and owns the outcome. You don't sign off on a resident's plan because they're confident — you sign off because you've interrogated the reasoning. That's the standard we need to hold AI to, and it doesn't happen by accident. It happens by design.

The non-negotiables are pretty concrete. Every output has to be traceable to its evidence — you click through and you see the guideline, the study, the threshold that triggered the flag. Outputs have to be reproducible, meaning the same patient data returns the same answer every time. The system has to know what it doesn't know and say so instead of filling gaps with plausible-sounding language. And escalation has to be threshold-driven, not "flag everything out of range."

One of your key concerns is “passive deferral,” where clinicians accept AI outputs too quickly. Based on your experience at Longevitix, what practical strategies can healthcare providers implement to foster critical engagement with AI-generated insights while avoiding alert fatigue?

As an emergency medicine-trained physician and someone who has experience implementing EHRs for multiple emergency departments, I can appreciate the trap of "alert fatigue."

A few things help in practice.

First, make escalation earn its place. Don't surface single data points; instead build systems that surface patterns. A meaningful trend across multiple biomarkers, a composite risk score crossing a defined threshold, a directional drift that wouldn't be visible in any one panel are examples of what deserves a physician's attention.

Second, force the system to show its work. Every recommendation arrives with the citation, the threshold, and the patient-specific data that triggered it, visible without leaving the page. Physicians are evidence-based, so show them the evidence.

Lastly, build override into the workflow as a normal clinical act, not a deviation. When the physician disagrees, the documentation should be friction-free. If overriding the algorithm takes three extra clicks and a justification box, the physician learns to stop overriding. That's the moment you've lost them.

As agentic AI systems increasingly draft notes, suggest orders, and synthesise patient data, how do you see the medico-legal landscape evolving? Where should accountability ultimately lie when AI-assisted clinical decisions lead to adverse outcomes?

The framework is certainly unsettled at present. As the industry works towards regulatory clarity, the accountability for a clinical decision sits with the physician who signed it. Bringing back the attending-resident example, the attending physician is ultimately responsible for the diagnostic and treatment plans, interpretation of the results, and documentation, all drafted by the resident. It's fairly similar.

What changes is the differentiation of accountability between clinical decisions made by physicians versus clinical decision support. As physicians carry the responsibility of signing off, we need supportive systems in efforts to build trust amongst physicians and patients, and maximise adoption. Vendors building AI-assisted clinical decision support tools need to be held accountable for their part. If a system produces non-reproducible outputs, hallucinates citations, or operates without an audit trail, the developer also carries weight because those are the outputs that influence a physician's decision-making. Professional liability carriers are already writing AI-specific endorsements and exclusions. These policies should protect physicians, not expose them. Decision-makers at medical institutions need to perform due diligence on behalf of physicians regarding AI-assisted clinical decision support tools by verifying the reasoning the system surfaced, documenting concurrence/overrides, and being able to defend either action. And lastly, patient informed consent language needs updating to ensure patients are aware of the margins of error and risks built in when they opt in for more data-driven and precision-based AI-assisted services.

Preventive and longevity medicine often involves integrating large volumes of longitudinal health data, including biomarkers and wearable-device inputs. How is AI transforming the way clinicians interpret this data, and what impact do you expect on patient outcomes over the next three to five years?

The problem in preventive medicine isn't that we lack data. In fact, we have more data than any physician can integrate even during a generous preventive 30-minute visit. Genomic data, speciality labs, whole-body MRIs, continuous glucose monitoring, sleep architecture, HRV, gait analysis, and so many more diagnostic tools are already available. None of that is useful if it sits in 15 different dashboards and nobody connects it.

The shift AI enables is from single-marker thinking to cross-system pattern recognition. This more naturally mirrors how physicians think. The cardiovascular system integrates with the gut, brain, metabolics, kidneys, sleep, wearable data, and hormones while surfacing meaningful trends across multi-system biomarkers, risk stratification, and identifying subtle directional drift. No clinician has the bandwidth to integrate all that manually. Certainly not for every patient.

In the next three to five years, I think the doctor-patient encounter is redefined to be a moment in continuous care rather than the foundational interaction. Advanced systems like Longevitix are setting up physicians to walk in with a pre-encounter summary that compresses years of longitudinal data into a structured, evidence-linked picture. The visit becomes a conversation about trajectory, wearable streams, and trade-offs, not a scramble to read labs. This supports patient outcomes that matter: earlier identification of metabolic and cardiovascular drift before it crosses a diagnostic threshold, fewer downstream interventions, and more time spent on the conversation that actually changes patient behaviour.

Many physicians remain sceptical about AI’s role in clinical practice. In your view, what distinguishes healthy scepticism from resistance to innovation, and how can healthcare leaders help clinicians build trust in these technologies without compromising patient safety?

Healthy scepticism is sound clinical judgment. A physician who refuses to act on output they can't trace is doing exactly what their training requires. It's not resistance if they ask to see the evidence. Most physicians have already met the wrong kind of AI: confident outputs with no traceable source, tools that contradict themselves on the same patient, protocols that ignore the medication list because the system was never built to read one. It's similar to how an attending physician challenges a resident to explain their thought process after a presentation.

Resistance is when the scepticism stops being about the evidence and starts being about the category. "I don't trust AI" as a blanket statement isn't a clinical position as much as it is a posture. The way to tell the difference is whether the clinician can articulate what would change their mind. If the answer is traceability, reproducible behaviour, validated performance across diverse populations, and physician oversight built into the architecture — that's a healthy sceptic, and you can earn their trust. It's proven every day when you think about an attending physician's evolving trust in a first-year resident physician compared to a final-year resident physician.

Leadership's role in building trust in these technologies is to foster AI literacy across the clinical workforce and put physicians at the AI strategy and governance tables. And I strongly encourage my physician colleagues to demand a seat at those tables.

Looking ahead three years, what do you believe will be the most significant change in day-to-day clinical workflows driven by agentic AI, and which skills will physicians need to develop to thrive in this new human-AI collaborative environment?

The biggest shift in three years isn't a new tool; rather, it's the structure of the clinical day. Pre-encounter, a physician will walk in with a structured briefing that already integrates labs, wearables, history, and a prioritised list of what to address. Intra-encounter, decision support triggers in real time—drug interactions, missed screenings, evidence flags—without having to leave the bedside. Post-encounter, the documentation is drafted, the follow-up is queued, the patient instructions are written. The administrative tax on clinical time finally starts to decrease.

Physicians are excellent at learning new topics. I think AI literacy as a clinical competency will truly help physicians thrive. The curriculum likely includes understanding the difference between deterministic and probabilistic systems, what a citation actually proves, and when to trust a model's confidence. Learning when to push back and how to document the reasoning so it's defensible will be critical. Further, with all of the longitudinal data integration occurring, physicians will need to learn how to interrogate it. And a human skill that becomes even more challenging is the conversation with the patient about what the data means for their life. That's still ours. That should always be ours.

Beyond clinical efficiency, where do you see the greatest untapped opportunity for agentic AI in healthcare—whether in preventive care, patient engagement, population health, or another area—and why?

The foremost impact I believe is access and the democratisation of advanced preventive healthcare. Concierge-level longitudinal medicine currently belongs only to the small slice of patients who can afford it. The same AI clinical decision support architecture that lets one physician hold the cross-system picture for fifty concierge patients could let a primary care doctor hold it for two thousand. Providing bandwidth to the physician allows patients to receive better care. Agentic AI is the first technology I've seen that could meaningfully change that ratio.

Population health is where the multiplier is largest, yet execution remains most challenging. The promise is the ability to continuously risk-stratify a population instead of in retrospective claims data, identify drift before it becomes a diagnosis, and route the right patients to the right level of care. The opportunity only materialises if the validation work matches the deployment ambition. That's a governance problem, not a technology problem.

Patient engagement is the underrated area. Most engagement tools today are reminder apps with a marketing budget. What actually moves adherence is a patient who understands their own trajectory — why this lab matters, why this intervention is necessary now, and what changes if they don't act. Agentic AI can produce personalised, evidence-grounded explanations at the patient's literacy level, in their language, in the physician's voice, and available between visits. That's not a chatbot. That's a continuity layer that closes the gap between encounters, where most clinical drift actually happens.

Lastly, the most obvious untapped opportunity for agentic AI in healthcare is preventive care. At Longevitix, we recognise that the physician constraint has never been "what should we do?" We have known for decades that intervention must begin 10 or 15 years before the diagnostic threshold. The constraint was integration. No clinician can hold years of continuous biomarkers, wearable streams, genomic context, and family history in their head during a 20-minute visit. Agentic AI breaks that wall. The system holds the longitudinal picture, surfaces the cross-system drift, and the physician makes the call.

If we build this generation of tools for traceability, equity, and physician oversight, prevention ceases to be a privilege. That's the opportunity worth chasing.