Acadiant RUBRICS: AI-Assisted Exam Assessment for Medical Education

The Challenge: The Bottleneck in Grading High-Stakes MD Exams and High-Frequency PA Exams

Medical education depends on rigorous, consistent assessment of clinical reasoning and documentation. As cohorts and assessments grow more complex, manual grading becomes a bottleneck—exhausting for faculty, costly for the university, and delayed for students.

"Faculty time is one of our most valuable resources. I am excited to collaborate with Acadiant on this cutting-edge technology, which has the potential to dramatically reduce the time faculty spend grading while improving consistency and interrater reliability. Innovations like this allow educators to spend less time on repetitive tasks and more time helping students become exceptional physicians."

Dr. Deborah DeWaay, MD, FACP, Senior Associate Dean for Undergraduate Medical Education, Department of Medical Education, Professor of Internal Medicine, University of South Florida

For high-stakes MD exams, physician faculty must read and score each response repeatedly against a detailed rubric and document the rationale for decisions, consistently and accurately. The work is repetitive, and the cost of hiring and retaining qualified graders, as well as the time the tasks demand, constrains the number of exams and ultimately the class size—in an environment with a shortage of trained physicians. For high-frequency Physician Assistant (PA) exams, the repetitive stress while maintaining high standards leads to faculty exhaustion and burnout. In both programmes, instructor time spent on rote scoring is time not spent providing meaningful feedback to help students' development.

The Solution: Acadiant RUBRICS

Acadiant RUBRICS is AI-assisted exam assessment software that scores students' responses against the university's own proprietary rubrics, produces a score and a written reason for every rubric line item, and requires the grader to actively confirm or override each result. The grader's decision is final, and the university remains the grader of record. AI assists; the human is always in the loop and responsible for every decision.

Feedback Summaries and Insights

Acadiant RUBRICS provides feedback summaries and insights for each graded student exam, which faculty can then tailor for conversation with students. The software includes AI-generated longitudinal analyses of students' work, AI-generated heat maps to see cohort trends, and specific tips to improve performance taken from the rubrics and faculty comments.

Confidentiality

The rubrics are the university's intellectual property and are not exposed externally or to students. AI models are not trained on the university's data and score only against the institution's rubrics, not external medical knowledge. Acadiant continuously researches and tests AI models to identify the most cost-effective and best-suited models for the task.

Deployment

Acadiant RUBRICS is deployed into the university's own computing environment, available through the Microsoft Azure Marketplace or the AWS Marketplace, under the control of the university's IT staff and compliance policies.

Analytics

Acadiant RUBRICS generates extensive analytics that academic faculty now produce manually. These include performance by rubric item, interrater consistency (including time spent and overrides), feedback insights for student follow-up meetings, and longitudinal student progress.

Future Features

As AI reliability and capability improve, Acadiant will continue to enhance the software to include ambient audio and video grading of standardized patient exams, and student self-study modules.

Expansion

The same assessment bottleneck exists across nursing, pharmacy, dentistry, veterinary medicine, and allied health, and Acadiant RUBRICS is built for all of them.

Adoption

Acadiant at the University of South Florida

USF Health Morsani College of Medicine selected Acadiant RUBRICS for its MD and PA programmes. The platform is equally applicable to nursing, pharmacy, dentistry, veterinary medicine, and allied health programmes.

As Acadiant CEO Bill Chute noted at a panel discussion of the British American Business Council of Central Florida, "There's something very special about USF as opposed to anywhere else, even in the state of Florida—the openness to experimentation, to judicious innovation, taking a bit of a risk." See USF World News: "With life sciences solutions in high demand, U.K.-Florida Innovation Corridor is thriving."

Expected Outcomes

Projected impact by stakeholder:

Faculty
Over 30 min down to 2 min per exam
  • Initial grading time per exam is expected to drop from more than 30 minutes to about 2 minutes, with more consistency across graders, giving faculty space to help students develop clinical reasoning rather than rote grading.
  • AI-generated feedback suggestions accompany each exam, providing faculty ready-made summaries to draw on in meetings with students.
Administration
Consistent grading at scale
  • Capacity to absorb more exams, and increase cohort size, without hiring additional graders, easing the constraint on enrolment at a time of projected physician shortages.
  • Faculty scores and overrides are transparently captured, strengthening interrater reliability and giving programmes consistent, comparable data on student performance.
Students
Faster results, timely feedback
  • Faster turnaround, timely feedback, and more consistent scoring.
  • More opportunities for feedback to help students sharpen their clinical reasoning, build on their strengths, and identify areas for development over the course of their study.

Join the Founding Cohort

Contact rubrics@acadiant.com to talk to us. We're happy to answer questions, demonstrate Acadiant RUBRICS, and provide references.