AI Model Services

We provide our members with the resources needed to deploy, evaluate, clinically test, and monitor AI models.

After you develop a machine learning model, you’ll need help deploying it for testing at Michigan Medicine. We have the systems, processes, and expertise needed to move your research forward.

For more information, contact:
Phil Jacokes, Managing Director, Digital Health Innovation

Model Deployment

We provide our members with the resources needed to deploy, validate, and evaluate AI models.

Our Model Deployment Team provides key resources to deploy models to make them available for real time health data access. We work seamlessly with our partners to provide crucial model deployment support including:

  • - Hosting models

  • - Connecting models to data pipelines

  • - Running prospective data through models

“The MPrOVE team developed the Ambulatory Surgery Center (ASC) model to predict if each specific surgical case is appropriate for surgery at an ASC. Digital Health Innovation provided critical expertise needed to implement this model within MiChart, helping organize surgical case and patient data and packaging the model for use with Epic’s Cloud Computing Platform. Following deployment, Digital Health Innovation supported model validation and advancements. They are amazing partners helping the digital health research community apply practical solutions to the challenges of clinical teams.”

James Henderson, PhD
Assistant Research Scientist

Model Evaluation

Digital Health Innovation uses evaluation metrics to understand your model’s performance by assessing the quality of predictions and determining its strengths and weaknesses.

Our Model Deployment team has expertise in analyzing AUC, sensitivity, specificity, PPV, cohort, and target outcome and will apply out expertise to answer question “is this model ready for people to use?”. This is an important step in determining if a model is ready for clinical testing.

For more information, contact:
Phil Jacokes, Managing Director, Digital Health Innovation

“We needed to evaluate three models to determine which could most effectively reduce readmission rates for us at Michigan Medicine. Digital Health Innovation provided the expertise to evaluate the models and delivered an unbiased analysis on the most effective model for us to use. What a great services!”

Name
Title

Model Testing

Digital Health Innovation is the exclusive partner for testing models within UMHS. Request our support today to see how we can help you test your model in the hospital!

Testing a model in the clinical setting at UMHS is a complicated process with many touch points including IRB, Michigan Medicine’s Health Information Technology & Services (HITS), the Clinical Intelligence Committee (CIC), Epic, hospitalists, nurses, and more.

We are here to guide you through these challenging pathways and to apply our best practices for testing and evaluating your model.

Our team can help:

  • Provide IRB guidance and consultation

  • Connect to subject matter experts and clinical champions within Michigan Medicine

  • Coaching our members prior to CIC approval meetings

  • Interact with HITS for MiChart integration

  • Endorsement from Clinical Operational Governance Committees and Leadership

  • Facilitate design sessions with clinicians, nurses, and support staff

  • Project Support/Management/Oversight

For more information, contact:
Phil Jacokes, Managing Director, Digital Health Innovation

“Digital Health Innovation guided me through the steps needed to develop a clinical trial within Mott’s Children’s Hospital. Their understanding of the workflows, governances, and systems needed for the clinical trial were essential to were essential for accelerating the PICTURE Pediatric trial .”

Sardar Ansari, PhD
Assistant Professor, Emergency Medicine

For more information, contact:
Phil Jacokes, Managing Director, Digital Health Innovation

Model Performance Monitoring

Monitoring your model enables you to analyze the accuracy of the prediction and allows you to tweak the model to optimize performance.

Model performance monitoring is the process of tracking and evaluating the performance of machine learning models. It helps ensure models are accurate, reliable, and effective. Model monitoring is needed to identify factors causing models to change over time including: data distribution changes, training-serving skew, data quality issues, and environmental shifts.

All models deployed in our hosting environment are monitored while they are tested in the clinical setting at UMHS. We work closely with our members to ensure that deployed models are working optimally while they are being tested.

“Model performance monitoring is critical in detecting the inevitable data shift and model degradation that will take place when a model is used in a clinical setting. As a Data Scientist, I need this information to ensure that my models are operating as optimally as possible. ”

Brittany Bauer, PhD
Max Harry Weil Institute for Critical Care Research and Innovation
Michigan Medicine