How Florence Works and Uses AI

Executive Summary  

Florence uses AI in a safe and controlled way.  Florence does not use AI to generate messages to patients; this removes the risk of hallucination and other typical AI safety concerns.  

Our architecture combines: 

1. deterministic, validated Protocols that decide the next best action (message sent to a patient, notification sent to the care team); with  

2. LLM-based Patient Intent Interpreter that determines the intent of each patient message and passes it to the Protocols. 

The configuration of all of Florence’s actions is 100% controlled by our customer’s clinical team. Florence is not a typical conversational AI chatbot.  

The LLM is used only to interpret incoming messages. No PHI is shared with the LLM. This protects patient privacy. 

You can view a brief video explaining our safe-AI architecture HERE.

Typical AI Safety Concerns   

These are the main safety concerns raised by Chief Medical Officers and compliance teams: 

• Regulatory compliance, including HIPAA and state privacy laws, and protection of PHI and PII. 

• Hallucination: LLM can generate incorrect or fabricated responses. 

• Lack of control and transparency risk: Clinicians and risk officers are wary of black-box systems. They want to understand how the system responds to patients reporting high blood pressure, side effects from the drugs, or that they can’t afford their medication. 

• Unreliable longitudinal execution and drift: LLMs are not designed to manage structured, multi-step care pathways over months. They struggle with state tracking and protocol adherence, which leads to drift. 

• Proven use at scale: Has the system managed large patient cohorts across conditions without safety incidents?