How Florence Works and Uses AI
Executive Summary Florence uses AI in a safe and controlled way. Florence does not use AI to generate messages to patients; this removes the risk of hallucination and other typical AI safety concerns. Our architecture combines: deterministic, validated Protocols that decide the next best action (message sent to a patient, notification sent to the care team); with LLM-based Patient Intent Interpreter that determines the intent of each patient message and passes it to the Protocols. The configuration of all of Florence’s actions is 100% controlled by our customer’s clinical team. Florence is not a typical conversational AI chatbot. The LLM is used only to interpret incoming messages. No PHI is shared with the LLM. This protects patient privacy. You can view a brief video explaining our safe-AI architecture HERE.Typical AI Safety Concerns These are the main safety concerns raised by Chief Medical Officers and compliance teams: Regulatory compliance, including HIPAA and state privacy laws, and protection of PHI and PII. Hallucination: LLM can generate incorrect or fabricated responses. Lack of control and transparency risk: Clinicians and risk officers are wary of black-box systems. They want to understand how the system responds to patients reporting high blood pressure, side effects from the drugs, or that they can’t afford their medication. Unreliable longitudinal execution and drift: LLMs are not designed to manage structured, multi-step care pathways over months. They struggle with state tracking and protocol adherence, which leads to drift. Proven use at scale: Has the system managed large patient cohorts across conditions without safety incidents?
