From Messy Med-Reports to Smart Insights: Building a FHIR-Powered Medical RAG with Milvus

We've all been there: staring at a stack of PDF medical reports, trying to remember if that one blood test result from three years ago was "normal" or "borderline." In the age of AI, why is our personal health data still trapped in static documents?

Today, we are building a Medical-Grade Personal Knowledge Base. We aren't just doing basic PDF parsing; we are implementing a sophisticated Medical RAG (Retrieval-Augmented Generation) pipeline. By combining the FHIR standard for clinical data interoperability, Milvus for high-performance vector search, and BGE Embeddings for semantic precision, we will transform scattered PDFs into a searchable, time-aware health history.

The Architecture: From Pixels to Structured Knowledge

Handling medical data requires more than just a Ctrl+F search. We need to normalize data so the AI understands that "High BP" and "Hypertension" refer to the same clinical concept.

Here is how the data flows from a raw PDF to a queryable insight:

graph TD
    A[Raw PDF Reports] --> B[Unstructured.io Partitioning]
    B --> C{LLM Extraction}
    C -->|Map to Standard| D[FHIR JSON Resources]
    D --> E[BGE Embeddings Model]
    E --> F[(Milvus Vector Database)]
    G[User Query: 'How has my glucose changed?'] --> H[Query Embedding]
    H --> I[Milvus Semantic Search]
    I --> J[Contextual Augmented Response]
    F --> I

Prerequisites

To follow this tutorial, you'll need:

• Python 3.9+
• Unstructured.io: For heavy-duty PDF partitioning.
• Milvus: Our vector powerhouse (using Milvus Lite for easy setup).
• BGE Embeddings: Specifically BAAI/bge-small-en-v1.5 for a great balance of speed and accuracy.
• FHIR (Fast Healthcare Interoperability Resources): The industry standard for health data.

Step 1: Parsing PDF with Unstructured.io

Medical reports are notoriously messy—tables, headers, and footnotes everywhere. We use unstructured to extract clean text while maintaining document hierarchy.

from unstructured.partition.pdf import partition_pdf

# Partitioning the PDF into manageable elements
elements = partition_pdf(
    filename="blood_test_report_2023.pdf",
    infer_table_structure=True,
    strategy="hi_res"
)

# Join the text for processing
raw_text = "\n\n".join([str(el) for el in elements])
print(f"Extracted {len(elements)} elements from the document.")

Step 2: Normalizing to FHIR Standard

Standardizing data is the "secret sauce" of medical AI. Using the FHIR standard ensures that our RAG system understands clinical context. Instead of storing "Blood Sugar: 110", we store a Observation resource.

import json

# A simplified FHIR Observation schema
fhir_observation = {
    "resourceType": "Observation",
    "status": "final",
    "code": {
        "coding": [{"system": "http://loinc.org", "code": "2339-0", "display": "Glucose"}]
    },
    "valueQuantity": {
        "value": 110,
        "unit": "mg/dL"
    },
    "effectiveDateTime": "2023-10-27T10:00:00Z"
}

# In a real app, an LLM would map the 'raw_text' to this JSON structure

Step 3: Vectorizing with BGE & Storing in Milvus

Now we need to make this data searchable. We use BGE Embeddings to turn our FHIR resources into vectors and store them in Milvus.

from pymilvus import MilvusClient
from sentence_transformers import Transformer

# Initialize BGE Embeddings
model = SentenceTransformer('BAAI/bge-small-en-v1.5')
client = MilvusClient("health_history.db")

# Create a collection in Milvus
client.create_collection(
    collection_name="medical_records",
    dimension=384  # Dimension for bge-small
)

# Convert FHIR to string and embed
data_str = json.dumps(fhir_observation)
vector = model.encode(data_str)

# Insert into Milvus
client.insert(
    collection_name="medical_records",
    data=[{"id": 1, "vector": vector, "text": data_str}]
)

Advanced Patterns & Production Readiness

While this setup works for a personal project, building a production-grade medical system involves HIPAA compliance, complex FHIR terminology mapping (SNOMED CT, LOINC), and handling multi-modal data (like X-rays).

For deeper dives into advanced medical data patterns and building more robust healthcare AI, I highly recommend checking out the technical deep-dives at WellAlly Blog. They cover production-ready RAG architectures that are specifically tuned for data privacy and clinical accuracy.

Step 4: Semantic Retrieval

Finally, we can query our knowledge base. Instead of keyword matching, we search for the meaning of the query.

query = "What were my last blood sugar readings?"
query_vector = model.encode(query)

results = client.search(
    collection_name="medical_records",
    data=[query_vector],
    limit=3,
    output_fields=["text"]
)

for res in results[0]:
    print(f"Found Record: {res['entity']['text']}")

Conclusion: The Future of Personal Health

By combining FHIR, Milvus, and BGE, we've moved from "dumb" PDFs to a structured, semantically searchable medical knowledge base. This is the foundation for an AI health assistant that can truly track your longitudinal health history.

What's next?

1. Multi-document tracking: Compare results across different years.
2. Agentic RAG: Let the AI suggest when you should schedule your next check-up based on the data.

Are you building something in the Medical AI space? Drop a comment below or share your thoughts on health data privacy!

If you found this tutorial helpful, don't forget to ❤️ and 🦄! For more high-level architectural patterns, visit the WellAlly Blog.