Add support for multiple model APIs by ParamThakkar123 · Pull Request #23 · JuliaHealth/HealthLLM.jl

ParamThakkar123 · 2026-04-17T12:18:09Z

Summary

This PR adds comprehensive support for multiple model APIs in HealthLLM.jl, enabling users to easily switch between different LLM providers for text generation and embeddings.

Key Changes

Provider Interface: Designed a common ModelProvider and EmbeddingProvider interface for consistent API usage.
Supported Providers: Implemented providers for HuggingFace, Groq, Gemini, OpenAI, Anthropic, and Ollama.
Configuration: API keys and settings configured via environment variables.
Error Handling: Added retry logic for API calls with exponential backoff for rate limits and timeouts.
Registry: Created a ModelRegistry for managing and switching providers dynamically.
Updated Modules: Modified Embedding, Query, and main modules to use the new provider system.
Demos: Added example scripts for OpenAI and Anthropic providers.
Tests: Included unit tests for providers and registry.
Documentation: Updated README with usage instructions.

Usage Example

using HealthLLM

# Create and use a provider
provider = OpenAIProvider()
response = generate(provider, "Hello, world!")

# Register for easy access
register_provider("my_openai", provider)

This enhancement significantly expands HealthLLM.jl's capabilities by supporting a wide range of LLM services.

… modules

…to model_apis

… to use library functions

…xible RAG generation

…main module

Copilot

Pull request overview

This PR introduces a provider-based abstraction in HealthLLM.jl to support multiple LLM backends for text generation and embeddings, plus a small registry and new demos/tests to exercise provider construction and switching.

Changes:

Added Providers and Registry modules and wired them into the main HealthLLM module exports.
Updated query/index-building entry points to accept provider objects (with some backward-compat overloads).
Added RAG helpers (PromptTemplate, simple embedding/indexing, retrieval, and answer generation) plus demos and basic provider/registry tests.

Reviewed changes

Copilot reviewed 13 out of 15 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
`src/providers.jl`	Defines model + embedding provider abstractions and concrete providers (HF/Groq/Ollama/Gemini/OpenAI/Anthropic) with retry helper.
`src/registry.jl`	Adds a simple global provider registry with register/get/list helpers.
`src/rag.jl`	Adds simple RAG utilities (chunking, hashing embedder, retrieval, prompt templating, answer generation).
`src/query.jl`	Refactors query generation to accept providers and adds a backward-compat overload.
`src/embedding.jl`	Changes `build_index_rag` to require an `EmbeddingProvider`.
`src/HealthLLM.jl`	Includes new modules and exports provider/registry/RAG APIs.
`test/runtests.jl`	Runs new provider/registry tests.
`test/providers_test.jl`	Adds basic tests for provider construction + registry.
`demo/openai.jl`	Adds OpenAI demo script.
`demo/anthropic.jl`	Adds Anthropic demo script.
`demo/groq.jl`	Adds Groq demo script.
`demo/gemini.jl`	Adds Gemini demo script.
`README.md`	Documents provider usage and prompt templating.
`Project.toml`	Adds `HTTP` dependency for direct API calls.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-17T12:50:59Z

+    # Default to some embedder, but since RAGTools might expect different, perhaps error
+    error("Please provide an embedder::EmbeddingProvider")


The previous build_index_rag(cfg, files; embedder_kwargs=...) method now unconditionally throws, which is a breaking change despite the “backward compatibility” comment. If you want to deprecate this overload, consider keeping the old behavior (or a reasonable default embedder) and emitting a deprecation warning, or introduce a new function name for the provider-based API.

Suggested change

# Default to some embedder, but since RAGTools might expect different, perhaps error

error("Please provide an embedder::EmbeddingProvider")

Base.depwarn(

"build_index_rag(cfg, files; embedder_kwargs=...) is deprecated; " *

"pass embedder::EmbeddingProvider explicitly to build_index_rag(cfg, files; embedder=..., embedder_kwargs=...)",

:build_index_rag,

)

return RAGTools.build_index(cfg, files; embedder_kwargs=embedder_kwargs)

Copilot · 2026-04-17T12:51:00Z

+Set your API keys in `.env` file or environment variables:
+
+```bash
+HUGGINGFACE_TOKEN=your_hf_token
+GROQ_API_KEY=your_groq_key
+GEMINI_API_KEY=your_gemini_key
+OPENAI_API_KEY=your_openai_key
+ANTHROPIC_API_KEY=your_anthropic_key
+```


README instructs setting HUGGINGFACE_TOKEN, but the implementation reads ENV["HF_TOKEN"] for HuggingFace providers/embedders. This mismatch will lead to authentication failures when users follow the docs. Update the README to match the code’s env var name (or change the code to accept HUGGINGFACE_TOKEN).

Copilot · 2026-04-17T12:51:00Z

+end
+
+if !isinteractive()
+    main()


This demo calls main() under both if abspath(PROGRAM_FILE) == @__FILE__ and if !isinteractive(). When executed as a script, both conditions are typically true, so main() runs twice. Keep only one of these entry-point guards to avoid double execution.

Suggested change

end

if !isinteractive()

main()

Copilot · 2026-04-17T12:51:01Z

+end
+
+if !isinteractive()
+    main()


This demo calls main() under both if abspath(PROGRAM_FILE) == @__FILE__ and if !isinteractive(). When executed as a script, both conditions are typically true, so main() runs twice. Keep only one of these entry-point guards to avoid double execution.

Suggested change

end

if !isinteractive()

main()

Copilot · 2026-04-17T12:51:02Z

+    chunks = String[]
+    chars = collect(text)
+    pos = 1
+    while pos <= length(chars)
+        endpos = min(pos + chunk_size - 1, length(chars))
+        push!(chunks, strip(join(chars[pos:endpos])))
+        pos = endpos + 1


chunk_text accepts an overlap argument but never uses it when advancing pos, so there is no actual overlap between chunks. This makes the API misleading and reduces retrieval quality. Use overlap when computing the next pos (e.g., advance by chunk_size - overlap) or remove the parameter.

Suggested change

chunks = String[]

chars = collect(text)

pos = 1

while pos <= length(chars)

endpos = min(pos + chunk_size - 1, length(chars))

push!(chunks, strip(join(chars[pos:endpos])))

pos = endpos + 1

chunk_size > 0 || throw(ArgumentError("chunk_size must be positive"))

overlap >= 0 || throw(ArgumentError("overlap must be non-negative"))

overlap < chunk_size || throw(ArgumentError("overlap must be smaller than chunk_size"))

chunks = String[]

chars = collect(text)

pos = 1

step = chunk_size - overlap

while pos <= length(chars)

endpos = min(pos + chunk_size - 1, length(chars))

push!(chunks, strip(join(chars[pos:endpos])))

pos += step

Copilot · 2026-04-17T12:51:03Z

+end
+
+if !isinteractive()
+    main()


This demo calls main() under both if abspath(PROGRAM_FILE) == @__FILE__ and if !isinteractive(). When executed as a script, both conditions are typically true, so main() runs twice. Keep only one of these entry-point guards to avoid double execution.

Suggested change

end

if !isinteractive()

main()

Copilot · 2026-04-17T12:51:03Z

+end
+
+if !isinteractive()
+    main()


This demo calls main() under both if abspath(PROGRAM_FILE) == @__FILE__ and if !isinteractive(). When executed as a script, both conditions are typically true, so main() runs twice. Keep only one of these entry-point guards to avoid double execution.

Suggested change

end

if !isinteractive()

main()

Copilot · 2026-04-17T12:51:03Z

+function with_retry(f::Function, max_retries::Int=3, backoff::Float64=1.0)
+    for attempt in 1:max_retries
+        try
+            return f()
+        catch e
+            if attempt == max_retries
+                rethrow(e)
+            end
+            if isa(e, HTTP.ExceptionRequest.StatusError) && e.status in [429, 500, 502, 503, 504]
+                sleep(backoff * attempt)
+            else
+                rethrow(e)
+            end


with_retry checks isa(e, HTTP.ExceptionRequest.StatusError), but HTTP.jl status exceptions are typically HTTP.Exceptions.StatusError. As written, this branch likely never matches, so rate-limit / 5xx responses won't be retried. Update the exception type check to the correct HTTP exception type (and keep the status filter) so retries actually occur.

Copilot · 2026-04-17T12:51:04Z

+export ModelProvider, EmbeddingProvider, HuggingFaceProvider, GroqProvider, OllamaProvider, GeminiProvider, OpenAIProvider, AnthropicProvider, HuggingFaceEmbedder
+
+abstract type ModelProvider end
+abstract type EmbeddingProvider end
+
+# Abstract methods
+function generate(provider::ModelProvider, prompt::String; kwargs...)
+    error("generate not implemented for $(typeof(provider))")
+end
+
+function embed(provider::EmbeddingProvider, texts::Vector{String}; kwargs...)
+    error("embed not implemented for $(typeof(provider))")
+end


generate/embed are used unqualified in other modules (e.g., Query and RAG), but they are not exported from Providers. Either export generate and embed from this module or require callers to reference them as Providers.generate / Providers.embed; otherwise downstream modules will hit UndefVarError at load time.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

ParamThakkar123 added 6 commits April 17, 2026 17:34

Add support for multiple model APIs: providers, registry, and updated…

dee7ffa

… modules

Complete multi-model API support: error handling, demos, tests, docs

8f159e2

Merge branch 'main' of https://github.com/JuliaHealth/HealthLLM.jl in…

d5013fd

…to model_apis

Refactor demos: move RAG utilities to src/rag.jl module, update demos…

fb0dd41

… to use library functions

Add customizable prompts with PromptTemplate, like LangChain, for fle…

48a07b4

…xible RAG generation

Fix relative imports in rag.jl to use specific submodules instead of …

95205e0

…main module

ParamThakkar123 requested a review from Copilot April 17, 2026 12:45

Copilot started reviewing on behalf of ParamThakkar123 April 17, 2026 12:46 View session

Copilot AI reviewed Apr 17, 2026

View reviewed changes

Apply suggestion from @Copilot

56b8936

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for multiple model APIs#23

Add support for multiple model APIs#23
ParamThakkar123 wants to merge 7 commits intomainfrom
model_apis

ParamThakkar123 commented Apr 17, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Copilot AI Apr 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		# Default to some embedder, but since RAGTools might expect different, perhaps error
		error("Please provide an embedder::EmbeddingProvider")

-    # Default to some embedder, but since RAGTools might expect different, perhaps error
-    error("Please provide an embedder::EmbeddingProvider")
+    Base.depwarn(
+        "build_index_rag(cfg, files; embedder_kwargs=...) is deprecated; " *
+        "pass embedder::EmbeddingProvider explicitly to build_index_rag(cfg, files; embedder=..., embedder_kwargs=...)",
+        :build_index_rag,
+    )
+    return RAGTools.build_index(cfg, files; embedder_kwargs=embedder_kwargs)

Conversation

ParamThakkar123 commented Apr 17, 2026

Summary

Key Changes

Usage Example

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants