And using shinychat::chat_server() instead.
Currently, commons_server() does 3 things:
- Creates a
commons_server_start span.
- Could be removed considering that Shiny already has instrumentation for the session's lifecycle?
- Creates a conversation id.
- "Pre-warms" the client (i.e., builds the index, etc).
- We might want to revisit how this works, but regardless, it might be better if this was explicit rather than implicit. Essentially the (recommended) API would change from:
client <- ...
commons_server("chat", client)
to
client <- ...
client$prewarm()
chat_server("chat", client)
I'm not sure if it's possible today, but at least at some point it should be possible to prewarm/cache the index independently of the shiny runtime. In that case, you might not want the pre-warming at all (i.e., you only update the index as the knowledge base gets updated).
And using
shinychat::chat_server()instead.Currently,
commons_server()does 3 things:commons_server_startspan.gen_ai.conversation.idshinychat#343to
I'm not sure if it's possible today, but at least at some point it should be possible to prewarm/cache the index independently of the shiny runtime. In that case, you might not want the pre-warming at all (i.e., you only update the index as the knowledge base gets updated).