Separate health check readiness from Stellar dependency checks

[DeFiVC]

Bug Description

The health check endpoint at /health (lines 209-235 in src/server.ts) calls Stellar Horizon and Soroban RPC as part of the health check. If either Stellar service is down, the health check returns 503 even though the API itself is healthy.

Location

src/server.ts lines 209-235

The Problem

The health check performs 4 checks:

1. Database (SELECT 1)
2. Redis (PING)
3. Stellar Horizon (root call)
4. Soroban RPC (health check)

If any of these fail, the entire health check returns 503 "degraded". This means:

• A Stellar network outage makes the API appear "down" to load balancers
• Kubernetes/Docker health checks may restart healthy API instances
• Users cannot access non-Stellar features (course browsing, quiz taking) during Stellar outages

Recommended Fix

Separate the health check into:

1. Liveness (/health/live) - Is the process alive? (always 200)
2. Readiness (/health/ready) - Can it serve requests? (DB + Redis only)
3. Full Health (/health) - Detailed status including external dependencies

The readiness check should only verify core infrastructure (DB + Redis), not external blockchain services. Stellar-dependent endpoints can handle their own degradation.

Current State

Note: /health/live already exists and returns { status: "ok" }. However, /health/ready has the same 4-check structure as /health, making it equally fragile.

Acceptance Criteria

• /health/live - Returns 200 if process is alive (already done)
• /health/ready - Returns 200 if DB and Redis are healthy (remove Stellar checks)
• /health - Keep as detailed status endpoint for monitoring dashboards
• Update documentation to explain the difference between readiness and liveness

Severity

medium - Can cause unnecessary container restarts and load balancer failures during Stellar outages.

[guptakumarranjeet150]

Hey team,

I am interested in working on this issue.
Plan:

1. Build out the handlers for Separate health check readiness from Stellar dependency checks ensuring proper HTTP status codes and error propagation.
2. Add integration test suites for standard invocations and boundary/error conditions.

Looking forward to collaborating on this—please assign!

[Emoji-dot]

@Emoji-dot has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

I would like to work on this issue

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Emoji-dot to this issue.

[Peolite001]

@Peolite001 has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

I would love to work on this issue. Kindly assign me

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Peolite001 to this issue.

[adeboladee]

@adeboladee has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

Kindly assign this task to me maintainer,i would fix this bug and conflict free

ℹ️ Repo Maintainers: To accept this application, review their application or assign @adeboladee to this issue.

[kolawoleoluwaferanmi014-a11y]

@kolawoleoluwaferanmi014-a11y has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

Hello Maintainer, I’m interested in working on this issue and have reviewed the requirements carefully. I’m confident I can deliver a quality implementation that satisfies the acceptance criteria. Please consider assigning it to me.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @kolawoleoluwaferanmi014-a11y to this issue.

[Onefie10]

@Onefie10 has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

Hello. I want to work on this before 2pm. Thank you.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Onefie10 to this issue.

[Fayedamz]

@Fayedamz has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

Hello Maintainer, I'd love to work on this issue.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Fayedamz to this issue.

[official-rex]

@official-rex has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

I’m interested in working on this can deliver before the end of wave

ℹ️ Repo Maintainers: To accept this application, review their application or assign @official-rex to this issue.

[joewee-x]

@joewee-x has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

Hi Maintainer, I'm a smart contract/full-stack dev (Rust, Soroban, Cairo, TypeScript) focused on secure, clean blockchain code. I've reviewed the issue and can deliver a complete solution within 24 hours. Please assign it to me

ℹ️ Repo Maintainers: To accept this application, review their application or assign @joewee-x to this issue.

[ife-daniel]

@ife-daniel has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

I can do the recommended fixes
Assign me

ℹ️ Repo Maintainers: To accept this application, review their application or assign @ife-daniel to this issue.

[Kao-thar]

@Kao-thar has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

Hi, I have looked at the issue and I understand what's needed. I would love to work on this.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Kao-thar to this issue.

[dammy-lab]

@dammy-lab has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

I can work on this issue because it is a focused backend reliability and health-check improvement involving API infrastructure, dependency isolation, and service availability. The main work is to separate process liveness from application readiness and external dependency health.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @dammy-lab to this issue.

[oscar24357]

@oscar24357 has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

Hello maintainer, can I work on this issue please.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @oscar24357 to this issue.