Skip to content

feat: tip message media support (images, videos, GIFs) - #122

Merged
heymariam merged 3 commits into
Dorisio:mainfrom
Abulbayty:solver/issue-64-71-b7180
Oct 3, 2026
Merged

heymariam merged 3 commits into
Dorisio:mainfrom
Abulbayty:solver/issue-64-71-b7180

Conversation

@Abulbayty

Copy link
Copy Markdown
Contributor

Overview

Tip messages can now carry images and short videos. A tipper reserves an upload slot, puts the bytes in object storage, the API verifies what the bytes actually are, scans them for malware and charges the user's storage quota, and only then can the media be attached to a tip.

The repository had nothing for this: no media model, no upload path, no scanning, no storage driver, no quota. This adds the whole vertical slice — TipMedia plus MediaQuota, a media domain (types, storage, scanner, processor, queue, service, factory, routes), the payment-side wiring so POST /api/v1/transactions/tip accepts mediaIds, and the worker job that turns a verified upload into renditions.

Three decisions are worth calling out. The declared contentType and the file name are both attacker-controlled, so detectMimeType sniffs the magic bytes and the result must match the declared type exactly — a file whose bytes disagree is refused and deleted, and a rejected or in-flight upload is never served even to its owner. A scanner that cannot run is not a pass: an unreachable clamd marks the media failed and surfaces the error, and MEDIA_SCANNER=none builds a scanner that refuses everything, so a misconfigured deployment fails closed instead of accepting anything. And media is attached once — the lookup enforces ownership, ready status and "not already on a tip" in one query, so an id that fails any of those rules makes the whole tip request fail with every problem listed rather than half-attaching.

Related Issue

Addresses the behaviour described in #64.

Changes

Model and persistence

  • [ADD] prisma/migrations/20260927200000_tip_media/migration.sql / [MODIFY] prisma/schema.prisma
    • TipMedia: the row is created before the bytes exist (it is the upload reservation) and tracks kind, status, the declared mimeType/fileName/storageKey, the measured sizeBytes, probed width/height/durationSeconds, the derivative list, processingStatus/processingError, the scan verdict (scanner, scanSignature, scanCompletedAt), uploadedAt/attachedAt and the owning tipId.
    • MediaQuota per user (usedBytes, fileCount, limitBytes).
    • Indexes for the three real access patterns (a user's uploads by status and recency, a tip's media in attachment order, the sweep for stale reservations) plus a partial index for attachable media, which Prisma's schema language cannot express — declared in the migration and noted where it is defined.

Media domain

  • [ADD] src/domains/media/media.types.ts

    • Status/kind/derivative/processing enums, the allowed mime types and their extensions, magic-byte signatures for jpeg/png/gif/webp/mp4/webm (RIFF containers are checked for the WEBP marker, MP4 for ftyp, and a short buffer is not accepted), per-kind byte limits, MAX_MEDIA_PER_TIP, and the Zod schemas for reservation, proxy upload and listing.
  • [ADD] src/domains/media/media.storage.ts

    • S3MediaStorage presigns SigV4 query URLs (UNSIGNED-PAYLOAD, expiry capped at the 7-day maximum, session token, custom endpoint/path-style for MinIO and R2) so the browser PUTs straight to the bucket and reads redirect to a short-lived GET. LocalMediaStorage writes under a configurable root behind a guard that refuses any key escaping it. buildCdnUrl switches response URLs to the CDN when one is configured.
  • [ADD] src/domains/media/media.scanner.ts

    • ClamAvScanner speaks the INSTREAM protocol to CLAMAV_HOST, bounded by CLAMAV_TIMEOUT_MS and a byte cap. EicarScanner recognises the EICAR test file and nothing else, and records itself as eicar so it can never be mistaken for a real engine. FailClosedScanner refuses every upload, and is what a clamav configuration without a host falls back to. NoScanScanner exists for the processing worker, which uploads already passed.
  • [ADD] src/domains/media/media.processor.ts

    • SharpImageProcessor (optional dependency, loaded dynamically) produces preview/optimised webp derivatives and records dimensions, skipping images already within the derivative sizes. FfmpegVideoProcessor probes dimensions and duration, extracts a poster frame, and transcodes to webm only when asked; missing binaries skip with an explanation instead of failing the upload. KindRoutingProcessor sends each kind to the right one.
  • [ADD] src/domains/media/media.queue.ts, src/domains/media/media.factory.ts

    • Media jobs reuse the existing image-processing queue and carry only a mediaId, so a retry processes the row's current state rather than a stale payload. The factory builds the storage/scanner/processor stack from configuration in one place, so routes and worker cannot drift apart.
  • [ADD] src/domains/media/media.service.ts

    • requestUpload() checks the declared type and the quota before any byte moves and reserves a pending row; writeUploadedContent() handles the proxy path; completeUpload() verifies the stored size against the real limit, sniffs the magic bytes against the declared type, scans, commits the quota, queues processing and returns the ready view. Rejects delete the bytes; an unreachable scanner marks the media failed rather than clean.
    • processMedia() runs from the worker and records both the derivatives and why they were skipped. listMine()/getMedia()/findById()/canRead() cover reads, with visibility tied to the owning tip. attachToTip() enforces owner + ready + unattached and reports every problem at once. deleteMedia() refuses media a tip is using and refunds the quota. getQuota() separates committed from reserved bytes, and pruneStaleUploads() closes abandoned reservations.
  • [ADD] src/domains/media/media.routes.ts / [MODIFY] src/index.ts

    • POST /api/v1/media/uploads, PUT /api/v1/media/uploads/:mediaId/content, POST .../complete, GET /api/v1/media/quota, GET /api/v1/media (status/kind/attached filters, paged), GET /api/v1/media/:mediaId, GET .../content (owner, or anyone for media attached to a visible tip; redirects to a presigned URL on S3), DELETE /api/v1/media/:mediaId and the admin-only POST /api/v1/media/maintenance/prune. Registered via registerMediaRoutes in both registration blocks.

Tip integration

  • [MODIFY] src/domains/payments/payment.types.ts, payment.schemas.ts

    • CreateTipSchema accepts mediaIds (at most 4) on both the request-validation schema and the service-facing one, and TipResponse carries media.
  • [MODIFY] src/domains/payments/payment.service.ts

    • createTip() resolves the requested media against owner + ready + unattached before writing the tip, so a foreign, in-flight or already-attached id fails the request instead of silently dropping; the tip is created with the media connected, attachedAt is stamped afterwards as best-effort bookkeeping (a failure there is logged, not surfaced, because the tip is already committed). TIP_RESPONSE_SELECT includes the ready media in attachment order and formatTipResponse() renders it as MediaViews.
  • [MODIFY] src/domains/payments/payment.routes.ts

    • The 201 schema for tip creation is a fast-json-stringify schema, which strips anything it does not declare — media is now declared, otherwise the client would never see the media it just attached.
  • [MODIFY] src/lib/workers/image-processing.worker.ts, src/lib/workers/index.ts

    • The image-processing worker recognises the media job by name and derives it; startWorkers() forwards the Prisma client to it, and a standalone worker process creates its own.

Config, tests and docs

  • [MODIFY] src/config/env.ts, .env.example

    • MEDIA_STORAGE, MEDIA_LOCAL_ROOT, the MEDIA_S3_* set, MEDIA_CDN_BASE_URL, MEDIA_UPLOAD_MODE/MEDIA_UPLOAD_TTL_SECONDS, the size and quota limits, MEDIA_SCANNER (defaulting to clamav in production and eicar elsewhere) with CLAMAV_*, and MEDIA_PROCESSOR/MEDIA_TRANSCODE_VIDEO/MEDIA_FFMPEG_PATH/MEDIA_FFPROBE_PATH.
  • [ADD] src/domains/media/__tests__/ (6 suites) and src/domains/payments/__tests__/tip-media.test.ts

    • 94 tests against an in-memory Prisma stand-in and in-memory storage/scanner/processor fakes: mime sniffing per format and the rejections (wrong container, short buffer, plain text), limit and pagination schemas, the full upload lifecycle (quota only committed after verification, idempotent completion, missing bytes, type/size/scan rejections that delete the bytes, a broken scanner failing closed), derivative production and its skip/failure paths, every attachment rule (foreign, in-flight, already attached, too many, duplicates, all problems reported at once), the CDN/presign URLs and local-storage traversal guard, quota accounting, pruning, and the payment side (media attached and stamped, unattachable id refused before the tip is written, no media selected means the media table is never touched, a stamping failure does not fail the tip).
  • [ADD] docs/MEDIA.md

    • The lifecycle and its state machine, why the bytes decide the type, the scanner options and what "fails closed" means, storage/CDN/upload modes, quota accounting including reservations, access control, the media block in TipResponse and deletion rules.

Verification Results

Method: GitHub Contents/Git API only (no clone, no worktree/sandbox, no gh CLI).
Read from Dorisio/backend@main: src/index.ts, src/config/env.ts, prisma/schema.prisma, src/domains/payments/{payment.service,payment.schemas,payment.types,payment.routes}.ts, src/lib/queue.ts, src/lib/workers/*, src/middleware/{auth,rbac,validation,rate-limit}.ts, src/utils/{errors,logger,roles,pagination,jwt}.ts, src/types/response.ts, .env.example.

Executed:
- The new and modified media and payment modules type-check with `tsc --strict --noEmit` against the real dependency types (fastify 5.12.5, zod 3.25, node 22) in a throwaway harness outside the repository, with `@prisma/client` standing in as a type-only module and `jsonwebtoken`/`axios`/Redis replaced by equivalents: zero errors. That pass caught three real defects, all fixed: Fastify v5's `reply.redirect(url, code)` argument order (the v4 form does not compile), `parseOrThrow`'s generic (with `z.ZodType<T>` the schema's input union leaked into its output type), and passing a bare array where `ErrorDetails` (an object with an index signature) is expected. `src/index.ts`, `src/config/env.ts` and `src/lib/workers/*` are edited additively (an added import plus one call/branch/config block) and were reviewed by diff rather than compiled, because their transitive imports are outside the harness.
- The committed test suites were run from that harness: 6 files, 94 tests, 0 failures. They cover mime sniffing per format and its rejections, the schema bounds, the whole upload lifecycle (quota committed only after verification, idempotent completion, missing bytes, type/size/scan rejections that delete the bytes, an unreachable scanner failing closed), derivative production plus its skip/failure paths, every attachment rule, presign/CDN URL construction and the local-storage traversal guard, quota accounting and reserved-vs-committed bytes, pruning, and the payment-side media wiring. The in-memory Prisma stand-in the suites use is the one committed here.

Manual review:
- The 201 response schema in `payment.routes.ts` declares `media` explicitly (verified by serialising a sample response through fast-json-stringify) — without it the property is stripped and the feature is invisible over HTTP.
- The existing `payment.service.test.ts` uses a mock Prisma without a `tipMedia` delegate; the media lookup is guarded on the delegate existing and is never reached when no media is requested, so those tests are unaffected.
- Fields referenced by the services and routes were cross-checked against the schema model by model (`TipMedia.derivatives`, `TipMedia.attachedAt`, `MediaQuota.limitBytes`, `Tip.media`).

Not run in this environment: npm run lint / type-check / test:run / build need a local checkout, which this task is restricted from creating; CI runs them. The new suites follow vitest.config.ts's include patterns and import from vitest.
Acceptance Criteria Status
Media upload endpoint POST /api/v1/media/uploads reserves a slot and returns an upload target (presigned S3 URL, or the API path in proxy mode), plus PUT .../content, POST .../complete, GET /api/v1/media/quota, GET /api/v1/media, GET /api/v1/media/:mediaId and DELETE /api/v1/media/:mediaId
Support images (jpg, png, gif, webp) image/jpeg, image/png, image/gif and image/webp are accepted, verified by magic bytes rather than by the declared type
Support video uploads (mp4, webm) video/mp4 and video/webm are accepted with their own larger size limit, verified by magic bytes
Virus scanning for uploads ClamAvScanner over the INSTREAM protocol in production, EicarScanner for development, FailClosedScanner when scanning is unavailable or disabled — an unreachable scanner marks the media failed instead of clean
Image resizing and optimization SharpImageProcessor produces preview (640px) and optimised (1600px) webp derivatives and records dimensions, skipping images already within the derivative sizes
Video transcoding FfmpegVideoProcessor probes dimensions and duration, extracts a poster frame and transcodes to VP9/webm when MEDIA_TRANSCODE_VIDEO is set; missing binaries skip and keep the original
Storage in cloud (S3) S3MediaStorage presigns SigV4 upload/download URLs (session token, custom endpoint and path-style supported), with LocalMediaStorage for development
Media preview generation preview and thumbnail derivatives are generated, stored alongside the original, exposed as previewUrl/thumbnailUrl and served through GET /api/v1/media/:id/content?variant=…
CDN delivery for performance MEDIA_CDN_BASE_URL switches every URL in a media or tip response over to the CDN; without it the API serves the bytes itself and redirects to a presigned URL on S3
Quota per user (total storage limit) MediaQuota tracks committed bytes per user, in-flight reservations are counted separately and against the limit, GET /api/v1/media/quota reports used/reserved/limit/remaining, and pruning releases abandoned reservations
File validation Declared type, declared size, the size actually stored, and the magic bytes are all checked; a mismatch is refused and the bytes are deleted
Full test coverage 6 new suites / 94 tests covering the upload lifecycle, scanning, processing, storage and presign URLs, quota, attachment rules, the request schemas and the payment-side wiring
Existing tests pass Not run here (no local checkout); CI runs npm run test:run. The existing payment tests use a mock Prisma without a media delegate and are guarded against it

Closes #64

@drips-wave

drips-wave Bot commented Sep 27, 2026

Copy link
Copy Markdown

@Abulbayty Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@heymariam

Copy link
Copy Markdown
Contributor

@Abulbayty well done and thanks for contributing, resolve the conflicts so we can check and merge

Resolves merge conflicts against Dorisio/backend@c6cc8e7 (22 commit(s) behind) so the PR is mergeable.
@heymariam
heymariam merged commit a7b3938 into Dorisio:main Oct 3, 2026
2 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tip Message Media Support (Images, Videos in Tips)

2 participants