RedShirt.Example.Api

Forkable ASP.NET Core API template: rename the namespace, keep the scaffolding. Includes JWT authorization, optional Roslyn-generated data access, file uploads, OpenAPI clients via NSwag, and a local Docker Compose stack.

Template

Template Philosophy

The central philosophy of this template is flexibility and preparedness. A template maintained with lessons from past projects can provide a stable foundation from which to launch future projects. The use of a template is more sustainable and resource-efficient than adapting directly from past projects.

This template on its own will likely be more feature-rich than any one project needs. Because of this, this template is designed to make it convenient to prune away unused components. It is much easier to delete an unneeded component than it is to generate a new component.

Features

Repo features in more detail:

• Initialisation script for quick and convenient namespace adjustment.
• Use of NSwag to automatically document endpoints to OpenAPI standard and to generate client code for an interop package.
  • Recommended next step: Exporting the interop package as a NuGet for use in other projects.
• Roslyn source generation for MariaDB/Dapper data-access scaffolding (services, repositories, search requests, and related DI) from annotated DTO models.
  • This is demonstrated in the implementation for accessing the Order data store.
  • For more information on source generation, see docs/patterns/rosalyn-source-generation.md.
• Configurable rate limiting using a sliding window system:
  • Uses either Redis or in-memory for storing limits.
• JWT bearer authentication and role-based authorization (optional; Keycloak in the local Compose stack).
  • Realm roles (admin, developer, analyst, billing) map to permission claims; endpoints authorize on those permissions, not on role names.
  • Resource-based authorization on orders: callers without api:unrestricted (admin) may only access rows whose CustomerId matches the JWT customer_id claim.
• Example connectors to secondary APIs:
  • The "Foo" connector connects to the imaginary Foo API using a static key.
  • The "Bar" connector connects to the imaginary Foo API using a bearer token obtained using a OAuth Client Credentials request.
  • Both connectors support key rotation: Credentials are refreshed out of the chosen secret manager if/when the current credentials cease working, allowing for keys to be rotated without restarting the application. Key rotation on both connectors is throttled by a refresh cooldown parameter preventing bad credentials in the secret manager from being continually polled.
• Configuration is based on environment variables.
• File upload and tracking examples.
• Server-sent events over MQTT.
  • For more information, see docs/patterns/server-sent-events.md
• Documentation on common data handling practices:
  • Event Sourcing

Objects

This project provides several different examples of data storage techniques:

• ExampleItem stores data in a DynamoDB table.
• Order stores data in a MariaDB table, accessed via Dapper with implementations generated by Rosalyn source generation.
  • This system was designed to rapidly generate access clients for basic CRUD tables.
• Product stores data in a MariaDB table, accessed via Dapper.
  • This example is directly adapted from the source-generated code.
• Customer stores data in a MariaDB table, accessed via Entity Framework.

Uploads

This template provides a system for accepting, tracking, and validating file uploads.

For more information on the uploads system, see docs/patterns/uploads.md.

Configuration

Configuration environment variables for the upload system.

Setting	Environment variable
Max request body size (nullable)	UPLOADS__MAX_UPLOAD_SIZE_BYTES
Unverified bucket	UPLOADS__BUCKET_UNVERIFIED_ITEMS
Verified bucket	UPLOADS__BUCKET_VERIFIED_ITEMS
Presigned URL lifetime (minutes)	UPLOADS__PRESIGNED_URL_LIFETIME_MINUTES

Related: Schema

Database DDL for this API lives in a separate repository: RedShirt.Example.Schema.

That project owns MariaDB/MySQL schema versioning (using the DbUp library to apply incremental SQL scripts). This API assumes those tables already exist and does not create or migrate them. The intent of this dedicated schema was to enforce separation of concerns and prevent the API from having the power to affect the schema on a fundamental level. When developing against the local Compose stack, apply schema updates from the Schema repo before starting the API (see test/local/).

Initialisation

Below are the recommended steps for using this as a template:

1. To change the namespace of the API en-masse for your purposes, use the init-repo.sh script:

   bash init-repo.sh New.Namespace.Here

2. Write database accessors and/or service connectors based on your application's needs.

3. Prune database accessors and/or service connectors that your application does not need.

4. The example of Server-Sent Events needed a bespoke handler, the standard NSwag clients did not know how to support event streams. This handler can be found in the RedShirt.Example.Api.Interop project, defined in EventStreamListener.cs.

   • The EventStreamListener.cs contains the abstract EventStreamListener and the applied MessagesEventStreamListener, which sets some defaults.
   • If you are implementing Server-Sent Events but not as /messages as this example does, then I would recommend adjusting the comments described in MessagesEventStreamListener.
   • If you are not implementing Server-Side Events in your applied application, then I would recommend deleting EventStreamListener.cs outright.
     • The pattern of emulating the generated swagger clients by throwing a SwaggerException required the manual declaration of SwaggerException and ProblemDetails to avoid a chicken-and-egg problem on build where a cold build without generated code failed because the manual code was expecting it. If you are removing EventStreamListener, then you might also want to consider removing these manual classes and adjusting nswag.json to generate them again.

5. Consider revising/pruning the Markdown files such as this README or those located in the docs/ directory. They heavily assume that they are speaking for a general template and not for an applied application.

Secret Manager

Many components of this template rely on an implementation of the ISecretManagerService interface to function. Use of a secret manager is highly encouraged for values that could have sensitive information such as credentials to external services.

Currently, the template provides 3 possible implementations:

• AWS SSM Parameter Store
• Azure Key Vault
• Docker Secrets (Secondary Link)

Docker Secrets

While the other secret managers are more straightforward with their plans, Docker Secrets has some caveats and assumptions that should be documented.

In general, I would encourage the use of a secret manager other than Docker Secrets. Compared to other options it lacks flexibility. However, it may be exactly what is needed for a small-scale environment.

Other notes:

• Unlike other secret managers, a container instance's secrets cannot be rotated without restarting the
• Secrets are assumed to be files in /run/secrets
  • This directory can be overridden by setting a new path in COMMON__SECRETS__DOCKER__DIRECTORY
• Secret keys are assumed to be roughly equal to the underlying file specified in the Docker stack configuration. If the file does not meet these guidelines and because the compose file specifies another target path within the container, then the secret manager has nothing with which to resolve a key to a file containing a value. After checking the exact name of the key under the secret directory, the secret manager will attempt the path with a few file extensions (though realistically, only the flat key match will probably be useful).
  • For example, with no overriding directory the key foo-password will be searched for under the following absolute paths:
    • /run/secrets/foo-password
    • /run/secrets/foo-password.txt
    • /run/secrets/foo-password.json
  • In general, it's advised not to meddle with secret targets within the container at all if you plan to use them with this template's Docker secret manager.
• The implementation has not been tested under Docker Swarm, and as such hasn't been tested with external secrets.

Background Services

In addition to acting as an API, this application can also run background services.

To do so, you can do the following:

1. Declare class that inherits from BackgroundService

   public sealed class ExampleHostedService(IStuffDoerService stuffDoerService) : BackgroundService
   {
       protected override async Task ExecuteAsync(CancellationToken stoppingToken)
       {
           try
           {
               // Perform action. It's assumed that the implementation runs continually.
               await stuffDoerService.DoStuffAsync(stoppingToken);
           }
           catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
           {
           }
       }
   }

2. Add a hosted service when setting up your service collection:

   services.AddHostedService<ExampleHostedService>()

Doing this could be useful for:

• Periodic internal maintenance within the scope of the API instance.
• When there is a single instance of the API, it could be used for broader app maintenance such as periodically invoking CQRS handlers.
  • This case is an infrastructure shortcut to avoid the overhead of creating/maintaining a separate worker application for simple tasks. This option should not be explored or extracted out into a separate worker application the moment that the environment plan involves multiple instances of the API running in an environment.

This a standard feature of a hosted .NET application and not a special feature of this template, but I figured that it was worth documenting here.

Configuration

This API is expected to be run out of a docker container, so it relies on environment variables for most of its configuration.

For configuration examples, see the api section of the test/local/docker-compose.yaml file.

Path Base

When the API is hosted under a URL prefix (for example behind a reverse proxy at /example), set API__PATH_BASE to that prefix (for example /example) so that routing and link generation treat requests relative to that base. Leave it unset (or null, or blank) when the app is served at the site root.

Rate Limiting Configuration

This API is built with the option for rate limiting, using a sliding window system backed either by Redis or an in-memory system.

To make use of rate limiting, you must either:

• Use the [EnableRateLimiting("example")] attribute to name a policy (unlike the example on this list item, using constants for this is strongly encouraged).
• Set and configure a default policy to require rate limiting across all endpoints (unless ruled out by the use of the [DisableRateLimiting] attribute applied to an endpoint or controller).

Resources:

• For configuration examples, see the api section of the test/local/docker-compose.yaml file. Rate limiting is defined in environment variables beginning in RATE_LIMITING.
• To better understand the configuration definitions, refer to the classes in the Configuration/ folder of the Common.RateLimiting project

Development

Tips for local development.

Debugging Source Generation

See Debugging source generation.

Testing

For local testing, see the test/local folder. That guide covers bringing up MariaDB and applying schema updates via RedShirt.Example.Schema.