chore: Add linux support and containerize the project by clouths · Pull Request #5 · OpenPerpetuum/PerpetuumServer2

clouths · 2026-03-15T20:35:00Z

Linux Support, Containerization & CI Automation

Overview

Enables full Linux cross-platform support, containerizes the entire Perpetuum server stack with Docker Compose, adds performance optimizations, developer tooling, and automated CI test pipelines.

Key Changes

1. Linux Compatibility & Modernization

SkiaSharp Migration: Replaced Windows-only System.Drawing.Common with SkiaSharp across terrain, bitmap, color conversion, and pathfinding modules.
TransactionScope Fix: Added DbConnectionManager to share database connections within transaction scopes, preventing MSDTC promotion errors on Linux.
Container Stack: Added complete Docker Compose configuration (compose.yml, Dockerfiles for server, migration, db, asset, and test runner) with Linux sleep watchdog to prevent SQL Server scheduler spin.

2. Performance Optimizations

SIMD Acceleration: Added SimdMath with vector intrinsics for spatial calculations and distance evaluations.
Terrain & Memory Improvements: Introduced CompactPassabilityMask and HeightfieldMetadata to reduce memory footprints.
Memory Management: Configured .NET 8 DATAS dynamic GC heap scaling.

3. Developer Tooling & CLI Client

CLI Client (Perpetuum.CliClient): Interactive console client supporting server handshake, authentication, chat channels, and character management.
Automation Scripts: Added create-account.sh, smoke-test.sh, check_assets.py, and cli.sh.
Database Migrations: Automated dynamic patch discovery and snapshot hash caching (perpetuumsa_migrated.bak) for near-instant container boots.

4. Testing & CI Pipeline

Workflows (.github/workflows/docker-tests.yml): GitHub Actions jobs for containerized unit and integration testing.
Data Setup Action (.github/actions/setup-test-data/action.yml): Automated SteamCMD server data (App 693060) and Google Drive gamma layer downloads with caching.
Test Suites:
- Unit Tests: Coverage for SkiaSharp image operations, SIMD vector math, CLI client logic, passability masks, and terrain layers.
- Integration Tests: Live database validation, content definitions, and terrain invariant tests.
- Smoke Tests: End-to-end container verification, TCP connection, authentication, and packet exchanges.

How to test

Run unit tests

make test-unit

Start the local stack (docker compose)
⚠️ Note that you need to download custom assets to launch the server. Open readme in docker compose section to get started ⚠️

make up

Run integration tests

make test-integration

…r linux, add template for perpetuum.ini, add Makefile for some helper commands, update server to take DistributedTransactions as argument to be able to disable it for linux

…date server image to include GDI+

aqpanaciy · 2026-03-16T12:04:00Z

Regarding using DistributedTransactions, we use a single database and the connection string doesn't change. Therefore, using DistributedTransactions is not necessary. I'm not very knowledgeable about the intricacies of database operation, but after reading the documentation, I found out that:

After closing the connection to the database, it is not closed immediately, but is returned to the pool, and when reopened, it is taken from there.
For each request inside a TransactionScope, you can open and close your own connection; this is a well-known technique.

Therefore, you can remove: TransactionManager.ImplicitDistributedTransactions = true; in PerpetuumBootstrapper.cs
AND
Rremove: dbConnection.EnlistTransaction(Transaction.Current); in DbQuery.cs (This line of code is responsible for forcing the use of distributed transactions for the current transaction)

With these changes, I was able to launch the server on Linux. And there are no errors when working with the database. I hope this information will at least be useful.

…ipt and fix migration when using a directory

clouths · 2026-03-16T23:10:19Z

Regarding using DistributedTransactions, we use a single database and the connection string doesn't change. Therefore, using DistributedTransactions is not necessary. I'm not very knowledgeable about the intricacies of database operation, but after reading the documentation, I found out that:
1. After closing the connection to the database, it is not closed immediately, but is returned to the pool, and when reopened, it is taken from there.

2. For each request inside a TransactionScope, you can open and close your own connection; this is a well-known technique.
Therefore, you can remove: TransactionManager.ImplicitDistributedTransactions = true; in PerpetuumBootstrapper.cs AND Rremove: dbConnection.EnlistTransaction(Transaction.Current); in DbQuery.cs (This line of code is responsible for forcing the use of distributed transactions for the current transaction)

With these changes, I was able to launch the server on Linux. And there are no errors when working with the database. I hope this information will at least be useful.

This PR disables it but I have left a default value (true) to keep the same behavior to keep compatibility for the Windows build. If removing the EnlistTransaction do the trick, we might be able to disable it if the live server using the Windows build don't have any issue.

…kept

…creen issue on client when trying to enter a zone. Now the client seems to work (feature validation is in progress). Disabled EnlistTransaction to ensure DistributedTransactions are disabled

…rd to the host.

clouths · 2026-03-24T02:59:23Z

At this point, the PR is almost ready for review.

The client is able to connect and perform all actions tested (chat, enter/leave chat, buy from market, unpack item, activate bot, undock/dock, move, assignments, looting, combat, tutorial, rift)

What is left todo?

Add readme how how to setup development environment (server, assets, gamma layers) (can be done in a follow-up PR)

… the EnlistTransaction configurable with the DistributedTransactions state

clouths · 2026-03-28T01:42:06Z

At this point, the documentation has been added and the DistributedTransactions config is now correctly applied in the codebase.

…d of SKSurface, update usage of canvas.DrawText to fix Y position by taking into account the font size since the origin is on top left instead of bottom left.

…, update comment

…e zone size.

…nto linux+containerize

aqpanaciy · 2026-08-30T18:00:05Z

I successfully tested this branch in Docker.Desktop on Windows. I did have to manually update the submodules to the latest versions, though.

…mpose architecture, fix FakeDb missing {, update submodules for asset and db

clouths · 2026-08-31T01:26:39Z

All unit and integration tests pass now.

Added Live_36 to the migration with the Raw_SQL for now until the patch is complete.

…nto linux+containerize

…R for this CI action This reverts commit 87cd9d8.

- Add SHA-256 change detection and compressed DB backup restoration - Automate chronological execution for Pre_Alpha_* and Live_* patches - Add clean-cache Makefile target and README Mermaid migration diagram

…tling - Add SimdMath with AVX-512 / AVX2 / Vector128 distance calculation fallbacks - Add HeightfieldMetadata 16x16 chunk bounding grid for raycast optimization - Add CompactPassabilityMask 1-bit per tile bitmask - Throttle empty zone update loop to 1 Hz when no players are present - Enable DOTNET_GCDynamicAdaptationMode for dynamic GC heap sizing - Add unit test coverage for SIMD math and terrain structures

…promotion on Linux - Introduce DbConnectionManager to track and reuse open connections per ambient Transaction - Update DbQuery.ExecuteHelper to avoid multiple connection checkouts in the same scope - Fix System.PlatformNotSupportedException thrown on Linux during player logout and transactions - Add unit tests verifying connection reuse, cleanup on commit, and cleanup on abort

clouths · 2026-09-04T02:36:51Z

I've reworked the migration to load files automatically without having to add new patch in the migration.sh file with a cache mechanism to accelerate migration (development QoL).

I also had some instances of Distributed transaction happening due to Enlist some times which was causing errors, to fix this issue, I refactored the DbConnectionManager to avoid multiple connections in the same scope to reuse them. When distributed transactions is enabled, the behavior is unchanged. This refactor will allow in theory better scaling with many players/npcs.

I also added SIMD acceleration, did some optimization on the loaded assets by caching the essential content. I also added idle optimization to slow down tick when no player is on an island from from 20hz to 1hz while keeping precision on events on the island. Overall saving (40-60% reduction) 1.5-2.5GB on RAM and (60-90% reduction) 0.6-0.9 CPU core. One notable change is that a zone metadata can now fit in L2 cache (from 36MB to 512KB per zone).

…00% on linux

…nto linux+containerize

- Implement Perpetuum.CliClient interactive console tool and tests - Add smoke-test.sh, check_assets.py, and create-account.sh scripts - Fix terrain layer bounds clamping and coordinate validation - Add COVERAGE.md and update test documentation

…nto linux+containerize

aqpanaciy · 2026-09-22T09:12:40Z

+            // Throttle unit physics, AI, and visibility processing when no players are in the zone
+            if (_players.IsEmpty)
+            {
+                _idleUpdateTimer.Update(time);
+                if (!_idleUpdateTimer.Passed)
+                {
+                    RiftManager?.Update(time);
+                    RelicManager?.Update(time);
+                    MiningLogHandler.Update(time);
+                    HarvestLogHandler.Update(time);
+                    return;
+                }
+
+                _idleUpdateTimer.Reset();
+            }
+


From my local testing, NPC-vs-NPC mechanics have slowed down. It appears to disrupt all processes related to time accumulation. The PBS energy consumption system, PBSActiveObject functionality (including periodic actions and boost states), periodic saving of PBS objects, and object expiration/deletion timers cease to function correctly. All of these rely on UpdateUnits(time).

Suggested change

// Throttle unit physics, AI, and visibility processing when no players are in the zone

if (_players.IsEmpty)

{

_idleUpdateTimer.Update(time);

if (!_idleUpdateTimer.Passed)

{

RiftManager?.Update(time);

RelicManager?.Update(time);

MiningLogHandler.Update(time);

HarvestLogHandler.Update(time);

return;

}

_idleUpdateTimer.Reset();

}

var elapsed = _idleUpdateTimer.Update(time).Elapsed;

if (_players.IsEmpty && !_idleUpdateTimer.Passed)

{

return;

}

_idleUpdateTimer.Reset();

time = elapsed;

aqpanaciy · 2026-09-22T09:12:46Z

        }

        private readonly ShiftedConsumerTimer _updateUnitsTimer = new ShiftedConsumerTimer(500);
+        private readonly IntervalTimer _idleUpdateTimer = new IntervalTimer(1000);


250 ms is already five times the standard interval. I think 1 second (20 times longer) is excessive.

Suggested change

private readonly IntervalTimer _idleUpdateTimer = new IntervalTimer(1000);

private readonly IntervalTimer _idleUpdateTimer = new IntervalTimer(250);

aqpanaciy · 2026-09-22T13:46:58Z

Did I understand correctly that this optimization isn't used on the server itself? Plants growth needs to be taken into account; they dynamically change their height and whether or not they act as a solid block.

aqpanaciy · 2026-09-22T14:34:31Z

Yes, the change solves the issue on Linux. However, the tests do not cover all possible use cases—specifically, the tests for DistributedTransactions = true. We need to ensure that this change does not cause a regression on Windows.

clouths added 2 commits March 15, 2026 15:08

Add submodules for OPResource + OPDB

6e48f0d

Add container support using linux containers, add migration script fo…

5af7381

…r linux, add template for perpetuum.ini, add Makefile for some helper commands, update server to take DistributedTransactions as argument to be able to disable it for linux

clouths self-assigned this Mar 15, 2026

Update mssql-tools to a more recent version (18) to fix migration, up…

26512e6

…date server image to include GDI+

Fix comment in .env.local, remove preparePatch from the migration scr…

0689ed4

…ipt and fix migration when using a directory

clouths added 7 commits March 16, 2026 20:03

Move db persistent storage into a docker volume

1073ced

Most fo the refactor from System.Drawing to SkiaSharp is done.

0fa79c6

Fix migration order of copy to ensure the generated perpetuum.ini is …

decdcfd

…kept

Add DefinitionConfig refactor

67f8314

Update db submodule + migration script to use P34 and P35 full sql files

b31e821

Update port mapping to allow more ports for the server to fix black s…

1eba63b

…creen issue on client when trying to enter a zone. Now the client seems to work (feature validation is in progress). Disabled EnlistTransaction to ensure DistributedTransactions are disabled

Fix port mapping, defined extra SERVER_PORTS for port range to forewa…

984ed0a

…rd to the host.

Disable EnlistTransactaction

0bf693c

clouths marked this pull request as ready for review March 25, 2026 01:31

Add documentation in the readme for how to setup the containers, make…

c19dedf

… the EnlistTransaction configurable with the DistributedTransactions state

Update db image to use mssql-2025

17c38fa

aqpanaciy reviewed Mar 28, 2026

View reviewed changes

Comment thread src/Perpetuum/BitmapExtensions.cs Outdated

aqpanaciy reviewed Mar 28, 2026

View reviewed changes

Comment thread src/Perpetuum.RequestHandlers/Zone/StatsMapDrawing/ZoneDrawStatMap.cs Outdated

clouths added 2 commits March 28, 2026 18:37

Fix WithCanvas extension to preserve the Bitmap using SKCanvas instea…

afe07ea

…d of SKSurface, update usage of canvas.DrawText to fix Y position by taking into account the font size since the origin is on top left instead of bottom left.

Fix font.size to use Size property

cba0e76

clouths changed the title ~~WIP chore: Add linux support and containerize the project~~ chore: Add linux support and containerize the project Mar 28, 2026

aqpanaciy reviewed Mar 29, 2026

View reviewed changes

Comment thread src/Perpetuum.RequestHandlers/Zone/ZoneSetLayerWithBitMap.cs Outdated

clouths added 4 commits March 29, 2026 11:49

Fix pixel check to use Alpha instead of the Luminance

8a40610

Fix typo, replace usage of manual creation of Area with size.ToArea()…

005b8ce

…, update comment

Fix PresenceConfiguration to use the Configuration size instead of th…

14a1e65

…e zone size.

Merge develop branch into linux+containerize

c1a425d

clouths added 2 commits August 29, 2026 23:32

Merge branch 'develop' of github.com:OpenPerpetuum/PerpetuumServer2 i…

a10dcd8

…nto linux+containerize

Try fix merge of FakeDb.cs causing ci build error

3adb6bf

aqpanaciy reviewed Aug 30, 2026

View reviewed changes

Comment thread src/Perpetuum.Tests/Fakes/Data/FakeDb.cs

clouths added 2 commits August 30, 2026 16:31

Add make commands to run tests (unit+integration) using the docker co…

d8c5168

…mpose architecture, fix FakeDb missing {, update submodules for asset and db

Add Live_36 to migration, all integration tests passes now

0af068c

Add unit and integration tests

8256cf6

clouths requested a review from aqpanaciy August 31, 2026 02:33

aqpanaciy reviewed Aug 31, 2026

View reviewed changes

Comment thread docs/codebase/TESTING.md

clouths added 8 commits September 1, 2026 20:21

Merge branch 'develop' of github.com:OpenPerpetuum/PerpetuumServer2 i…

a75e424

…nto linux+containerize

Update db submodule

4953b3d

Add CI to run tests with docker

87cd9d8

Revert "Add CI to run tests with docker" for now, I'll create a new P…

2d884d9

…R for this CI action This reverts commit 87cd9d8.

perf(migration): automate patch discovery and add snapshot caching

c9d5ec0

- Add SHA-256 change detection and compressed DB backup restoration - Automate chronological execution for Pre_Alpha_* and Live_* patches - Add clean-cache Makefile target and README Mermaid migration diagram

Update db submodule

bb0d416

clouths added 5 commits September 5, 2026 12:03

Fix an issue with SQL server when waking up from sleep using 1-core@1…

b8911a5

…00% on linux

Merge branch 'develop' of github.com:OpenPerpetuum/PerpetuumServer2 i…

54346f3

…nto linux+containerize

Merge branch 'develop' of github.com:OpenPerpetuum/PerpetuumServer2 i…

7c495b5

…nto linux+containerize

Add CI for unit and integration tests

94ab729

clouths mentioned this pull request Sep 22, 2026

[test] Testing docker-ci-workflow #61

Closed

clouths requested a review from aqpanaciy September 22, 2026 05:16

aqpanaciy reviewed Sep 22, 2026

View reviewed changes

Navigation Menu

chore: Add linux support and containerize the project - #5

chore: Add linux support and containerize the project#5
clouths wants to merge 38 commits into
developfrom
linux+containerize

clouths commented Mar 15, 2026 •

edited

Loading

Uh oh!

aqpanaciy commented Mar 16, 2026

Uh oh!

clouths commented Mar 16, 2026

Uh oh!

clouths commented Mar 24, 2026 •

edited

Loading

Uh oh!

clouths commented Mar 28, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aqpanaciy commented Aug 30, 2026

Uh oh!

Uh oh!

clouths commented Aug 31, 2026

Uh oh!

Uh oh!

clouths commented Sep 4, 2026

Uh oh!

aqpanaciy Sep 22, 2026

Uh oh!

aqpanaciy Sep 22, 2026

Uh oh!

aqpanaciy Sep 22, 2026

Uh oh!

aqpanaciy Sep 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	private readonly IntervalTimer _idleUpdateTimer = new IntervalTimer(1000);
	private readonly IntervalTimer _idleUpdateTimer = new IntervalTimer(250);

Conversation

clouths commented Mar 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Linux Support, Containerization & CI Automation

Overview

Key Changes

1. Linux Compatibility & Modernization

2. Performance Optimizations

3. Developer Tooling & CLI Client

4. Testing & CI Pipeline

How to test

Uh oh!

aqpanaciy commented Mar 16, 2026

Uh oh!

clouths commented Mar 16, 2026

Uh oh!

clouths commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clouths commented Mar 28, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aqpanaciy commented Aug 30, 2026

Uh oh!

Uh oh!

clouths commented Aug 31, 2026

Uh oh!

Uh oh!

clouths commented Sep 4, 2026

Uh oh!

aqpanaciy Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

aqpanaciy Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

aqpanaciy Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

aqpanaciy Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

clouths commented Mar 15, 2026 •

edited

Loading

clouths commented Mar 24, 2026 •

edited

Loading