Add fast-redeploy scripts for container-based development (#10156)

[ErykKul]

Introduces three scripts for rapid iteration during Dataverse development:

• dev-start-frd.sh: One-time setup with exploded WAR deployment
• dev-frd.sh: Fast redeploy after code changes (~12s vs ~54s, 4.5x faster)
• dev-down-frd.sh: Clean shutdown of dev environment

Also adds docker-compose.override.yml to remove the 2GB memory limit which is insufficient for local development (set for GitHub Actions CI).

The workflow complements existing IDE-based hot reload options and provides a fast feedback loop for CLI-based development.

What this PR does / why we need it:

This PR introduces a fast-redeploy workflow for container-based Dataverse development, addressing the slow iteration cycle that developers face when working with containers. Currently, making a simple code change requires a full rebuild cycle (~54 seconds total: stopping containers, rebuilding with Maven, and restarting). This PR reduces that to ~12 seconds (4.5x faster) by:

1. Using an exploded WAR deployment that allows incremental compilation
2. Syncing only changed files (compiled classes and webapp resources) to the running container
3. Using Payara's asadmin deploy --force for hot redeployment without container restarts

The workflow preserves database state between iterations and requires zero infrastructure changes to the existing docker-compose setup. It's completely optional and complements existing IDE-based hot reload workflows.

Investigation: I have initially explored using Payara's .reload marker file mechanism, but discovered it only works for applications deployed to the autodeploy directory (/opt/payara/appserver/glassfish/domains/domain1/autodeploy/), not for applications deployed to custom paths like /opt/payara/deployments/. The asadmin deploy --force approach proved more reliable for our bind-mounted exploded WAR architecture.

Which issue(s) this PR closes:

• Closes add Visual Studio Code to guides #10156 (extends scope from VS Code-only documentation to a universal CLI-based fast-redeploy workflow)

Special notes for your reviewer:

1. Memory Override File: docker-compose.override.yml is committed intentionally. The 2GB limit from GitHub Actions CI is insufficient for any local development machine. Docker Compose automatically loads this file when present, but it won't interfere with other workflows that use explicit -f flags.

2. DB Initialization Detection: dev-start-frd.sh uses a clever permission-based check to detect if PostgreSQL has already initialized the database. It tests whether ls docker-dev-volumes/postgresql/data fails (permission denied means postgres user owns it, meaning it's initialized). This allows conditional JPA DDL generation without requiring sudo.

3. Conditional DDL Generation: The script only modifies persistence.xml to disable table creation (ddl-generation=none) when the database is already initialized. First-time bootstrap keeps create-tables so EclipseLink creates the schema.

4. Performance Numbers: The ~12s figure is conservative (actual testing averaged ~12.9s). The ~54s baseline includes the complete traditional workflow: docker compose down (11.4s) + mvn -Pct clean package (17.7s) + docker compose up (~24s).

5. Hardware Disclaimer: Both the documentation and release notes include disclaimers that performance varies by hardware, so developers should use these as relative comparisons, not absolute guarantees.

Suggestions on how to test this:

1. Fresh Bootstrap Test:

   cd /home/eryk/projects/dataverse
   git checkout feature/fast-redeploy-scripts
   
   # Clean slate (if you have existing volumes)
   rm -rf docker-dev-volumes/
   
   # Run initial setup
   ./scripts/dev/dev-start-frd.sh
   # Should complete successfully with containers running and app deployed
   # Database should be initialized with schema

2. Fast Redeploy Test:

   # Make a simple code change
   # e.g., modify src/main/java/edu/harvard/iq/dataverse/api/Info.java
   # Change a return value in the version() method
   
   # Run fast redeploy
   time ./scripts/dev/dev-frd.sh
   # Should complete in ~12-15 seconds
   
   # Verify change is live
   curl http://localhost:8080/api/info/version

3. Multiple Iteration Test:

   # Make several code changes and redeploy each time
   # Verify performance remains consistent (~12s per iteration)
   ./scripts/dev/dev-frd.sh
   # ... make another change ...
   ./scripts/dev/dev-frd.sh
   # ... make another change ...
   ./scripts/dev/dev-frd.sh

4. Clean Shutdown Test:

   # Stop the environment cleanly
   ./scripts/dev/dev-down-frd.sh
   # Should stop all containers gracefully
   
   # Verify containers are stopped
   docker ps | grep dataverse
   # (should show no running containers)

5. Restart After Shutdown Test:

   # After running dev-down-frd.sh, try restarting
   ./scripts/dev/dev-start-frd.sh
   # Should detect existing DB and skip DDL generation
   # Should preserve all data from previous session

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

No user interface changes. This is a developer workflow enhancement only.

Is there a release notes update needed for this change?:

Yes, included in this PR: doc/release-notes/10156-fast-redeploy-scripts.md

The release note documents:

• All three scripts and their purposes
• The docker-compose.override.yml file
• Performance metrics (4.5x faster)
• Complete typical workflow
• Hardware performance disclaimer

Additional documentation:

Yes, comprehensive documentation added to the Container Guide:

• New "Fast Redeploy (Command-Line)" section in doc/sphinx-guides/source/container/dev-usage.rst
• Positioned after the "Redeploying" section
• Includes:
  • Initial setup instructions
  • Iterative workflow
  • Shutdown procedure
  • Memory configuration explanation
  • Complete workflow example
  • Limitations and caveats
  • Hardware performance disclaimer

[Copilot]

Pull Request Overview

This PR introduces fast redeploy scripts for container-based development, providing a command-line workflow that achieves ~12 second redeployment cycles (4.5x faster than the traditional 54-second full rebuild). The approach uses exploded WAR deployment with incremental compilation.

• New shell scripts for fast iterative development workflow without full container rebuilds
• Docker Compose override file to remove memory constraints during local development
• Comprehensive documentation with usage instructions and limitations

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
scripts/dev/dev-start-frd.sh	Initial setup script that builds/extracts WAR, detects database state, and deploys to containerized Payara
scripts/dev/dev-frd.sh	Fast redeploy script that incrementally compiles and syncs changes to running container
scripts/dev/dev-down-frd.sh	Cleanup script to stop and remove dev containers
docker-compose.override.yml	Removes 2GB memory limit for local development environment
doc/sphinx-guides/source/container/dev-usage.rst	Documentation for the fast redeploy workflow
doc/release-notes/10156-fast-redeploy-scripts.md	Release notes describing the new feature

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ErykKul]

I tested it on my slow laptop, fast redeploy took 42 seconds, extra 30 seconds from the 12s I mentioned. I did try touching .reload i.s.o. asadmin redeploy, it did not work on the base image, but with other image the timings were similar, so I did not try to make it work on the base image. I think that 42 seconds is still fine, but maybe it could be better.

[jp-tosca]

@pdurbin you may want to look at this!

[pdurbin]

Yes! I didn't realize @ErykKul had made this PR! I plan to update Skipper ( https://github.com/pdurbin/skipper ) so that I stop skipping @ErykKul!

[poikilotherm]

I did a quick shot to see if we could simplify here by using the official Payara Server Maven Plugin. Apparently, it's broken, at least for now.

[poikilotherm]

I'm still curious if we can transform this into something where Maven uses a watcher plugin and executes a script in the container for us. That way we could make the scripts now outside of the container part of the base image and reuse from there. A lot less terminal handling involved that way. I may play around with this some more...

[pdurbin]

Works great! Thanks, @ErykKul! ❤️ Merging!