diff --git a/CHANGELOG.md b/CHANGELOG.md index 6a07d51..8933631 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,12 @@ - Run Litestream commands without a shell and raise on command failures and timeouts. - Support parsed JSON output from Litestream 0.5 commands with `json: true`. +- Bundle Litestream 0.5.17 release archives and verify their SHA-256 checksums during packaging. +- Replace the removed `generations`, `snapshots`, and `wal` wrappers and rake tasks with `ltx` and `status`. +- Use daemon-free `databases`, `status`, and `ltx` calls for dashboard data. +- Show local status and LTX files in the dashboard. +- Generate Litestream 0.5 configuration with a single replica and global snapshot settings. +- Document restoration and migration from Litestream 0.3 backups. ## [0.14.0] - 2025-06-14 diff --git a/Gemfile.lock b/Gemfile.lock index 51f11ba..d3a5def 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -204,7 +204,6 @@ GEM rubocop (>= 1.48.1, < 2.0) rubocop-ast (>= 1.30.0, < 2.0) ruby-progressbar (1.13.0) - rubyzip (2.3.2) securerandom (0.4.1) sqlite3 (2.6.0-arm64-darwin) sqlite3 (2.6.0-x86_64-linux-gnu) @@ -244,7 +243,6 @@ DEPENDENCIES minitest (~> 5.0) rails rake (~> 13.0) - rubyzip standard (~> 1.3) BUNDLED WITH diff --git a/README.md b/README.md index c9173e1..0ce676a 100644 --- a/README.md +++ b/README.md @@ -85,16 +85,22 @@ The gem streamlines the configuration process by providing a default configurati The default configuration file looks like this if you only have one SQLite database: ```yaml +snapshot: + interval: 24h + retention: 24h + dbs: - path: storage/production.sqlite3 - replicas: - - type: s3 - path: storage/production.sqlite3 - bucket: $LITESTREAM_REPLICA_BUCKET - access-key-id: $LITESTREAM_ACCESS_KEY_ID - secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY + replica: + type: s3 + path: storage/production.sqlite3 + bucket: $LITESTREAM_REPLICA_BUCKET + access-key-id: $LITESTREAM_ACCESS_KEY_ID + secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY ``` +In Litestream 0.5, snapshot interval and retention are global settings rather than per-replica settings. + This is the default for Amazon S3. The full range of possible replica types (e.g. other S3-compatible object storage servers) are covered in Litestream's [replica guides](https://litestream.io/guides/#replica-guides). The gem also provides a default initializer file at `config/initializers/litestream.rb` that allows you to configure various variables referenced in the configuration file in Ruby. By providing a Ruby interface to these environment variables, you can use your preferred method of storing secrets. For example, the default generated file uses Rails' encrypted credentials to store your secrets. @@ -179,7 +185,7 @@ You can restore any replicated database at any point using the gem's provided `l > [!NOTE] > During the restoration process, you need to prevent any interaction with ActiveRecord/SQLite, such as from a running `rails server` or `rails console` instance. If there is any interaction, Rails might regenerate the production database and prevent restoration via litestream. If this happens, you might get a "cannot restore, output path already exists" error. -1. Rename the production (`production.sqlite3`, `production.sqlite3-shm`, and `production.sqlite3-wal`) databases (**recommended**) or alternatively delete. To delete the production databases locally, you can run the following at your own risk: +1. Rename the production database and its SQLite sidecar files (**recommended**) or alternatively delete them. To delete the production databases locally, you can run the following at your own risk: ```shell # DANGEROUS OPERATION, consider renaming database files instead bin/rails db:drop DISABLE_DATABASE_ENVIRONMENT_CHECK=1 @@ -215,26 +221,19 @@ You can forward arguments in whatever order you like, you simply need to ensure -if-replica-exists Returns exit code of 0 if no backups found. --parallelism NUM - Determines the number of WAL files downloaded in parallel. - Defaults to 8 - --replica NAME - Restore from a specific replica. - Defaults to replica with latest data. - --generation NAME - Restore from a specific generation. - Defaults to generation with latest data. - --index NUM - Restore up to a specific WAL index (inclusive). - Defaults to use the highest available index. +-txid TXID + Restore through a specific 16-character hexadecimal transaction ID. -timestamp TIMESTAMP Restore to a specific point-in-time. Defaults to use the latest available backup. +-json + Print the restore result as JSON. + +-dry-run + Print the restore plan without writing the database. + -config PATH Specifies the configuration file. Defaults to /etc/litestream.yml @@ -387,81 +386,59 @@ bin/rails litestream:databases This will return a list of databases and their configured replicas: ``` -path replicas +path replica /Users/you/Code/your-app/storage/production.sqlite3 s3 ``` -You can also list the generations of a specific database: - -```shell -bin/rails litestream:generations -- --database=storage/production.sqlite3 -``` - -This will list all generations for the specified database, including stats about their lag behind the primary database and the time range they cover: - -``` -name generation lag start end -s3 a295b16a796689f3 -156ms 2024-04-17T00:01:19Z 2024-04-17T00:01:19Z -``` - -You can list the snapshots available for a database: +You can inspect local replication status for all databases or one specific database: ```shell -bin/rails litestream:snapshots -- --database=storage/production.sqlite3 +bin/rails litestream:status -- --database=storage/production.sqlite3 ``` -This command lists snapshots available for that specified database: +This reads local state and does not require a running Litestream process: ``` -replica generation index size created -s3 a295b16a796689f3 1 4645465 2024-04-17T00:01:19Z +database status local_txid wal_size +/Users/you/Code/your-app/storage/production.sqlite3 ok 000000000000000a 128 kB ``` -Finally, you can list the wal files available for a database: +You can list all remote LTX files, including the level-9 snapshot: ```shell -bin/rails litestream:wal -- --database=storage/production.sqlite3 +bin/rails litestream:ltx -- --database=storage/production.sqlite3 --level=all ``` -This command lists wal files available for that specified database: +The command returns the compaction level, transaction range, byte size, and creation time: ``` -replica generation index offset size created -s3 a295b16a796689f3 1 0 2036 2024-04-17T00:01:19Z +level min_txid max_txid size created +0 0000000000000008 000000000000000a 1013 2026-09-08T03:16:43Z ``` ### Running commands from Ruby In addition to the provided rake tasks, you can also run Litestream commands directly from Ruby. The gem provides a `Litestream::Commands` module that wraps the Litestream CLI commands. This is particularly useful for the introspection commands, as you can use the output in your Ruby code. -Commands raise `Litestream::Commands::CommandFailedException` when the Litestream binary exits non-zero, and the exception message includes stderr. Pass `json: true` to receive parsed JSON when using Litestream 0.5 or newer (Litestream 0.3 does not support the `-json` flag), and use `timeout:` to limit how many seconds a command may run. - -The `Litestream::Commands.databases` method returns an array of hashes with the "path" and "replicas" keys for each database: - -```ruby -Litestream::Commands.databases -# => [{"path"=>"/Users/you/Code/your-app/storage/production.sqlite3", "replicas"=>"s3"}] -``` - -The `Litestream::Commands.generations` method returns an array of hashes with the "name", "generation", "lag", "start", and "end" keys for each generation: +Pass `json: true` to return the 0.5 CLI's JSON values with string keys. The `Litestream::Commands.databases` method returns each database path and replica type: ```ruby -Litestream::Commands.generations('storage/production.sqlite3') -# => [{"name"=>"s3", "generation"=>"5f4341bc3d22d615", "lag"=>"3s", "start"=>"2024-04-17T19:48:09Z", "end"=>"2024-04-17T19:48:09Z"}] +Litestream::Commands.databases(json: true) +# => [{"path"=>"/Users/you/Code/your-app/storage/production.sqlite3", "replica"=>"s3"}] ``` -The `Litestream::Commands.snapshots` method returns an array of hashes with the "replica", "generation", "index", "size", and "created" keys for each snapshot: +The `status` method returns local state without connecting to the replica or requiring a daemon: ```ruby -Litestream::Commands.snapshots('storage/production.sqlite3') -# => [{"replica"=>"s3", "generation"=>"5f4341bc3d22d615", "index"=>"0", "size"=>"4645465", "created"=>"2024-04-17T19:48:09Z"}] +Litestream::Commands.status("storage/production.sqlite3", json: true) +# => [{"database"=>"storage/production.sqlite3", "status"=>"ok", "local_txid"=>"000000000000000a", "wal_size"=>"128 kB"}] ``` -The `Litestream::Commands.wal` method returns an array of hashes with the "replica", "generation", "index", "offset","size", and "created" keys for each wal: +The `ltx` method lists remote LTX files. An unreplicated database returns an empty array: ```ruby -Litestream::Commands.wal('storage/production.sqlite3') -# => [{"replica"=>"s3", "generation"=>"5f4341bc3d22d615", "index"=>"0", "offset"=>"0", "size"=>"2036", "created"=>"2024-04-17T19:48:09Z"}] +Litestream::Commands.ltx("storage/production.sqlite3", json: true, "--level" => "all") +# => [{"level"=>0, "min_txid"=>"0000000000000008", "max_txid"=>"000000000000000a", "size"=>1013, "timestamp"=>"2026-09-08T03:16:43Z"}] ``` You can also restore a database programmatically using the `Litestream::Commands.restore` method, which returns the path to the restored database: @@ -481,14 +458,29 @@ The full set of commands available to the `litestream` executable are covered in ```shell litestream databases [arguments] -litestream generations [arguments] DB_PATH|REPLICA_URL +litestream info [arguments] +litestream list [arguments] +litestream ltx [arguments] DB_PATH +litestream register [arguments] litestream replicate [arguments] -litestream restore [arguments] DB_PATH|REPLICA_URL -litestream snapshots [arguments] DB_PATH|REPLICA_URL +litestream reset [arguments] +litestream restore [arguments] DB_PATH +litestream start [arguments] +litestream status [arguments] [DB_PATH] +litestream stop [arguments] +litestream sync [arguments] +litestream unregister [arguments] litestream version -litestream wal [arguments] DB_PATH|REPLICA_URL ``` +### Upgrading from 0.3 + +Litestream 0.5 changes each database from a `replicas:` list to a single `replica:` map and moves retention and snapshot settings into the root-level `snapshot:` block. Existing `replicas:` configuration files must be edited by hand before upgrading. The `databases` command and Ruby method now return a `"replica"` key instead of `"replicas"`. + +The removed `generations`, `snapshots`, and `wal` command methods and rake tasks now raise an error pointing to `ltx`, which replaces all three forms of remote backup introspection. + +The 0.5 `restore` command auto-detects both 0.3 and LTX backups and prefers whichever backup is newer. Existing 0.3 `generations/` objects are left in place; you can delete them after the LTX history covers your full retention window. To roll back, reinstall a 0.14.x release of this gem, which bundles Litestream 0.3. + ### Using in development By default, if you install the gem and configure via `puma.rb` or `Procfile`, Litestream will not start in development. diff --git a/app/views/litestream/processes/show.html.erb b/app/views/litestream/processes/show.html.erb index d8c8e31..e9cc0f0 100644 --- a/app/views/litestream/processes/show.html.erb +++ b/app/views/litestream/processes/show.html.erb @@ -56,63 +56,51 @@
-
- <% database['generations'].each do |generation| %> -
- - <%= generation['generation'] %> - (<%= generation['lag'] %> lag) - +
+ <% if database['error'] %> +

<%= database['error'] %>

+ <% else %> +
+
Status
+
<%= database.dig('status', 'status') %>
+
Local TXID
+
<%= database.dig('status', 'local_txid') %>
+
WAL size
+
<%= number_to_human_size database.dig('status', 'wal_size') %>
+
-
-
Start
-
- - - -
+
+ <% if database['ltx'] == [] %> +

No LTX files yet

+ <% else %> + + + + + + + + + + -
End
-
- - - -
- -
-
Snapshots
-
-
LevelMin TXIDMax TXIDSizeCreated
- - - - - - - - - - <% generation['snapshots'].each do |snapshot| %> - - - - - - <% end %> - -
Created atIndexSize
- - - - - <%= snapshot['index'] %> - - <%= number_to_human_size snapshot['size'] %> -
- - -
-
+ + <% database['ltx'].each do |ltx| %> + + <%= ltx['level'] %> + <%= ltx['min_txid'] %> + <%= ltx['max_txid'] %> + <%= number_to_human_size ltx['size'] %> + + + + + + + <% end %> + + + <% end %> <% end %>
diff --git a/lib/litestream.rb b/lib/litestream.rb index 159faa9..075d2d3 100644 --- a/lib/litestream.rb +++ b/lib/litestream.rb @@ -107,20 +107,18 @@ def replicate_process end def databases - databases = Commands.databases + databases = Commands.databases(json: true) databases.each do |db| - generations = Commands.generations(db["path"]) - snapshots = Commands.snapshots(db["path"]) - db["path"] = db["path"].gsub(Rails.root.to_s, "[ROOT]") - - db["generations"] = generations.map do |generation| - id = generation["generation"] - replica = generation["name"] - generation["snapshots"] = snapshots.select { |snapshot| snapshot["generation"] == id && snapshot["replica"] == replica } - .map { |s| s.slice("index", "size", "created") } - generation.slice("generation", "name", "lag", "start", "end", "snapshots") + path = db["path"] + begin + db["status"] = Commands.status(path, json: true).first + db["ltx"] = Commands.ltx(path, :json => true, "--level" => "all") + rescue Commands::CommandFailedException => error + db["error"] = error.message end + + db["path"] = path.gsub(Rails.root.to_s, "[ROOT]") end end diff --git a/lib/litestream/commands.rb b/lib/litestream/commands.rb index 780719b..4bbcb6c 100644 --- a/lib/litestream/commands.rb +++ b/lib/litestream/commands.rb @@ -124,22 +124,30 @@ def databases(**argv) execute("databases", argv) end - def generations(database, **argv) - raise DatabaseRequiredException, "database argument is required for generations command, e.g. litestream:generations -- --database=path/to/database.sqlite" if database.nil? + def ltx(database, **argv) + raise DatabaseRequiredException, "database argument is required for ltx command, e.g. litestream:ltx -- --database=path/to/database.sqlite" if database.nil? - execute("generations", argv, database) + execute("ltx", argv, database) end - def snapshots(database, **argv) - raise DatabaseRequiredException, "database argument is required for snapshots command, e.g. litestream:snapshots -- --database=path/to/database.sqlite" if database.nil? + # Litestream 0.5 filters `status` by the absolute database path, while + # `ltx` and `restore` match the path as written in the config file, so a + # relative path is expanded here to keep the documented + # `--database=storage/production.sqlite3` form working. + def status(database = nil, **argv) + execute("status", argv, database && File.expand_path(database)) + end - execute("snapshots", argv, database) + def generations(*) + raise CommandFailedException, "`generations` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)" end - def wal(database, **argv) - raise DatabaseRequiredException, "database argument is required for wal command, e.g. litestream:wal -- --database=path/to/database.sqlite" if database.nil? + def snapshots(*) + raise CommandFailedException, "`snapshots` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)" + end - execute("wal", argv, database) + def wal(*) + raise CommandFailedException, "`wal` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)" end private diff --git a/lib/litestream/generators/litestream/templates/config.yml.erb b/lib/litestream/generators/litestream/templates/config.yml.erb index 43009db..f789f8a 100644 --- a/lib/litestream/generators/litestream/templates/config.yml.erb +++ b/lib/litestream/generators/litestream/templates/config.yml.erb @@ -9,13 +9,18 @@ # `replicate` command. # # For more details, see: https://litestream.io/reference/config/ +# Retention and snapshot settings are global in Litestream 0.5. +snapshot: + interval: 24h + retention: 24h + dbs: <%- production_sqlite_databases.each do |database| -%> - path: <%= database %> - replicas: - - type: s3 - bucket: $LITESTREAM_REPLICA_BUCKET - path: <%= database %> - access-key-id: $LITESTREAM_ACCESS_KEY_ID - secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY + replica: + type: s3 + bucket: $LITESTREAM_REPLICA_BUCKET + path: <%= database %> + access-key-id: $LITESTREAM_ACCESS_KEY_ID + secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY <%- end -%> diff --git a/lib/litestream/upstream.rb b/lib/litestream/upstream.rb index 0029efd..c25bead 100644 --- a/lib/litestream/upstream.rb +++ b/lib/litestream/upstream.rb @@ -1,14 +1,25 @@ module Litestream module Upstream - VERSION = "v0.3.13" + VERSION = "0.5.17" # rubygems platform name => upstream release filename NATIVE_PLATFORMS = { "aarch64-linux" => "litestream-#{VERSION}-linux-arm64.tar.gz", - "arm64-darwin" => "litestream-#{VERSION}-darwin-arm64.zip", + "arm64-darwin" => "litestream-#{VERSION}-darwin-arm64.tar.gz", "arm64-linux" => "litestream-#{VERSION}-linux-arm64.tar.gz", - "x86_64-darwin" => "litestream-#{VERSION}-darwin-amd64.zip", - "x86_64-linux" => "litestream-#{VERSION}-linux-amd64.tar.gz" + "x86_64-darwin" => "litestream-#{VERSION}-darwin-x86_64.tar.gz", + "x86_64-linux" => "litestream-#{VERSION}-linux-x86_64.tar.gz" } + + CHECKSUMS = { + "litestream-0.5.17-linux-x86_64.tar.gz" => "cfb371176d164437ae869f8351cfde49bd1804ae71c61923f75c9cba9c9c006d", + "litestream-0.5.17-linux-arm64.tar.gz" => "f8ca4a050095c1efbda2c4365172e61bf9d955ea0d9ac42f448b52e51819baa5", + "litestream-0.5.17-darwin-x86_64.tar.gz" => "891875af09db152e93a4b31a8a79f538ce7ce702c132803cfe0a831e7cb1b7db", + "litestream-0.5.17-darwin-arm64.tar.gz" => "e211f68ff7658d19f193f2914417afdf8f89a053ff8f263e5d6b3b1d3bbc7b08" + } + + def self.download_url(filename) + "https://github.com/benbjohnson/litestream/releases/download/v#{VERSION}/#{filename}" + end end end diff --git a/lib/tasks/litestream_tasks.rake b/lib/tasks/litestream_tasks.rake index 387701b..0cf4945 100644 --- a/lib/tasks/litestream_tasks.rake +++ b/lib/tasks/litestream_tasks.rake @@ -32,30 +32,35 @@ namespace :litestream do puts Litestream::Commands::Output.format(Litestream::Commands.databases(**options)) end - desc "List all generations for a database or replica, for example `rake litestream:generations -- -database=storage/production.sqlite3`" - task generations: :environment do + desc "List all LTX files for a database or replica, for example `rake litestream:ltx -- -database=storage/production.sqlite3`" + task ltx: :environment do options = parse_argv_options database = options.delete(:"--database") || options.delete(:"-database") - puts Litestream::Commands::Output.format(Litestream::Commands.generations(database, **options)) + puts Litestream::Commands::Output.format(Litestream::Commands.ltx(database, **options)) end - desc "List all snapshots for a database or replica, for example `rake litestream:snapshots -- -database=storage/production.sqlite3`" - task snapshots: :environment do + desc "Show replication status, for example `rake litestream:status -- -database=storage/production.sqlite3`" + task status: :environment do options = parse_argv_options database = options.delete(:"--database") || options.delete(:"-database") - puts Litestream::Commands::Output.format(Litestream::Commands.snapshots(database, **options)) + puts Litestream::Commands::Output.format(Litestream::Commands.status(database, **options)) end - desc "List all wal files for a database or replica, for example `rake litestream:wal -- -database=storage/production.sqlite3`" - task wal: :environment do - options = parse_argv_options - database = options.delete(:"--database") || options.delete(:"-database") + desc "Explain the removal of the generations command" + task generations: :environment do + abort "`generations` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)" + end - puts Litestream::Commands::Output.format( - Litestream::Commands.wal(database, **options) - ) + desc "Explain the removal of the snapshots command" + task snapshots: :environment do + abort "`snapshots` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)" + end + + desc "Explain the removal of the wal command" + task wal: :environment do + abort "`wal` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)" end private diff --git a/litestream.gemspec b/litestream.gemspec index e004350..e19400f 100644 --- a/litestream.gemspec +++ b/litestream.gemspec @@ -34,7 +34,6 @@ Gem::Specification.new do |spec| spec.add_dependency "railties", rails_version end spec.add_development_dependency "rails" - spec.add_development_dependency "rubyzip" # For more information and examples about making a new gem, check out our # guide at: https://bundler.io/guides/creating_gem.html diff --git a/rakelib/package.rake b/rakelib/package.rake index f1a047f..c4d134a 100644 --- a/rakelib/package.rake +++ b/rakelib/package.rake @@ -58,12 +58,13 @@ require "rubygems/package" require "rubygems/package_task" require "open-uri" +require "digest" +require "stringio" require "zlib" -require "zip" require_relative "../lib/litestream/upstream" def litestream_download_url(filename) - "https://github.com/benbjohnson/litestream/releases/download/#{Litestream::Upstream::VERSION}/#{filename}" + Litestream::Upstream.download_url(filename) end LITESTREAM_RAILS_GEMSPEC = Bundler.load_gemspec("litestream.gemspec") @@ -77,11 +78,13 @@ Litestream::Upstream::NATIVE_PLATFORMS.each do |platform, filename| LITESTREAM_RAILS_GEMSPEC.dup.tap do |gemspec| exedir = File.join(gemspec.bindir, platform) # "exe/x86_64-linux" exepath = File.join(exedir, "litestream") # "exe/x86_64-linux/litestream" + checksum_path = "#{exepath}.sha256" + expected_checksum = Litestream::Upstream::CHECKSUMS.fetch(filename) exepaths << exepath # modify a copy of the gemspec to include the native executable gemspec.platform = platform - gemspec.files += [exepath, "LICENSE-DEPENDENCIES"] + gemspec.files += [exepath, checksum_path, "LICENSE-DEPENDENCIES"] # create a package task gem_path = Gem::PackageTask.new(gemspec).define @@ -89,28 +92,35 @@ Litestream::Upstream::NATIVE_PLATFORMS.each do |platform, filename| task "gem:#{platform}" => [gem_path] directory exedir - file exepath => [exedir] do + validation_task = "validate-litestream-#{platform}" + task validation_task do + checksum_matches = File.exist?(checksum_path) && File.read(checksum_path).strip == expected_checksum + FileUtils.rm_f([exepath, checksum_path]) unless checksum_matches + end + + file exepath => [exedir, validation_task] do release_url = litestream_download_url(filename) warn "Downloading #{exepath} from #{release_url} ..." - # lazy, but fine for now. URI.open(release_url) do |remote| # standard:disable Security/Open - if release_url.end_with?(".zip") - Zip::File.open_buffer(remote) do |zip_file| - zip_file.extract("litestream", exepath) - end - elsif release_url.end_with?(".gz") - Zlib::GzipReader.wrap(remote) do |gz| - Gem::Package::TarReader.new(gz) do |reader| - reader.seek("litestream") do |file| - File.binwrite(exepath, file.read) - end + archive = remote.read + actual_checksum = Digest::SHA256.hexdigest(archive) + unless actual_checksum == expected_checksum + raise "Checksum mismatch for #{filename}: expected #{expected_checksum}, got #{actual_checksum}" + end + + Zlib::GzipReader.wrap(StringIO.new(archive)) do |gz| + Gem::Package::TarReader.new(gz) do |reader| + reader.seek("litestream") do |file| + File.binwrite(exepath, file.read) end end end end FileUtils.chmod(0o755, exepath, verbose: true) + File.write(checksum_path, "#{expected_checksum}\n") end + file checksum_path => [exepath] end end diff --git a/test/controllers/test_processes_controller.rb b/test/controllers/test_processes_controller.rb index 3f15a6e..af4f732 100644 --- a/test/controllers/test_processes_controller.rb +++ b/test/controllers/test_processes_controller.rb @@ -5,17 +5,12 @@ class Litestream::TestProcessesController < ActionDispatch::IntegrationTest stubbed_process = {pid: "12345", status: "sleeping", started: DateTime.now} stubbed_databases = [ {"path" => "[ROOT]/storage/test.sqlite3", - "replicas" => "s3", - "generations" => [ - {"generation" => SecureRandom.hex, - "name" => "s3", - "lag" => "23h59m59s", - "start" => "2024-05-02T11:32:16Z", - "end" => "2024-05-02T11:33:10Z", - "snapshots" => [ - {"index" => "0", "size" => "4145735", "created" => "2024-05-02T11:32:16Z"} - ]} - ]} + "replica" => "s3", + "status" => {"database" => "storage/test.sqlite3", "status" => "ok", "local_txid" => "000000000000000a", "wal_size" => 131072}, + "ltx" => [ + {"level" => 0, "min_txid" => "0000000000000008", "max_txid" => "000000000000000a", "size" => 1013, "timestamp" => "2026-09-08T03:16:43Z"} + ]}, + {"path" => "[ROOT]/storage/error.sqlite3", "replica" => "s3", "error" => "replica unavailable"} ] Litestream.stub :replicate_process, stubbed_process do Litestream.stub :databases, stubbed_databases do @@ -28,10 +23,13 @@ class Litestream::TestProcessesController < ActionDispatch::IntegrationTest assert_select "time", stubbed_process[:started].to_formatted_s(:db) end - assert_select "#databases li", 1 do + assert_select "#databases li", 2 + assert_select "#databases li:first-child" do assert_select "h2 code", stubbed_databases[0]["path"] - assert_select "details##{stubbed_databases[0]["generations"][0]["generation"]}" + assert_select "tbody tr", 1 + assert_select "td", text: "000000000000000a" end + assert_select "#databases li:last-child .text-red-600", "replica unavailable" end end end diff --git a/test/dummy/config/litestream.yml b/test/dummy/config/litestream.yml index 53280aa..50c2e46 100644 --- a/test/dummy/config/litestream.yml +++ b/test/dummy/config/litestream.yml @@ -9,12 +9,17 @@ # `replicate` command. # # For more details, see: https://litestream.io/reference/config/ +# Retention and snapshot settings are global in Litestream 0.5. +snapshot: + interval: 24h + retention: 24h + dbs: - path: storage/test.sqlite3 - replicas: - - type: s3 - bucket: $LITESTREAM_REPLICA_BUCKET - path: test - endpoint: http://localhost:9000 - access-key-id: $LITESTREAM_ACCESS_KEY_ID - secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY + replica: + type: s3 + bucket: $LITESTREAM_REPLICA_BUCKET + path: test + endpoint: http://localhost:9000 + access-key-id: $LITESTREAM_ACCESS_KEY_ID + secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY diff --git a/test/generators/test_install.rb b/test/generators/test_install.rb index ad7cda9..6805f0c 100644 --- a/test/generators/test_install.rb +++ b/test/generators/test_install.rb @@ -22,6 +22,10 @@ def after_teardown assert_match "- path: storage/test.sqlite3", content assert_match "- path: storage/queue.sqlite3", content assert_match "- path: storage/errors.sqlite3", content + assert_match "snapshot:", content + assert_match "interval: 24h", content + assert_match "retention: 24h", content + assert_match "replica:", content assert_match "bucket: $LITESTREAM_REPLICA_BUCKET", content assert_match "access-key-id: $LITESTREAM_ACCESS_KEY_ID", content assert_match "secret-access-key: $LITESTREAM_SECRET_ACCESS_KEY", content diff --git a/test/litestream/test_commands.rb b/test/litestream/test_commands.rb index 72dd8c1..1a7c4c7 100644 --- a/test/litestream/test_commands.rb +++ b/test/litestream/test_commands.rb @@ -473,402 +473,83 @@ def test_databases_read_from_custom_configured_litestream_config_path end end - class TestGenerationsCommand < TestCommands - def test_generations_with_no_options + class TestLtxCommand < TestCommands + def test_ltx_with_no_options stub = proc do |cmd| executable, command, *argv = cmd assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "generations", command - assert_equal 3, argv.size - assert_equal "--config", argv[0] - assert_match Regexp.new("dummy/config/litestream.yml"), argv[1] - assert_equal "db/test.sqlite3", argv[2] + assert_equal "ltx", command + assert_equal ["--config", Rails.root.join("config/litestream.yml").to_s, "db/test.sqlite3"], argv end - Litestream::Commands.stub :run, stub do - Litestream::Commands.generations("db/test.sqlite3") - end - end - def test_generations_with_boolean_option - stub = proc do |cmd| - executable, command, *argv = cmd - assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "generations", command - assert_equal 4, argv.size - assert_equal "--config", argv[0] - assert_match Regexp.new("dummy/config/litestream.yml"), argv[1] - assert_equal "--if-db-not-exists", argv[2] - assert_equal "db/test.sqlite3", argv[3] - end Litestream::Commands.stub :run, stub do - Litestream::Commands.generations("db/test.sqlite3", "--if-db-not-exists" => nil) + Litestream::Commands.ltx("db/test.sqlite3") end end - def test_generations_with_string_option + def test_ltx_with_options stub = proc do |cmd| - executable, command, *argv = cmd - assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "generations", command - assert_equal 5, argv.size - assert_equal "--config", argv[0] - assert_match Regexp.new("dummy/config/litestream.yml"), argv[1] - assert_equal "--parallelism", argv[2] - assert_equal "10", argv[3] - assert_equal "db/test.sqlite3", argv[4] - end - Litestream::Commands.stub :run, stub do - Litestream::Commands.generations("db/test.sqlite3", "--parallelism" => 10) + assert_equal ["--config", "CONFIG", "--level", "all", "-json", "db/test.sqlite3"], cmd.drop(2) end - end - def test_generations_with_config_option - stub = proc do |cmd| - executable, command, *argv = cmd - assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "generations", command - assert_equal 3, argv.size - assert_equal "--config", argv[0] - assert_equal "CONFIG", argv[1] - assert_equal "db/test.sqlite3", argv[2] - end Litestream::Commands.stub :run, stub do - Litestream::Commands.generations("db/test.sqlite3", "--config" => "CONFIG") - end - end - - def test_generations_sets_replica_bucket_env_var_from_config_when_env_var_not_set - Litestream.replica_bucket = "mybkt" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.generations("db/test.sqlite3") - end - - assert_equal "mybkt", ENV["LITESTREAM_REPLICA_BUCKET"] - assert_nil ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_nil ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_generations_sets_replica_key_id_env_var_from_config_when_env_var_not_set - Litestream.replica_key_id = "mykey" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.generations("db/test.sqlite3") - end - - assert_nil ENV["LITESTREAM_REPLICA_BUCKET"] - assert_equal "mykey", ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_nil ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_generations_sets_replica_access_key_env_var_from_config_when_env_var_not_set - Litestream.replica_access_key = "access" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.generations("db/test.sqlite3") - end - - assert_nil ENV["LITESTREAM_REPLICA_BUCKET"] - assert_nil ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_equal "access", ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_generations_sets_all_env_vars_from_config_when_env_vars_not_set - Litestream.replica_bucket = "mybkt" - Litestream.replica_key_id = "mykey" - Litestream.replica_access_key = "access" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.generations("db/test.sqlite3") + Litestream::Commands.ltx("db/test.sqlite3", "--config" => "CONFIG", "--level" => "all", :json => true) end - - assert_equal "mybkt", ENV["LITESTREAM_REPLICA_BUCKET"] - assert_equal "mykey", ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_equal "access", ENV["LITESTREAM_SECRET_ACCESS_KEY"] end - def test_generations_does_not_set_env_var_from_config_when_env_vars_already_set - ENV["LITESTREAM_REPLICA_BUCKET"] = "original_bkt" - ENV["LITESTREAM_ACCESS_KEY_ID"] = "original_key" - ENV["LITESTREAM_SECRET_ACCESS_KEY"] = "original_access" - - Litestream.replica_bucket = "mybkt" - Litestream.replica_key_id = "mykey" - Litestream.replica_access_key = "access" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.generations("db/test.sqlite3") + def test_ltx_requires_database + error = assert_raises Litestream::Commands::DatabaseRequiredException do + Litestream::Commands.ltx(nil) end - assert_equal "original_bkt", ENV["LITESTREAM_REPLICA_BUCKET"] - assert_equal "original_key", ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_equal "original_access", ENV["LITESTREAM_SECRET_ACCESS_KEY"] + assert_match "database argument is required for ltx command", error.message end end - class TestSnapshotsCommand < TestCommands - def test_snapshots_with_no_options + class TestStatusCommand < TestCommands + def test_status_with_no_database stub = proc do |cmd| executable, command, *argv = cmd assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "snapshots", command - assert_equal 3, argv.size - assert_equal "--config", argv[0] - assert_match Regexp.new("dummy/config/litestream.yml"), argv[1] - assert_equal "db/test.sqlite3", argv[2] + assert_equal "status", command + assert_equal ["--config", Rails.root.join("config/litestream.yml").to_s], argv end - Litestream::Commands.stub :run, stub do - Litestream::Commands.snapshots("db/test.sqlite3") - end - end - def test_snapshots_with_boolean_option - stub = proc do |cmd| - executable, command, *argv = cmd - assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "snapshots", command - assert_equal 4, argv.size - assert_equal "--config", argv[0] - assert_match Regexp.new("dummy/config/litestream.yml"), argv[1] - assert_equal "--if-db-not-exists", argv[2] - assert_equal "db/test.sqlite3", argv[3] - end Litestream::Commands.stub :run, stub do - Litestream::Commands.snapshots("db/test.sqlite3", "--if-db-not-exists" => nil) + Litestream::Commands.status end end - def test_snapshots_with_string_option + def test_status_expands_a_relative_database_path stub = proc do |cmd| - executable, command, *argv = cmd - assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "snapshots", command - assert_equal 5, argv.size - assert_equal "--config", argv[0] - assert_match Regexp.new("dummy/config/litestream.yml"), argv[1] - assert_equal "--parallelism", argv[2] - assert_equal "10", argv[3] - assert_equal "db/test.sqlite3", argv[4] - end - Litestream::Commands.stub :run, stub do - Litestream::Commands.snapshots("db/test.sqlite3", "--parallelism" => 10) + assert_equal File.expand_path("db/test.sqlite3"), cmd.last end - end - def test_snapshots_with_config_option - stub = proc do |cmd| - executable, command, *argv = cmd - assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "snapshots", command - assert_equal 3, argv.size - assert_equal "--config", argv[0] - assert_equal "CONFIG", argv[1] - assert_equal "db/test.sqlite3", argv[2] - end Litestream::Commands.stub :run, stub do - Litestream::Commands.snapshots("db/test.sqlite3", "--config" => "CONFIG") + Litestream::Commands.status("db/test.sqlite3") end end - def test_snapshots_sets_replica_bucket_env_var_from_config_when_env_var_not_set - Litestream.replica_bucket = "mybkt" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.snapshots("db/test.sqlite3") - end - - assert_equal "mybkt", ENV["LITESTREAM_REPLICA_BUCKET"] - assert_nil ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_nil ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_snapshots_sets_replica_key_id_env_var_from_config_when_env_var_not_set - Litestream.replica_key_id = "mykey" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.snapshots("db/test.sqlite3") - end - - assert_nil ENV["LITESTREAM_REPLICA_BUCKET"] - assert_equal "mykey", ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_nil ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_snapshots_sets_replica_access_key_env_var_from_config_when_env_var_not_set - Litestream.replica_access_key = "access" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.snapshots("db/test.sqlite3") - end - - assert_nil ENV["LITESTREAM_REPLICA_BUCKET"] - assert_nil ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_equal "access", ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_snapshots_sets_all_env_vars_from_config_when_env_vars_not_set - Litestream.replica_bucket = "mybkt" - Litestream.replica_key_id = "mykey" - Litestream.replica_access_key = "access" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.snapshots("db/test.sqlite3") - end - - assert_equal "mybkt", ENV["LITESTREAM_REPLICA_BUCKET"] - assert_equal "mykey", ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_equal "access", ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_snapshots_does_not_set_env_var_from_config_when_env_vars_already_set - ENV["LITESTREAM_REPLICA_BUCKET"] = "original_bkt" - ENV["LITESTREAM_ACCESS_KEY_ID"] = "original_key" - ENV["LITESTREAM_SECRET_ACCESS_KEY"] = "original_access" - - Litestream.replica_bucket = "mybkt" - Litestream.replica_key_id = "mykey" - Litestream.replica_access_key = "access" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.snapshots("db/test.sqlite3") - end - - assert_equal "original_bkt", ENV["LITESTREAM_REPLICA_BUCKET"] - assert_equal "original_key", ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_equal "original_access", ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - end - - class TestWalCommand < TestCommands - def test_wal_with_no_options + def test_status_with_database_and_options stub = proc do |cmd| - executable, command, *argv = cmd - assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "wal", command - assert_equal 3, argv.size - assert_equal "--config", argv[0] - assert_match Regexp.new("dummy/config/litestream.yml"), argv[1] - assert_equal "db/test.sqlite3", argv[2] + assert_equal ["--config", "CONFIG", "--no-expand-env", "-json", File.expand_path("db/test.sqlite3")], cmd.drop(2) end - Litestream::Commands.stub :run, stub do - Litestream::Commands.wal("db/test.sqlite3") - end - end - def test_wal_with_boolean_option - stub = proc do |cmd| - executable, command, *argv = cmd - assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "wal", command - assert_equal 4, argv.size - assert_equal "--config", argv[0] - assert_match Regexp.new("dummy/config/litestream.yml"), argv[1] - assert_equal "--if-db-not-exists", argv[2] - assert_equal "db/test.sqlite3", argv[3] - end Litestream::Commands.stub :run, stub do - Litestream::Commands.wal("db/test.sqlite3", "--if-db-not-exists" => nil) + Litestream::Commands.status("db/test.sqlite3", "--config" => "CONFIG", "--no-expand-env" => nil, :json => true) end end + end - def test_wal_with_string_option - stub = proc do |cmd| - executable, command, *argv = cmd - assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "wal", command - assert_equal 5, argv.size - assert_equal "--config", argv[0] - assert_match Regexp.new("dummy/config/litestream.yml"), argv[1] - assert_equal "--parallelism", argv[2] - assert_equal "10", argv[3] - assert_equal "db/test.sqlite3", argv[4] - end - Litestream::Commands.stub :run, stub do - Litestream::Commands.wal("db/test.sqlite3", "--parallelism" => 10) - end - end - - def test_wal_with_config_option - stub = proc do |cmd| - executable, command, *argv = cmd - assert_match Regexp.new("exe/test/litestream"), executable - assert_equal "wal", command - assert_equal 3, argv.size - assert_equal "--config", argv[0] - assert_equal "CONFIG", argv[1] - assert_equal "db/test.sqlite3", argv[2] - end - Litestream::Commands.stub :run, stub do - Litestream::Commands.wal("db/test.sqlite3", "--config" => "CONFIG") - end - end - - def test_wal_sets_replica_bucket_env_var_from_config_when_env_var_not_set - Litestream.replica_bucket = "mybkt" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.wal("db/test.sqlite3") - end - - assert_equal "mybkt", ENV["LITESTREAM_REPLICA_BUCKET"] - assert_nil ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_nil ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_wal_sets_replica_key_id_env_var_from_config_when_env_var_not_set - Litestream.replica_key_id = "mykey" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.wal("db/test.sqlite3") - end - - assert_nil ENV["LITESTREAM_REPLICA_BUCKET"] - assert_equal "mykey", ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_nil ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_wal_sets_replica_access_key_env_var_from_config_when_env_var_not_set - Litestream.replica_access_key = "access" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.wal("db/test.sqlite3") - end - - assert_nil ENV["LITESTREAM_REPLICA_BUCKET"] - assert_nil ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_equal "access", ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_wal_sets_all_env_vars_from_config_when_env_vars_not_set - Litestream.replica_bucket = "mybkt" - Litestream.replica_key_id = "mykey" - Litestream.replica_access_key = "access" - - Litestream::Commands.stub :run, nil do - Litestream::Commands.wal("db/test.sqlite3") - end - - assert_equal "mybkt", ENV["LITESTREAM_REPLICA_BUCKET"] - assert_equal "mykey", ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_equal "access", ENV["LITESTREAM_SECRET_ACCESS_KEY"] - end - - def test_wal_does_not_set_env_var_from_config_when_env_vars_already_set - ENV["LITESTREAM_REPLICA_BUCKET"] = "original_bkt" - ENV["LITESTREAM_ACCESS_KEY_ID"] = "original_key" - ENV["LITESTREAM_SECRET_ACCESS_KEY"] = "original_access" - - Litestream.replica_bucket = "mybkt" - Litestream.replica_key_id = "mykey" - Litestream.replica_access_key = "access" + class TestRemovedCommands < TestCommands + %w[generations snapshots wal].each do |command| + define_method(:"test_#{command}_raises_with_migration_message") do + error = assert_raises Litestream::Commands::CommandFailedException do + Litestream::Commands.public_send(command, "db/test.sqlite3") + end - Litestream::Commands.stub :run, nil do - Litestream::Commands.wal("db/test.sqlite3") + assert_equal "`#{command}` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)", error.message end - - assert_equal "original_bkt", ENV["LITESTREAM_REPLICA_BUCKET"] - assert_equal "original_key", ENV["LITESTREAM_ACCESS_KEY_ID"] - assert_equal "original_access", ENV["LITESTREAM_SECRET_ACCESS_KEY"] end end @@ -883,7 +564,7 @@ def setup while [ "$#" -gt 0 ]; do case "$1" in --table) - printf 'path replicas\n/tmp/app.sqlite3 s3\n' + printf 'path replica\n/tmp/app.sqlite3 s3\n' exit 0 ;; --json-object) @@ -892,7 +573,7 @@ def setup exit 0 ;; --json-array) - printf '[{"path":"/tmp/app.sqlite3","replicas":["s3"]}]\n' + printf '[{"path":"/tmp/app.sqlite3","replica":"s3"}]\n' exit 0 ;; --json-empty-list) @@ -950,7 +631,7 @@ def teardown end def test_successful_table_output - assert_equal [{"path" => "/tmp/app.sqlite3", "replicas" => "s3"}], run_with_fake { Litestream::Commands.databases("--table" => nil) } + assert_equal [{"path" => "/tmp/app.sqlite3", "replica" => "s3"}], run_with_fake { Litestream::Commands.databases("--table" => nil) } end def test_successful_json_object @@ -958,7 +639,7 @@ def test_successful_json_object end def test_successful_json_array - expected = [{"path" => "/tmp/app.sqlite3", "replicas" => ["s3"]}] + expected = [{"path" => "/tmp/app.sqlite3", "replica" => "s3"}] assert_equal expected, run_with_fake { Litestream::Commands.databases(json: true, "--json-array": nil) } end @@ -1060,8 +741,8 @@ def wait_for_pid(pid_file) class TestOutput < ActiveSupport::TestCase def test_output_formatting_generates_table_with_data data = [ - {path: "/storage/database.db", replicas: "s3"}, - {path: "/storage/another-database.db", replicas: "s3"} + {path: "/storage/database.db", replica: "s3"}, + {path: "/storage/another-database.db", replica: "s3"} ] result = Litestream::Commands::Output.format(data) @@ -1070,23 +751,23 @@ def test_output_formatting_generates_table_with_data assert_equal 3, lines.length assert_includes lines[0], "path" - assert_includes lines[0], "replicas" + assert_includes lines[0], "replica" assert_includes lines[1], "/storage/database.db" assert_includes lines[2], "/storage/another-database.db" end def test_output_formatting_generates_formatted_table data = [ - {path: "/storage/database.db", replicas: "s3"}, - {path: "/storage/another-database.db", replicas: "s3"} + {path: "/storage/database.db", replica: "s3"}, + {path: "/storage/another-database.db", replica: "s3"} ] result = Litestream::Commands::Output.format(data) lines = result.split("\n") - replicas_pos = lines[0].index("replicas") - assert_equal replicas_pos, lines[1].index("s3") - assert_equal replicas_pos, lines[2].index("s3") + replica_pos = lines[0].index("replica") + assert_equal replica_pos, lines[1].index("s3") + assert_equal replica_pos, lines[2].index("s3") end end end diff --git a/test/litestream/test_upstream.rb b/test/litestream/test_upstream.rb new file mode 100644 index 0000000..062aefa --- /dev/null +++ b/test/litestream/test_upstream.rb @@ -0,0 +1,16 @@ +require "test_helper" + +class TestUpstream < ActiveSupport::TestCase + def test_every_native_platform_has_a_checksum + Litestream::Upstream::NATIVE_PLATFORMS.each_value do |filename| + assert Litestream::Upstream::CHECKSUMS.key?(filename), "missing checksum for #{filename}" + end + end + + def test_release_urls_use_version_tag_and_unprefixed_filenames + Litestream::Upstream::NATIVE_PLATFORMS.each_value do |filename| + assert_equal "https://github.com/benbjohnson/litestream/releases/download/v0.5.17/#{filename}", Litestream::Upstream.download_url(filename) + refute filename.start_with?("litestream-v") + end + end +end diff --git a/test/tasks/test_litestream_tasks.rb b/test/tasks/test_litestream_tasks.rb index 344c3ec..79dfd31 100644 --- a/test/tasks/test_litestream_tasks.rb +++ b/test/tasks/test_litestream_tasks.rb @@ -9,6 +9,8 @@ def setup Rake::Task["litestream:replicate"].reenable Rake::Task["litestream:restore"].reenable Rake::Task["litestream:databases"].reenable + Rake::Task["litestream:ltx"].reenable + Rake::Task["litestream:status"].reenable Rake::Task["litestream:generations"].reenable Rake::Task["litestream:snapshots"].reenable Rake::Task["litestream:wal"].reenable @@ -138,129 +140,102 @@ def test_databases_task_with_arguments_without_separator end end - class TestGenerationsTask < TestLitestreamTasks - def test_generations_task_with_only_database_using_single_dash + class TestLtxTask < TestLitestreamTasks + def test_ltx_task_with_only_database_using_single_dash ARGV.replace ["--", "-database=db/test.sqlite3"] fake = Minitest::Mock.new fake.expect :call, nil, ["db/test.sqlite3"] - Litestream::Commands.stub :generations, fake do - Rake.application.invoke_task "litestream:generations" + Litestream::Commands.stub :ltx, fake do + Rake.application.invoke_task "litestream:ltx" end fake.verify end - def test_generations_task_with_only_database_using_double_dash + def test_ltx_task_with_only_database_using_double_dash ARGV.replace ["--", "--database=db/test.sqlite3"] fake = Minitest::Mock.new fake.expect :call, nil, ["db/test.sqlite3"] - Litestream::Commands.stub :generations, fake do - Rake.application.invoke_task "litestream:generations" + Litestream::Commands.stub :ltx, fake do + Rake.application.invoke_task "litestream:ltx" end fake.verify end - def test_generations_task_with_arguments - ARGV.replace ["--", "-database=db/test.sqlite3", "--if-db-not-exists"] + def test_ltx_task_with_arguments + ARGV.replace ["--", "-database=db/test.sqlite3", "--level=all"] fake = Minitest::Mock.new - fake.expect :call, nil, ["db/test.sqlite3"], "--if-db-not-exists": nil - Litestream::Commands.stub :generations, fake do - Rake.application.invoke_task "litestream:generations" + fake.expect :call, nil, ["db/test.sqlite3"], "--level": "all" + Litestream::Commands.stub :ltx, fake do + Rake.application.invoke_task "litestream:ltx" end fake.verify end - def test_generations_task_with_arguments_without_separator + def test_ltx_task_with_arguments_without_separator ARGV.replace ["-database=db/test.sqlite3"] fake = Minitest::Mock.new fake.expect :call, nil, [nil] - Litestream::Commands.stub :generations, fake do - Rake.application.invoke_task "litestream:generations" + Litestream::Commands.stub :ltx, fake do + Rake.application.invoke_task "litestream:ltx" end fake.verify end end - class TestSnapshotsTask < TestLitestreamTasks - def test_snapshots_task_with_only_database_using_single_dash + class TestStatusTask < TestLitestreamTasks + def test_status_task_with_database ARGV.replace ["--", "-database=db/test.sqlite3"] fake = Minitest::Mock.new fake.expect :call, nil, ["db/test.sqlite3"] - Litestream::Commands.stub :snapshots, fake do - Rake.application.invoke_task "litestream:snapshots" + Litestream::Commands.stub :status, fake do + Rake.application.invoke_task "litestream:status" end fake.verify end - def test_snapshots_task_with_only_database_using_double_dash - ARGV.replace ["--", "--database=db/test.sqlite3"] + def test_status_task_without_database + ARGV.replace ["--"] fake = Minitest::Mock.new - fake.expect :call, nil, ["db/test.sqlite3"] - Litestream::Commands.stub :snapshots, fake do - Rake.application.invoke_task "litestream:snapshots" + fake.expect :call, nil, [nil] + Litestream::Commands.stub :status, fake do + Rake.application.invoke_task "litestream:status" end fake.verify end - def test_snapshots_task_with_arguments - ARGV.replace ["--", "-database=db/test.sqlite3", "--if-db-not-exists"] + def test_status_task_with_arguments + ARGV.replace ["--", "--database=db/test.sqlite3", "--json"] fake = Minitest::Mock.new - fake.expect :call, nil, ["db/test.sqlite3"], "--if-db-not-exists": nil - Litestream::Commands.stub :snapshots, fake do - Rake.application.invoke_task "litestream:snapshots" + fake.expect :call, nil, ["db/test.sqlite3"], "--json": nil + Litestream::Commands.stub :status, fake do + Rake.application.invoke_task "litestream:status" end fake.verify end - def test_snapshots_task_with_arguments_without_separator - ARGV.replace ["-database=db/test.sqlite3"] + def test_status_task_with_arguments_without_separator + ARGV.replace ["--json"] fake = Minitest::Mock.new fake.expect :call, nil, [nil] - Litestream::Commands.stub :snapshots, fake do - Rake.application.invoke_task "litestream:snapshots" + Litestream::Commands.stub :status, fake do + Rake.application.invoke_task "litestream:status" end fake.verify end end - class TestWalTask < TestLitestreamTasks - def test_wal_task_with_only_database_using_single_dash - ARGV.replace ["--", "-database=db/test.sqlite3"] - fake = Minitest::Mock.new - fake.expect :call, nil, ["db/test.sqlite3"] - Litestream::Commands.stub :wal, fake do - Rake.application.invoke_task "litestream:wal" - end - fake.verify - end - - def test_wal_task_with_only_database_using_double_dash - ARGV.replace ["--", "--database=db/test.sqlite3"] - fake = Minitest::Mock.new - fake.expect :call, nil, ["db/test.sqlite3"] - Litestream::Commands.stub :wal, fake do - Rake.application.invoke_task "litestream:wal" - end - fake.verify - end - - def test_wal_task_with_arguments - ARGV.replace ["--", "-database=db/test.sqlite3", "--if-db-not-exists"] - fake = Minitest::Mock.new - fake.expect :call, nil, ["db/test.sqlite3"], "--if-db-not-exists": nil - Litestream::Commands.stub :wal, fake do - Rake.application.invoke_task "litestream:wal" - end - fake.verify - end + class TestRemovedTasks < TestLitestreamTasks + %w[generations snapshots wal].each do |command| + define_method(:"test_#{command}_task_aborts_with_migration_message") do + _out, err = capture_io do + error = assert_raises(SystemExit) do + Rake.application.invoke_task "litestream:#{command}" + end + assert_equal 1, error.status + end - def test_wal_task_with_arguments_without_separator - ARGV.replace ["-database=db/test.sqlite3"] - fake = Minitest::Mock.new - fake.expect :call, nil, [nil] - Litestream::Commands.stub :wal, fake do - Rake.application.invoke_task "litestream:wal" + assert_equal "`#{command}` was removed in Litestream 0.5; use `ltx` (see README, Upgrading from 0.3)\n", err end - fake.verify end end end diff --git a/test/test_litestream.rb b/test/test_litestream.rb index 5c1f41d..991ad6c 100644 --- a/test/test_litestream.rb +++ b/test/test_litestream.rb @@ -85,4 +85,55 @@ def test_replicate_process_ps assert_equal info[:started].class, DateTime end end + + def test_databases_loads_status_and_ltx_and_isolates_command_failures + database_path = Rails.root.join("storage/test.sqlite3").to_s + failing_path = Rails.root.join("storage/failing.sqlite3").to_s + databases = [ + {"path" => database_path, "replica" => "file"}, + {"path" => failing_path, "replica" => "file"} + ] + status = [{"database" => database_path, "status" => "ok", "local_txid" => "000000000000000a", "wal_size" => 131_072}] + ltx = [{"level" => 0, "min_txid" => "0000000000000008", "max_txid" => "000000000000000a", "size" => 1013, "timestamp" => "2026-09-08T03:16:43Z"}] + databases_stub = proc do |**options| + assert_equal({json: true}, options) + databases.map(&:dup) + end + status_stub = proc do |path, **options| + assert_equal({json: true}, options) + raise Litestream::Commands::CommandFailedException, "status unavailable" if path == failing_path + + status + end + ltx_stub = proc do |path, **options| + assert_equal database_path, path + assert_equal({:json => true, "--level" => "all"}, options) + ltx + end + + Litestream::Commands.stub :databases, databases_stub do + Litestream::Commands.stub :status, status_stub do + Litestream::Commands.stub :ltx, ltx_stub do + result = Litestream.databases + + assert_equal "[ROOT]/storage/test.sqlite3", result[0]["path"] + assert_equal status.first, result[0]["status"] + assert_equal ltx, result[0]["ltx"] + assert_equal "status unavailable", result[1]["error"] + assert_equal "[ROOT]/storage/failing.sqlite3", result[1]["path"] + end + end + end + end + + def test_databases_propagates_non_command_errors + databases = [{"path" => Rails.root.join("storage/test.sqlite3").to_s, "replica" => "file"}] + + Litestream::Commands.stub :databases, databases do + Litestream::Commands.stub :status, proc { raise ArgumentError, "unexpected data" } do + error = assert_raises(ArgumentError) { Litestream.databases } + assert_equal "unexpected data", error.message + end + end + end end