diff --git a/README.md b/README.md index f77c4d5a..513d1b79 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,7 @@ Solid Queue can be used with SQL databases such as MySQL, PostgreSQL, or SQLite, - [Threads, processes, and signals](#threads-processes-and-signals) - [Database configuration](#database-configuration) - [Other configuration settings](#other-configuration-settings) + - [Validating the configuration](#validating-the-configuration) - [Lifecycle hooks](#lifecycle-hooks) - [Errors when enqueuing](#errors-when-enqueuing) - [Concurrency controls](#concurrency-controls) @@ -264,7 +265,7 @@ Here's an overview of the different options: ``` This will create a worker fetching jobs from all queues starting with `staging`. The wildcard `*` is only allowed on its own or at the end of a queue name; you can't specify queue names such as `*_some_queue`. These will be ignored. - + Also, if a wildcard (*) is included alongside explicit queue names, for example: `queues: [default, backend, *]`, then it would behave like `queues: *` Finally, you can combine prefixes with exact names, like `[ staging*, background ]`, and the behaviour with respect to order will be the same as with only exact names. @@ -408,6 +409,22 @@ There are several settings that control how Solid Queue works that you can set a - `clear_finished_jobs_after`: period to keep finished jobs around, in case `preserve_finished_jobs` is true — defaults to 1 day. When installing Solid Queue, [a recurring job](#recurring-tasks) is automatically configured to clear finished jobs every hour on the 12th minute in batches. You can edit the `recurring.yml` configuration to change this as you see fit. - `default_concurrency_control_period`: the value to be used as the default for the `duration` parameter in [concurrency controls](#concurrency-controls). It defaults to 3 minutes. +### Validating the configuration + +You can validate the Solid Queue configuration ahead of time, which is handy in deploy scripts or CI: + +```bash +# Using the bin/jobs binstub +bin/jobs check + +# Or via rake +bin/rails solid_queue:check +``` + +Both commands validate the same configuration for the current Rails environment, exit non-zero on any error, and print `Solid Queue configuration is valid` on success. They are tolerant of a missing database connection so they can run on CI or deploy hosts without DB credentials. + +`bin/jobs check` accepts the same flags as `bin/jobs start` (e.g. `--config_file`, `--recurring_schedule_file`, `--skip-recurring`). The rake task is configured via the same environment variables Solid Queue already honors: `SOLID_QUEUE_CONFIG`, `SOLID_QUEUE_RECURRING_SCHEDULE`, and `SOLID_QUEUE_SKIP_RECURRING`. To validate a specific environment's configuration, prefix with `RAILS_ENV`, for example `RAILS_ENV=production bin/jobs check`. + ## Lifecycle hooks @@ -781,7 +798,7 @@ SolidQueue.unschedule_recurring_task("my_dynamic_task") Only dynamic tasks can be unscheduled at runtime. Attempting to unschedule a static task (defined in `config/recurring.yml`) will raise an `ActiveRecord::RecordNotFound` error. -Tasks scheduled like this persist between Solid Queue's restarts and won't stop running until you manually unschedule them. +Tasks scheduled like this persist between Solid Queue's restarts and won't stop running until you manually unschedule them. ## Inspiration diff --git a/lib/solid_queue/cli.rb b/lib/solid_queue/cli.rb index 2f3f3e13..13af6588 100644 --- a/lib/solid_queue/cli.rb +++ b/lib/solid_queue/cli.rb @@ -30,5 +30,11 @@ def self.exit_on_failure? def start SolidQueue::Supervisor.start(**options.symbolize_keys) end + + desc :check, "Validates the Solid Queue configuration for the current Rails env without starting anything. Exits non-zero on errors." + def check + configuration = SolidQueue::Configuration.new(**options.symbolize_keys, skip_db_checks: true) + exit 1 unless configuration.check! + end end end diff --git a/lib/solid_queue/configuration.rb b/lib/solid_queue/configuration.rb index e63a000c..3ca640f7 100644 --- a/lib/solid_queue/configuration.rb +++ b/lib/solid_queue/configuration.rb @@ -69,6 +69,19 @@ def standalone? mode.fork? || @options[:standalone] end + def check!(out: $stdout, err: $stderr) + if valid? + out.puts "Solid Queue configuration is valid." + return true + end + + err.puts "Invalid Solid Queue configuration:" + errors.full_messages.each do |message| + message.each_line { |line| err.puts " #{line.chomp}" } + end + false + end + private attr_reader :options @@ -89,12 +102,21 @@ def ensure_valid_recurring_tasks end def ensure_correctly_sized_thread_pool - if (db_pool_size = SolidQueue::Record.connection_pool&.size) && db_pool_size < estimated_number_of_threads + if (db_pool_size = database_connection_pool_size) && db_pool_size < estimated_number_of_threads errors.add(:base, "Solid Queue is configured to use #{estimated_number_of_threads} threads but the " + "database connection pool is #{db_pool_size}. Increase it in `config/database.yml`") end end + # Returns nil instead of raising when skip_db_checks is set, so + # `bin/jobs check` works on CI/deploy hosts without DB credentials. + def database_connection_pool_size + SolidQueue::Record.connection_pool&.size + rescue ActiveRecord::ActiveRecordError + raise unless options[:skip_db_checks] + nil + end + def default_options { mode: ENV["SOLID_QUEUE_SUPERVISOR_MODE"] || :fork, diff --git a/lib/solid_queue/tasks.rb b/lib/solid_queue/tasks.rb index 91cd778b..d938c141 100644 --- a/lib/solid_queue/tasks.rb +++ b/lib/solid_queue/tasks.rb @@ -8,4 +8,10 @@ task start: :environment do SolidQueue::Supervisor.start end + + desc "validate the Solid Queue configuration for the current Rails env without starting any process" + task check: :environment do + configuration = SolidQueue::Configuration.new(skip_db_checks: true) + exit 1 unless configuration.check! + end end diff --git a/test/unit/cli_test.rb b/test/unit/cli_test.rb index 9ee0e233..0d106898 100644 --- a/test/unit/cli_test.rb +++ b/test/unit/cli_test.rb @@ -36,6 +36,27 @@ class CliTest < ActiveSupport::TestCase end end + test "check exits 0 and prints OK message for a valid configuration" do + out, err = capture_io do + assert_nothing_raised { SolidQueue::Cli.start([ "check", "--skip-recurring" ]) } + end + + assert_match "Solid Queue configuration is valid.", out + assert_empty err + end + + test "check exits 1 and prints invalid recurring task errors" do + out, err, exit_status = capture_check_run( + "--recurring_schedule_file", config_file_path(:recurring_with_invalid).to_s + ) + + assert_equal 1, exit_status + assert_match "Invalid Solid Queue configuration", err + assert_match "periodic_invalid_class", err + assert_match "periodic_incorrect_schedule", err + assert_empty out + end + private def configuration_from_cli(**cli_options) cli = SolidQueue::Cli.new([], cli_options) @@ -43,4 +64,18 @@ def configuration_from_cli(**cli_options) SolidQueue::Configuration.new(**options) end + + # capture_io re-raises SystemExit before returning the captured strings. + # It swallows it inside the block and returns its status alongside the IO. + def capture_check_run(*args) + status = nil + out, err = capture_io do + begin + SolidQueue::Cli.start([ "check", *args ]) + rescue SystemExit => e + status = e.status + end + end + [ out, err, status ] + end end diff --git a/test/unit/configuration_test.rb b/test/unit/configuration_test.rb index 34f69658..fc8fed2c 100644 --- a/test/unit/configuration_test.rb +++ b/test/unit/configuration_test.rb @@ -172,6 +172,54 @@ class ConfigurationTest < ActiveSupport::TestCase configuration.errors.full_messages.first end + test "skips DB-pool validation gracefully when skip_db_checks is set and the pool is unavailable" do + SolidQueue::Record.stubs(:connection_pool).raises(ActiveRecord::ConnectionNotEstablished) + + configuration = SolidQueue::Configuration.new( + workers: [ { queues: "background", threads: 50, polling_interval: 10 } ], + skip_recurring: true, + skip_db_checks: true + ) + + assert configuration.valid?, + "Expected configuration to be valid when DB pool is unavailable, got: #{configuration.errors.full_messages.inspect}" + end + + test "still raises DB connection errors during normal validation when skip_db_checks is not set" do + SolidQueue::Record.stubs(:connection_pool).raises(ActiveRecord::ConnectionNotEstablished) + + configuration = SolidQueue::Configuration.new( + workers: [ { queues: "background", threads: 50, polling_interval: 10 } ], + skip_recurring: true + ) + + assert_raises(ActiveRecord::ConnectionNotEstablished) { configuration.valid? } + end + + test "check! prints success message and returns true for a valid configuration" do + configuration = SolidQueue::Configuration.new(skip_recurring: true) + out = StringIO.new + err = StringIO.new + + assert configuration.check!(out: out, err: err) + assert_match "Solid Queue configuration is valid.", out.string + assert_empty err.string + end + + test "check! prints errors to err and returns false for an invalid configuration" do + configuration = SolidQueue::Configuration.new( + recurring_schedule_file: config_file_path(:recurring_with_invalid), + skip_db_checks: true + ) + out = StringIO.new + err = StringIO.new + + assert_not configuration.check!(out: out, err: err) + assert_empty out.string + assert_match "Invalid Solid Queue configuration:", err.string + assert_match "periodic_invalid_class", err.string + end + private def assert_processes(configuration, kind, count, **attributes) processes = configuration.configured_processes.select { |p| p.kind == kind } diff --git a/test/unit/rake_tasks_test.rb b/test/unit/rake_tasks_test.rb new file mode 100644 index 00000000..8b2de143 --- /dev/null +++ b/test/unit/rake_tasks_test.rb @@ -0,0 +1,50 @@ +# frozen_string_literal: true + +require "test_helper" +require "rake" + +class RakeTasksTest < ActiveSupport::TestCase + setup do + @previous_rake_application = Rake.application + @rake = Rake::Application.new + Rake.application = @rake + Rake::Task.define_task(:environment) + load File.expand_path("../../lib/solid_queue/tasks.rb", __dir__) + end + + teardown do + Rake.application = @previous_rake_application + end + + test "solid_queue:check exits 0 and prints OK message for a valid configuration" do + SolidQueue::Configuration.any_instance.stubs(:skip_recurring_tasks?).returns(true) + + out, err = capture_io do + assert_nothing_raised { @rake["solid_queue:check"].invoke } + end + + assert_match "Solid Queue configuration is valid.", out + assert_empty err + end + + test "solid_queue:check exits 1 and prints errors for an invalid configuration" do + SolidQueue::Configuration.any_instance.stubs(:invalid_tasks).returns( + [ stub(key: "broken", errors: stub(full_messages: [ "is invalid" ])) ] + ) + SolidQueue::Configuration.any_instance.stubs(:skip_recurring_tasks?).returns(false) + + status = nil + out, err = capture_io do + begin + @rake["solid_queue:check"].invoke + rescue SystemExit => e + status = e.status + end + end + + assert_equal 1, status + assert_empty out + assert_match "Invalid Solid Queue configuration:", err + assert_match "broken", err + end +end