Skip to content

[Bug]: Stale oc_job_runs entry causes false "Last background job execution ran more than one hour ago" warning and persistent running job #62659

Description

@PatrickKnoedler

⚠️ This issue respects the following points: ⚠️

Bug description

Stale oc_job_runs entry causes false "Last background job execution ran more than one hour ago" warning and persistent running job
1.Bug description
Steps performed

This installation was upgraded and afterwards thoroughly repaired using the standard Nextcloud maintenance commands.

The following repairs were successfully completed:

occ maintenance:repair
occ maintenance:repair --include-expensive
occ db:add-missing-indices
occ db:add-missing-columns

Database integrity is clean and the instance is operating normally.

Background jobs are configured in Cron mode.

Problem

The Administration Overview repeatedly reports:

The last background job execution ran more than one hour ago. Something seems wrong.

although background jobs are executed successfully.

background-job:history shows many successfully completed jobs.

However,

php occ background-job:running

always reports one permanently running job:

OCA\UpdateNotification\BackgroundJob\UpdateAvailableNotifications

whose runtime continuously increases.

Investigation

The corresponding job in oc_jobs was

id = 29882
class = OCA\UpdateNotification\BackgroundJob\UpdateAvailableNotifications

last_run = 1785310259
last_checked = 1785310259
reserved_at = 1785310259

execution_duration = 87 ms

Deleting the job using

php occ background-job:delete 29882

removed it from

php occ background-job:list

but it still appeared in

php occ background-job:running

This suggested that the running-job information was no longer coming from oc_jobs.

Investigation of oc_job_runs

The table contained an apparently orphaned run:

run_id = 111812855399272448
class_id = 111812855369912320
pid = 3845857
status = 0
duration = 0
ram_peak_usage = 0

The corresponding job no longer existed in oc_jobs.

Temporary workaround

Deleting only this stale row

DELETE FROM oc_job_runs
WHERE run_id = 111812855399272448;

immediately fixed the issue.

Afterwards

php occ background-job:running

returned no running jobs.

Running

php occ cron.php

completed normally and

php occ background-job:running

still returned no running jobs.

The Administration Overview immediately changed to

Last job ran a few seconds ago.

Problem reappears

After some time the Administration Overview again reports

The last background job execution ran more than one hour ago.

This suggests that another stale entry is created in oc_job_runs.

The issue therefore appears to be reproducible.

Expected behaviour

If a background job terminates unexpectedly, orphaned entries in oc_job_runs should not permanently affect

background-job:running
the Administration Overview
Cron health detection

There should either be

automatic cleanup,
timeout handling,
or orphan detection.
Environment
Nextcloud 34.0.2
PHP 8.3
MariaDB 10.11.14
Cron background jobs
Single server installation
Shared hosting (ALL-INKL)
Additional observations

While the warning is present:

background-job:history contains successful executions.
background-job:running reports exactly one running job indefinitely.
The runtime keeps increasing.
The corresponding oc_job_runs row contains
status = 0
duration = 0
ram_peak_usage = 0

Removing that row immediately clears the warning.

However, the problem eventually returns.

Questions
Is status = 0 in oc_job_runs expected to represent a running job indefinitely?
Is there any automatic cleanup for orphaned oc_job_runs records?
Could this be a regression introduced with the new oc_job_runs infrastructure in Nextcloud 34?
Files available on request

I can provide:

nextcloud.log
SQL dump of oc_job_runs
SQL dump of oc_jobs
complete occ outputs
additional debugging information

Steps to reproduce

  1. Install or upgrade to Nextcloud 34.0.2.
  2. Configure background jobs to use Cron.
  3. Let the system run normally for some time.
  4. Open Administration settings → Overview.
  5. Observe that the warning "The last background job execution ran more than one hour ago." appears although cron jobs are executed successfully.
  6. Run php occ background-job:history and verify that background jobs have completed successfully.
  7. Run php occ background-job:running and observe that OCA\UpdateNotification\BackgroundJob\UpdateAvailableNotifications is reported as running indefinitely.

Expected behavior

When background jobs are executed successfully via Cron, the Administration Overview should report the correct execution time.

php occ background-job:running should only list jobs that are actually running.

If a background job terminates unexpectedly, stale entries in oc_job_runs should be cleaned up automatically or ignored after an appropriate timeout. Orphaned oc_job_runs records should not cause the Administration Overview to permanently report that background jobs have not run.

Nextcloud Server version

34

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.3

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Updated from a MINOR version (ex. 32.0.1 to 32.0.2)

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

List of activated Apps

Nextcloud Signing status

Nextcloud Logs

Additional info

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    0. Needs triagePending check for reproducibility or if it fits our roadmap34-feedbackbug

    Type

    Projects

    Status
    To triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions