Skip to content

Add log rotation for AZNFS logs and make the log directory configurable - #332

Open
rajasi3010 wants to merge 1 commit into
mainfrom
personal/rajasimandal/aznfs-logrotate
Open

rajasi3010 wants to merge 1 commit into
mainfrom
personal/rajasimandal/aznfs-logrotate

Conversation

@rajasi3010

@rajasi3010 rajasi3010 commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

aznfs.log and the per-mount turbo*.log files are never deleted, so on a long lived machine they grow until they fill the disk, which takes down everything on the box and not just AZNFS. The log location was also hardcoded, so logs could not be moved to another filesystem.

Rotation was always intended. nfsv3mountscript.sh already carried this note:

# We append to the logfile w/o any support for re-opening log file for log rotation. Use copytruncate option in logrotate config.

The config was simply never written. This PR adds it.

Logs produced by AZNFS, and their retention

Log Path Before After
Mount helper + watchdog <logdir>/aznfs.log never deleted, unbounded rotated
Turbo client (per mount) <logdir>/turbo<mountpoint>.log never deleted, unbounded rotated
stunnel (per storage IP) /etc/stunnel/microsoft/aznfs/nfsv4_fileShare/logs/ deleted on unmount unchanged
systemd journal journald journald limits unchanged

Only the first two grew unbounded and those are what this PR fixes. stunnel logs are already removed by cleanup_stunnel_files() together with their .conf and .pid, and are deliberately left alone since rotating them could orphan files that the cleanup matches by exact name.

Changes

New src/aznfs.logrotate, a template installed to /opt/microsoft/aznfs/ and expanded into /etc/logrotate.d/aznfs:

size 100M, rotate 7, compress, delaycompress, missingok, notifempty, copytruncate
  • copytruncate is required because aznfsclient holds its log open for the life of the mount and cannot reopen it after a rename.
  • Rotation is size based with no daily/weekly. These are diagnostic logs, so the retained history should scale with how much is logged rather than with elapsed time. logrotate only runs daily either way, so a time based policy adds no disk safety, it only caps the history at rotate days. Note that size makes logrotate ignore time directives, so the two must not be combined. Azure's own waagent and cloud-init are likewise pure size based.

Configurable log directory. AZNFS_LOGDIR in /opt/microsoft/aznfs/data/config, overridable per invocation with the env variable of the same name, defaulting to the previous location so existing installs are unaffected. Turbo logs follow it and AZNFSC_LOGDIR still takes precedence for them.

The value must be an absolute path built from safe characters. & and | would corrupt the generated config via sed, * and ? would turn the log paths into globs matching unrelated files, and a relative path would depend on the caller's cwd. Those, along with a directory that cannot be created or is not writable, fall back to the default with a warning rather than failing the mount.

The generated config is regenerated only when the log directory changes, keyed off an # AZNFS_LOGDIR: marker, so a locally edited rotation policy survives mounts and package upgrades.

Packaging. The repo has two packaging scripts, package.sh and generate_package.sh, and the Azure build pipeline runs the latter. Both now stage the template, for deb, rpm and tarball as applicable. The config is generated at install time so rotation applies from the first boot, logrotate is added to all three Requires: variants including azurelinux, and /etc/logrotate.d/aznfs is removed on uninstall.

Hardening. mount.aznfs is installed setuid root and execs the mount script with the caller's environment, so honouring a caller supplied log directory there would let an unprivileged user make root create and write files at a path of their choosing. It now drops AZNFS_LOGDIR and AZNFSC_LOGDIR alongside the BASH_ENV and LD_PRELOAD it already dropped. The log directory is therefore administrator controlled, through the root owned config file. AZNFSC_LOGDIR predates this PR but is the same class of issue.

Behaviour when the log directory is changed

Only the newly configured directory is rotated. Long running processes resolve their log file once at startup, so the watchdog services have to be restarted. AZNFS prints this on the first mount after the change:

AZNFS log directory changed to '/var/log/aznfs'.
Restart the watchdog services so that they log there too:
    sudo systemctl restart aznfswatchdog aznfswatchdogv4
Logs from before the change remain in '/opt/microsoft/aznfs/data' and are not removed automatically.

Logs left in the old directory are not moved or deleted. Since rotation is size based, a log that stops growing never crosses the threshold again, so those files stay until they are removed by hand. They do not grow, so they are not a risk of filling the disk.

Testing

testing/test_logrotate.sh, 388 sandboxed unit tests that need neither root nor a mount and never touch real system paths. They cover path resolution and validation, config parsing, config generation and idempotency, log directory changes, that every packaging path ships the template, that the setuid mount helper drops the log directory overrides, and the real logrotate behaviour, including copytruncate against a live writer verified by inode, and retention actually deleting aged out rotations. The retention test drives six real size triggered rotations with each generation tagged by a distinct digit, and asserts that only rotate files are kept, that the older content is gone from disk, and that the total size stays bounded.

testing/test_logrotate_e2e.sh, 31 checks against a real Azure Files NFSv4.1 mount on a machine with AZNFS installed. It backs up and restores everything it touches, including the logs, and removes its backup only once the restore has completed cleanly.

Both suites were run against this branch. The unit suite has 388 cases and enforces that floor, so a run that silently loses tests fails rather than passes: as an unprivileged user 380 pass and 8 skip. As root more cases skip, because root ignores the permission bits several of them depend on, for the same 388 total. The E2E ran twice: 29 passed / 0 failed / 4 skipped against a live NFSv4.1 share, and 33 passed / 0 failed / 0 skipped against a live NFSv3 Turbo share, which is what retires the four Turbo skips. They have not been run on main or release/3.0.

The guards were mutation tested, each one was verified to fail when the code it protects is deliberately broken. That is what the suite is worth, rather than the pass count: three checks were found during review to be passing vacuously and were rewritten.

Known gaps

  • Turbo log rotation is now tested live. A premium block blob account with NFSv3 enabled was created, mounted with -o vers=3,turbo, and the E2E run against it: the four Turbo assertions passed rather than skipped. The rotation check appends to the real aznfsclient log, rotates it with the live policy, and asserts the inode is unchanged, which is what lets the client keep writing through its open descriptor. What it still does not do is observe aznfsclient itself writing after the truncation: the inode check is the mechanism that permits it, not a direct observation of it. Running against a real Turbo mount also found a bug in the E2E's own mount point guard, which allowed only nfs/nfs4/aznfs and refused fuse.aznfsclient; nothing sandboxed could have caught that.
  • The RPM %post scriptlet is now extracted and executed by the unit suite, the same way the deb postinst already was, so a shell error in it fails the tests. rpm's macro expansion has since been checked with a real rpmbuild: a bare %u in a scriptlet body is rewritten when a macro named u is defined, which would make aznfs_safe_logdir reject every directory, so the spec's stat and printf formats are escaped as %% and verified to reach the installed script as %. The deb copy stays unescaped, since nothing collapses %% there. Still not done: no package is built or installed by the tests, so this is verified at the scriptlet level rather than through a real rpm -i.
  • Validated on Ubuntu 24.04 (bash 5.2, logrotate 3.21, rpm 4.18.2) and RHEL 10.2 (bash 5.2.26, logrotate 3.22, rpm 4.19.1.1, SELinux enforcing), each as root and as an unprivileged user. On RHEL the %post scriptlet was built into an rpm and aznfs_safe_logdir extracted back out of the package and run: it accepts a safe directory and refuses a world writable one, and the stat/printf formats survive a build with u, G and a all defined as hostile macros. logrotate accepted every directive in the shipped template and rotating under SELinux enforcing produced no new AVC denials. The other supported distros (SLES 15, CentOS, Rocky, Ubuntu 18/20/22) are untested. The log directory rule is permission driven, so its verdict on /var/log legitimately differs by distro: Debian ships it root:syslog 0775 and it is refused as a log directory, RHEL ships it root:root 0755 and it is accepted. A distro shipping /var/log group writable by a group other than root or syslog would refuse the recommended /var/log/aznfs; none of the supported ones is known to, but that has only been confirmed on the two tested.
  • The logrotate scriptlet has been installed by a real rpm -i on RHEL 10.2. The functions are copied verbatim out of the spec into a separate probe package, so rpm's macro expansion, its shell, its $1 and its failure handling all apply: the policy is created at install time, is valid to logrotate, has no unsubstituted placeholder, honours a configured directory with size 25M / rotate 4, and a world writable directory neither breaks the install nor gets rotated. What is still not installed is the full package: the systemd units, stunnel directories and the Turbo binary are excluded from the probe so it cannot disturb a real installation, and building them needs the C++ build. dpkg -i has since been run too, on Ubuntu 24.04, with the same probe approach: the deb postinst is built from the verbatim aznfs_safe_logdir and install_logrotate_config, so dpkg's shell, its $1 and its failure handling all apply. The policy is created at install time with no unsubstituted placeholder, honours a configured directory with size 2M / rotate 3, and the log directory is created root:root 0755. What is still untested on either packaging format is replacing a real installation: the probe carries no systemd units, stunnel directories or Turbo binary, and the box used has a live NFS mount on it that an upgrade would disturb.
  • Rotation and retention are now demonstrated end to end on two distros, against the policy the product's own ensure_logrotate_config() generates at its shipped size 100M / rotate 7. Nine rotations are driven by genuinely exceeding the threshold with a plain logrotate run, and each generation is tagged with a distinct byte so survivors can be identified by content. Rotations accumulate 1 to 7, stay at 7 through generations 8 and 9, and the survivors are exactly generations 3 to 9 with 1 and 2 gone from disk: deleted, not renamed. copytruncate keeps the inode and a descriptor opened before the rotation stays valid, which is what lets aznfsclient keep writing. Identical results on Ubuntu 24.04 (logrotate 3.21) and RHEL 10.2 (logrotate 3.22, gzip 1.13, SELinux enforcing, as root with the policy's su root root intact and no AVC denials). A negative control confirms an undersized log is not rotated, so the threshold really is the trigger, and the retention assertion requires the survivors to be exactly the newest rotate generations rather than merely that the old ones are absent, so an empty directory cannot satisfy it. The README's disk budget of AZNFS_LOGSIZE x (AZNFS_LOGCOUNT + 2) was checked against this: the measured worst case is x (AZNFS_LOGCOUNT + 1), with the extra multiple covering the copytruncate copy, so the documented figure is conservative rather than wrong.
  • The distro's own schedule is now exercised rather than assumed. After the real dpkg -i above, rotation is driven by systemctl start logrotate.service, the unit the daily timer activates, which runs /usr/sbin/logrotate /etc/logrotate.conf against the shared /var/lib/logrotate/status. An oversized log rotates through that path, the inode is preserved by copytruncate, the rotation is recorded in the shared state file, and retention caps at rotate across repeated service runs. A small log survives a service run, so the size rule and not the schedule is what fires. What remains unexercised is the timer firing on its own clock: the unit is started explicitly rather than waited for.
  • The log directory check exists in four hand-maintained copies, which has produced four separate drift bugs so far. The guard previously compared only the deb and rpm copies: scripts/aznfs_install.sh was outside it entirely, which is why a umask fix had to be applied to three files by hand, and safe_logdir() in common.sh, the copy that decides at mount time, was compared to nothing. All four are now checked and are currently in sync. Both new guards are mutation tested: drifting aznfs_install.sh alone fails, and changing the group allowlist in common.sh alone fails. Install time and runtime disagreeing is how the installer creates a directory the mount then refuses, which is the bug class this catches.
  • Review found a privilege escalation in the policy generator and it is fixed here. The staged file was created by redirecting onto the predictable /etc/.aznfs-logrotate.tmp.$$. mount.aznfs.c has no umask call, so the setuid helper runs with the caller's umask: under umask 000 root created that file 0666. An unprivileged caller cannot create files in /etc, but does not need to, only write to the one root just made, before the chmod and mv installed it as a logrotate config that root then executes directives from. It is now staged with mktemp, which is 0600 whatever the umask, in common.sh and both packaging copies. Measured: redirection gives 666 under umask 000 and 644 under umask 022, mktemp gives 600 in both cases.
  • The umask fix applied earlier was incomplete: the guarded mkdir got its chmod 0755 but the fallback to the default directory did not, in all three packaging files. A fresh install under umask 0002 that fell back would create a 0775 directory the runtime check then refuses. The original test only matched the guarded mkdir, so it stayed green; it now fails on any mkdir not paired with a chmod. This is the fifth bug from these four copies, and the first the equality-based drift guard could not have caught, because the fix was applied consistently and incompletely.
  • Two further hardening changes came out of review. The log directory was created with mkdir -p, which succeeds on an entry that already exists and follows it, so a member of a group that can write the parent could plant a symlink at the target between validation and creation. Parents now use -p and the final component does not, in all four copies, so a planted entry fails with EEXIST. Measured: mkdir -p on a planted symlink succeeds, plain mkdir reports File exists. Separately mount.aznfs.c never reset umask, so every file the script created as root took the caller's; it now calls umask(0022) before execv, asserted by running the real binary under umask 000 and reading back what the exec'd script sees.
  • Open question for maintainers. A residual race remains and cannot be closed in shell: once the directory is validated and created, a member of an allowlisted group can still rename it and substitute a symlink before root writes, because the check and the write are separate syscalls and shell has no O_NOFOLLOW. The only complete fix is to stop treating group-writable parents as safe, which would refuse /var/log/aznfs on Debian and Ubuntu where /var/log is root:syslog 0775 — the directory this README recommends. Left as-is pending a decision, rather than changing documented behaviour silently.
  • Paths containing spaces are rejected by design. Supporting them would require quoting many pre-existing unquoted usages across the mount scripts, which is out of scope here.
  • testing/*.sh are repo only developer tooling, neither packaging script ships them.
  • The log directory rule accepts a group writable parent when the group is a system group, because /var/log is root:syslog 0775 and refusing it made the documented AZNFS_LOGDIR=/var/log/aznfs silently fall back. This was verified against a sandbox parent owned by a system group, not against a real /var/log with a live mount.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Packaging paths, log-directory fallback handling, and E2E state restoration have unresolved issues.

Review effort: Lite
Findings: 3 High severity · 2 Medium severity · 1 Low severity

Open (6)
What changed in this PR

Adds configurable AZNFS/Turbo log directories and size-based log rotation, with packaging, documentation, and test coverage.

Changes:

  • Validates AZNFS_LOGDIR with fallback behavior.
  • Generates compressed, copy-truncated rotation rules.
  • Updates installers, packages, documentation, and unit/E2E tests.
File Description
testing/​test_logrotate.sh Sandboxed unit and rotation tests
testing/​test_logrotate_e2e.sh Live-mount end-to-end rotation tests
src/​nfsv3mountscript.sh Configurable Turbo log directory
src/​aznfs.logrotate Logrotate policy template
scripts/​aznfs_install.sh Installer log-path handling
README.md Logging and rotation documentation
packaging/​aznfs/​RPM/​aznfs.spec RPM packaging and rotation setup
packaging/​aznfs/​DEBIAN/​postrm Rotation configuration cleanup
packaging/​aznfs/​DEBIAN/​postinst Debian rotation setup
packaging/​aznfs/​DEBIAN/​control Logrotate dependency
package.sh Template inclusion in packages
lib/​common.sh Log-path resolution and rotation generation

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread package.sh
Comment thread packaging/aznfs/DEBIAN/postinst Outdated
Comment thread packaging/aznfs/RPM/aznfs.spec
Comment thread lib/common.sh Outdated
Comment thread testing/test_logrotate_e2e.sh
Comment thread lib/common.sh Outdated
@rajasi3010
rajasi3010 force-pushed the personal/rajasimandal/aznfs-logrotate branch 2 times, most recently from b50827d to cf9fe20 Compare September 27, 2026 18:14
@rajasi3010
rajasi3010 requested a lite review from Copilot September 27, 2026 18:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread lib/common.sh
Comment thread lib/common.sh Outdated
@rajasi3010
rajasi3010 force-pushed the personal/rajasimandal/aznfs-logrotate branch from cf9fe20 to e496c34 Compare September 27, 2026 18:35
@rajasi3010
rajasi3010 requested a lite review from Copilot September 27, 2026 18:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate issues affect cleanup safety, configuration generation, path validation, and test portability.

Review effort: Lite
Findings: 1 High severity · 2 Medium severity

Open (3)
Resolved since last review (2)

Comment thread testing/test_logrotate_e2e.sh Outdated
Comment thread packaging/aznfs/DEBIAN/postinst Outdated
Comment thread packaging/aznfs/RPM/aznfs.spec Outdated
@rajasi3010
rajasi3010 force-pushed the personal/rajasimandal/aznfs-logrotate branch from e496c34 to d17383f Compare September 27, 2026 18:44
@rajasi3010
rajasi3010 requested a lite review from Copilot September 27, 2026 18:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate issues remain in runtime validation, failure handling, and E2E test safety.

Review effort: Lite
Findings: 2 High severity · 1 Medium severity

Open (3)
Resolved since last review (3)

Comment thread lib/common.sh Outdated
Comment thread testing/test_logrotate_e2e.sh Outdated
Comment thread lib/common.sh
@rajasi3010
rajasi3010 force-pushed the personal/rajasimandal/aznfs-logrotate branch from d17383f to bc56f67 Compare September 27, 2026 18:53
@rajasi3010
rajasi3010 requested a lite review from Copilot September 27, 2026 18:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved moderate test coverage and cleanup issues remain.

Review effort: Lite
Findings: 1 Medium severity · 1 Low severity

Open (2)
Resolved since last review (3)

Comment thread testing/test_logrotate_e2e.sh Outdated
Comment thread README.md Outdated
@rajasi3010
rajasi3010 force-pushed the personal/rajasimandal/aznfs-logrotate branch from bc56f67 to ade5fd8 Compare September 27, 2026 19:02
@rajasi3010
rajasi3010 requested a lite review from Copilot September 27, 2026 19:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Address the stale Turbo log policy and the three E2E cleanup, interruption, and permission-restoration issues.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
Resolved since last review (2)

Comment thread lib/common.sh
@rajasi3010
rajasi3010 force-pushed the personal/rajasimandal/aznfs-logrotate branch from ade5fd8 to f552975 Compare September 27, 2026 19:23
@rajasi3010
rajasi3010 requested a lite review from Copilot September 27, 2026 19:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Fix the invalid configured-directory fallback and make --repair-logs preserve legitimate rotated logs.

Review effort: Lite
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread testing/test_logrotate_e2e.sh Outdated
@rajasi3010
rajasi3010 force-pushed the personal/rajasimandal/aznfs-logrotate branch from f552975 to fdc739a Compare September 27, 2026 19:32
@rajasi3010
rajasi3010 requested a lite review from Copilot September 27, 2026 19:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread testing/test_logrotate.sh Outdated
Comment thread testing/test_logrotate_e2e.sh Outdated
Comment thread testing/test_logrotate_e2e.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved validation, tarball installation, and test-suite issues remain, including a critical self-check failure.

Review effort: Lite
Findings: 1 High severity

Open (1)
Resolved since last review (3)

Comment thread testing/test_logrotate.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The E2E alternate-path test is not exercising the intended behavior, and valid placeholder-containing paths can generate incorrect rotation policies.

Review effort: Lite
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread testing/test_logrotate_e2e.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved issues remain in tarball installation, directory permissions, CI coverage, and E2E validation.

Review effort: Lite
Findings: 2 Low severity

Open (2)
Resolved since last review (1)

Comment thread lib/common.sh Outdated
Comment thread src/aznfs.logrotate Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Address dangling symlink handling and generate the rotation policy for tarball installations.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
Resolved since last review (2)

Comment thread package.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Unresolved log-directory fallback, packaging error handling, and E2E reliability issues remain.

Review effort: Lite
Findings: None

Resolved since last review (1)

Comment thread lib/common.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical log-directory trust validation issues and additional test and cleanup defects remain unresolved.

Review effort: Lite
Findings: 4 High severity · 1 Medium severity

Open (5)

Comment thread lib/common.sh Outdated
Comment thread packaging/aznfs/DEBIAN/postinst
Comment thread packaging/aznfs/RPM/aznfs.spec
Comment thread scripts/aznfs_install.sh
Comment thread testing/test_logrotate.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Fix the skipped sticky-parent assertion and stop watchdog services before restoring E2E logs.

Review effort: Lite
Findings: 4 High severity · 1 Medium severity

Open (5)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread lib/common.sh
Comment thread packaging/aznfs/DEBIAN/postinst
Comment thread packaging/aznfs/RPM/aznfs.spec
Comment thread testing/test_logrotate_e2e.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread lib/common.sh Outdated
Comment thread packaging/aznfs/DEBIAN/postinst
Comment thread packaging/aznfs/RPM/aznfs.spec
Comment thread scripts/aznfs_install.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread lib/common.sh Outdated
Comment thread packaging/aznfs/DEBIAN/postinst Outdated
Comment thread packaging/aznfs/RPM/aznfs.spec Outdated
Comment thread scripts/aznfs_install.sh Outdated
Comment thread scripts/aznfs_install.sh Outdated
Comment thread packaging/aznfs/DEBIAN/postinst
Comment thread packaging/aznfs/RPM/aznfs.spec
Comment thread src/nfsv3mountscript.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread lib/common.sh
Comment thread scripts/aznfs_install.sh
Comment thread src/nfsv3mountscript.sh
Comment thread src/nfsv3mountscript.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved log-directory safety and rotation fallback issues remain, along with a test-count documentation mismatch.

Review effort: Lite
Findings: 4 High severity · 1 Medium severity

Open (5)
Resolved since last review (1)

Comment thread lib/common.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Fix the configured-directory fallback mismatch and the undefined test assertion helper.

Review effort: Lite
Findings: 1 High severity · 1 Medium severity

Open (2)
Resolved since last review (4)

Comment thread testing/test_logrotate.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Fix fallback rotation state and warn when installer settings are rejected.

Review effort: Lite
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Two moderate security and installation issues, plus one documentation nit, remain unresolved.

Review effort: Lite
Findings: None

Resolved since last review (1)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical E2E mount-target safety and moderate validation/cleanup issues remain unresolved.

Review effort: Lite
Findings: 1 High severity

Open (1)

Comment thread testing/test_logrotate_e2e.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved test-validity defects and a critical E2E log-directory validation gap remain.

Review effort: Lite
Findings: 1 High severity · 1 Medium severity · 1 Low severity

Open (3)
Resolved since last review (1)

Comment thread testing/test_logrotate_e2e.sh Outdated
Comment thread testing/test_logrotate.sh Outdated
Comment thread README.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Moderate fallback-path safety issues and broken test coverage remain unresolved.

Review effort: Lite
Findings: None

Resolved since last review (3)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical root-equivalent log-path validation bypasses and additional E2E safety and coverage issues remain unresolved.

Review effort: Lite
Findings: 4 High severity · 2 Medium severity

Open (6)

Comment thread lib/common.sh
Comment thread packaging/aznfs/DEBIAN/postinst
Comment thread packaging/aznfs/RPM/aznfs.spec
Comment thread scripts/aznfs_install.sh
Comment thread testing/test_logrotate_e2e.sh
Comment thread testing/test_logrotate_e2e.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread testing/test_logrotate.sh Outdated
Comment thread testing/test_logrotate_e2e.sh Outdated
Comment thread testing/test_logrotate_e2e.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate findings affect test validity, directory creation, and log-directory security.

Review effort: Lite
Findings: 1 High severity · 3 Medium severity

Open (4)
Resolved since last review (4)

Comment thread testing/test_logrotate.sh
Comment thread packaging/aznfs/DEBIAN/postinst Outdated
Comment thread packaging/aznfs/RPM/aznfs.spec Outdated
Comment thread scripts/aznfs_install.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved path-permission, tarball installation-order, CI coverage, and E2E validation issues remain.

Review effort: Lite
Findings: 1 Low severity

Open (1)
Resolved since last review (4)

Comment thread testing/test_logrotate.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

Review effort: Lite
Findings: 1 High severity · 3 Medium severity · 2 Low severity

Open (6)

Comment thread testing/test_logrotate_e2e.sh
Comment thread scripts/aznfs_install.sh Outdated
Comment thread src/nfsv3mountscript.sh Outdated
Comment thread testing/test_logrotate_e2e.sh Outdated
Comment thread testing/test_logrotate.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread lib/common.sh Outdated
Comment thread testing/test_logrotate.sh
Comment thread testing/test_logrotate_e2e.sh Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Comment thread scripts/aznfs_install.sh
Comment thread testing/test_logrotate_e2e.sh
Comment thread testing/test_logrotate_e2e.sh Outdated
Comment thread testing/test_logrotate_e2e.sh
aznfs.log and the per-mount turbo*.log files were never deleted, so on a
long lived machine they grow until they fill the disk. The log location was
hardcoded as well, so logs could not be moved to another filesystem.

Rotation was always intended, nfsv3mountscript.sh already carried the note
"We append to the logfile w/o any support for re-opening log file for log
rotation. Use copytruncate option in logrotate config.", the config was just
never written. This adds it.

Rotation policy (src/aznfs.logrotate, installed as a template and expanded
into /etc/logrotate.d/aznfs):

  - size and retention come from AZNFS_LOGSIZE and AZNFS_LOGCOUNT in
    /opt/microsoft/aznfs/data/config, defaulting to 100M and 7. The size is
    logrotate's own syntax, a number with an optional k/M/G suffix; zero is
    refused since "size 0" rotates on every run. A count of zero is allowed
    and means keep nothing. An unusable value falls back to the default with
    a warning rather than emitting a policy logrotate would reject.
  - compressed
  - copytruncate, since aznfsclient holds its log open for the life of the
    mount and cannot reopen it after a rename
  - purely size based, no daily/weekly. These are diagnostic logs, so the
    retained history should scale with how much is logged rather than with
    wall clock time. logrotate only runs daily either way, so a time based
    policy adds no disk safety, it only caps the history at 'rotate' days.
    Note that "size" makes logrotate ignore time directives, so the two must
    not be combined.

Configurable log directory:

  - AZNFS_LOGDIR in /opt/microsoft/aznfs/data/config, overridable per
    invocation with the AZNFS_LOGDIR env variable, defaulting to the
    previous location so existing installs are unaffected
  - turbo logs follow it, AZNFSC_LOGDIR still takes precedence
  - the value must be an absolute path built from safe characters. '&' and
    '|' would corrupt the generated config via sed, '*' and '?' would turn
    the log paths into globs matching unrelated files, and a relative path
    would depend on the caller's cwd. Such values, an uncreatable directory
    and a directory that exists but is not writable all fall back to the
    default with a warning rather than failing the mount.
  - the generated config is regenerated only when the log directory or
    either limit changes, keyed off "# AZNFS_LOGDIR:" and
    "# AZNFS_LOGPOLICY:" markers that record what it was generated for
    rather than what it currently says, so a locally edited rotation policy
    survives mounts and package upgrades
  - the log directory itself must be root owned and not group or other
    writable. Every directory above it must be root owned and not world
    writable, sticky or not: sticky protects entries that already exist, so
    it does nothing for a directory that does not exist yet, where whoever
    can write to the parent creates it, or a symlink in its place, first.
    That rules out /tmp and /var/tmp. A group writable parent is allowed
    only when the group is root or syslog, since /var/log is root:syslog
    0775 and refusing it would make the documented
    AZNFS_LOGDIR=/var/log/aznfs silently fall back. adm is deliberately not
    trusted, on Ubuntu it contains the login user. A member of an allowed
    group can still rename the directory between the check and the write;
    closing that needs the log created through a held directory descriptor
    with no-follow semantics, which a shell cannot do, so the trusted set is
    limited to daemon accounts. No
    component may be a symlink, checked explicitly rather than relying on a
    link's 0777 mode failing the checks above, and the log file itself is
    held to the same rule: -w follows a link and reports on its target, so a
    symlinked aznfs.log aimed at something writable would otherwise be
    appended to as root.
  - the path is checked as written rather than resolved first, since
    resolving it would judge a planted link by its target, and it is
    validated before anything is created as well as afterwards, since
    mkdir -p follows a symlinked component. The logs are written as root and
    aznfsclient holds its log open with ">>", which cannot be made to refuse
    a symlink, so the only durable protection is that nobody else can plant
    anything along that path. The installer and both packaging scriptlets
    apply the same rule.

Only the configured directory is rotated. Long running processes resolve the
log file once at startup, so the watchdog services have to be restarted after
changing AZNFS_LOGDIR, as documented in the README. Logs left in a previous
directory are not removed automatically, AZNFS logs where they are.

Packaging ships the template for deb, rpm and tarball, depends on logrotate
and removes /etc/logrotate.d/aznfs on uninstall. The deb and rpm maintainer
scripts render the config at install time, so rotation is in place from the
first boot. The tarball has no maintainer script and relies on common.sh,
which renders it the first time it is sourced, in the same run that creates
the log file, so a log is never growing without a policy covering it. The rpm %post is declared
with /bin/bash: it was already relying on bash only syntax, and the log
directory check added here uses process substitution, which fails outright
under a /bin/sh that is dash.

Tests:

  - testing/test_logrotate.sh, 388 sandboxed unit tests needing neither root
    nor a mount, covering path resolution and validation, config parsing,
    config generation and idempotency, log directory changes, and the real
    logrotate behaviour including copytruncate against a live writer and
    retention actually deleting aged out rotations. The suite is run both as an
    unprivileged user and as root, since root ignores the permission bits
    several cases depend on, and mount.aznfs only reaches execv as root
  - the deb postinst and the rpm %post logrotate scriptlets are both
    extracted and executed against the sandbox, rather than only inspected,
    so a shell error in either is caught here instead of at install time.
    The rpm copy escapes its stat and printf formats as %% because rpm
    expands macros in scriptlet bodies: verified with rpmbuild that a bare
    %u is rewritten when a macro named u is defined, and that %%u reaches
    the installed script as %u.
  - testing/test_logrotate_e2e.sh, 31 checks against a real Azure NFS mount
    on a machine with AZNFS installed, backing up and restoring everything it
    touches. Run on an NFSv4.1 share (29 passed, 0 failed, 4 skipped) and on a
    live NFSv3 Turbo share (33 passed, 0 failed, 0 skipped), the latter
    retiring the Turbo skips. It removes its backup on a clean restore and
    keeps it only when restoration was incomplete.

    Two checks were added after those runs and the suite has not been re-run
    since: every rotation it performed used "logrotate -f", which ignores the
    size threshold entirely (under -f even a 1-byte log rotates), and the one
    section labelled non-forced used "logrotate -d", a dry run that changes
    nothing. So nothing in it had shown that crossing "size" is what causes a
    rotation, which is the only trigger in production. The new checks grow the
    live log past the configured threshold and run logrotate with neither flag.
    Their logic was exercised standalone against the generated policy and both
    assertions were mutation tested: dropping copytruncate fails the inode
    check, and making the directory unwritable fails the rotation check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Unresolved path-safety, installer error-handling, and E2E test-isolation issues remain.

Review effort: Lite
Findings: 1 High severity

Open (1)
Resolved since last review (4)

Comment on lines +151 to +168
# Split on / only, with globbing off: unquoted ${mp//\// } would word split
# "/safe dir/target" into unrelated tokens and walk neither of them, and a
# component containing * would expand against the CWD.
#
mp_oldifs=$IFS
IFS=/
set -f
mp_walk=
for mp_part in $mp; do
[ -z "$mp_part" ] && continue
mp_walk="$mp_walk/$mp_part"
if [ -L "$mp_walk" ]; then
IFS=$mp_oldifs
set +f
echo "Mount point component '$mp_walk' is a symlink, refusing '$MOUNT_POINT'."
exit 1
fi
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants