feat(sftp): copy-data extension + 16MB channel window + keepAlive config - #273
Open
JDis03 wants to merge 3 commits into
Open
feat(sftp): copy-data extension + 16MB channel window + keepAlive config#273JDis03 wants to merge 3 commits into
JDis03 wants to merge 3 commits into
Conversation
SFTP v3 has no COPY operation in its base message set — the standard mechanism for this is SSH_FXP_EXTENDED (type 200), and OpenSSH 9.0+ (April 2022) defines the "copy-data" extension for exactly this: copy bytes between two open file handles entirely on the server (e.g. via copy_file_range() on Linux), no shell/exec involved. Works even for accounts restricted to internal-sftp with no shell access. - SftpClientImpl.create(): parse the extension-name/extension-data string pairs trailing the SSH_FXP_VERSION handshake reply (was previously ignored entirely — only the 4-byte version int was read). Parsed defensively: a malformed trailing pair stops parsing rather than failing the whole handshake, since extensions are optional. - New SftpClient.extensions: Set<String> exposing the parsed names (e.g. "copy-data", "posix-rename@openssh.com"). - New SftpClient.copyData(srcHandle, srcOffset, length, dstHandle, dstOffset) sending the "copy-data" SSH_FXP_EXTENDED request. Caller must have both handles already open; length=0 means "through EOF". - Regenerated sshlib/api.txt via metalava. Tests: - SftpClientImplTest: 4 new unit tests (extension-pair parsing with/ without extensions, copyData wire-payload correctness, OP_UNSUPPORTED error mapping) using the existing FakeSshSession harness. - SftpClientIntegrationTest: 2 new Docker tests against a real OpenSSH 9.9p2 server — confirms it actually advertises "copy-data" in its real VERSION reply, and that copyData() correctly copies bytes server-side (byte-for-byte verified). - SftpClientTest.FakeSftpClient: updated for the new interface members. - Full :sshlib:test suite (all Docker-backed SSH/SFTP/port-forwarding/ compat tests) still green. Bumped version 0.3.2-SNAPSHOT -> 0.3.3-SNAPSHOT, published to mavenLocal for the consuming clientssh app.
…nd-trip stall per 64KB)
sshj provides two keepalive mechanisms: - Heartbeater: SSH_MSG_IGNORE (no response expected) — anti-NAT/VPN idle timeout - KeepAliveRunner: keepalive@openssh.com GLOBAL_REQUEST — real dead-connection detection cbssh had NO keepalive mechanism. Long-lived connections behind VPNs/NATs/firewalls would die silently when the intermediary closed the TCP socket for being idle (typical timeout: 60-300s). This PR adds the Heartbeater equivalent: - New config field: keepAliveIntervalMs (Long, default 0 = disabled) - After successful authentication, a background coroutine sends SSH_MSG_IGNORE every N ms via SshConnection.writeIgnore() - Coroutine cancelled on disconnect() - Backward compatible: 0 default = no behaviour change Recommended: 15000 (15s, sshj default).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Three related SFTP/SSH improvements developed and battle-tested in a real Android SSH client (DarkSSH):
1. SFTP
copy-dataextension (server-side copy)SFTP v3 has no COPY operation in its base message set. OpenSSH 9.0+ (April 2022) defines the
copy-dataextension (SSH_FXP_EXTENDED, type 200) for copying bytes between two open file handles entirely on the server — no shell/exec involved, works even for accounts restricted to internal-sftp with no shell access.This enables server-side file copy/paste and move fallback in the app without downloading/uploading the data.
2. SFTP session channel window: 64KB → 16MB
The default
openSessionChannel()values (64KB initial window, 32KB max packet) caused severe SFTP download stalls: the server sends 64KB, waits for SSH_MSG_CHANNEL_WINDOW_ADJUST, sends 64KB, waits... One full round-trip per 64KB of transfer.Measured on a real Android device over WiFi: a pipelined read of a 712MB file went from ~6MB/s to ~30MB/s with the large window. (16MB matches what sshj uses;
maxPacketSizestays at 32KB because 256KB causedChannelClosedExceptionwhen the server responded to a large SFTP READ with an oversized SSH_MSG_CHANNEL_DATA.)3.
keepAliveIntervalMsconfig for SSH_MSG_IGNORE heartbeatsA periodic SSH_MSG_IGNORE (no response expected) to keep connections alive across NAT/VPN/firewall idle timeouts. Default 0 (disabled). Recommended for long-lived connections (e.g. background SFTP transfers).
Testing
./gradlew buildpasses: test + spotless + metalavaCheckCompatibility + koverVerifyChanges
sshlib/api.txt: registerSftpClient.copyData,SftpClient.extensions,SshClientConfig.keepAliveIntervalMsSshClientConfig: addkeepAliveIntervalMs(default 0 = disabled)SshClient: openSftp keeps callingopenSessionChannel()unchanged (no API change for callers)SshConnection:openSessionChanneldefaultinitialWindowSize64KB → 16MB (documented)SftpClient/SftpClientImpl:copyData+extensions(SSH_FXP_EXTENDED copy-data)