From e9076c4f32d7215ed6c7092833199b2270eb2cc8 Mon Sep 17 00:00:00 2001 From: Dzulfikar Ats Tsauri Date: Sun, 6 Sep 2026 18:06:59 +0700 Subject: [PATCH 1/4] =?UTF-8?q?feat:=20reclaim=20=E2=80=94=20find,=20weigh?= =?UTF-8?q?=20and=20trash=20regenerable=20trees?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A reclaim request walks a root for regenerable directories (node_modules, target, dist, build, .venv, __pycache__ and friends), sizes each one with the dirsize walker, seeds the dirsize cache with what it measured and answers a listing ranked heaviest first. A result is a listing like any other, so window, thumb, trash and undo work unchanged; rows the walk never sized are sized on demand. On a reclaim listing a directory row's s is the measured bytes, which is what the views read. The GUI: R scans the directory the pane stands in, b opens and cycles the eight readings — treemap, folders, sunburst, flame, bubbles, mind map, top sizes, age map — drawn by ui/js/ReclaimTree.js's layouts and ui/js/ ReclaimPaint.js's one drawer, over the same tree built from the rows. The strip's Quick Wins stages every listed tree for the trash through the existing trash request, so undo covers it. The walk is the search walk's shape: bounded slices, one match sized per tick, a cancel that is never behind more than one tree, streaming progress reclaiming lines and a terminal reclaimed line. Matches are never descended into, symlinks are never matched, other filesystems are never entered. docs/protocol.md documents the wire; tests/protocol.sh drives the real backend through eleven reclaim checks and Reclaim.rs carries nine sandbox unit tests. --- docs/protocol.md | 64 +++++++ keys.toml | 24 +++ src/backend/mod.rs | 2 + src/backend/proto.rs | 19 ++ src/backend/reclaim.rs | 374 ++++++++++++++++++++++++++++++++++++ src/backend/reclaimreq.rs | 61 ++++++ src/backend/run.rs | 53 +++++- src/backend/state.rs | 9 + tests/js/keymap.js | 2 +- tests/protocol.sh | 37 ++++ ui/Backend.qml | 19 ++ ui/Pane.qml | 4 + ui/PaneWire.qml | 34 ++++ ui/ReclaimMap.qml | 188 +++++++++++++++++++ ui/js/Focus.js | 16 +- ui/js/Keymap.js | 6 + ui/js/Nav.js | 3 + ui/js/Reclaim.js | 74 ++++++++ ui/js/ReclaimPaint.js | 105 +++++++++++ ui/js/ReclaimTree.js | 385 ++++++++++++++++++++++++++++++++++++++ ui/shell.qml | 29 ++- 21 files changed, 1499 insertions(+), 9 deletions(-) create mode 100644 src/backend/reclaim.rs create mode 100644 src/backend/reclaimreq.rs create mode 100644 ui/ReclaimMap.qml create mode 100644 ui/js/Reclaim.js create mode 100644 ui/js/ReclaimPaint.js create mode 100644 ui/js/ReclaimTree.js diff --git a/docs/protocol.md b/docs/protocol.md index 1e161802..5ac031ed 100644 --- a/docs/protocol.md +++ b/docs/protocol.md @@ -292,6 +292,70 @@ sends this. A row already answered is untouched; only the queue is cleared. No r A directory a `dirsizecancel` dropped can be asked for again straight away: cancelling forgets the row, so a later `dirsize` for it queues fresh work. +### reclaim + +`{"c":"reclaim","path":""}` + +Example: `{"c":"reclaim","path":"/home/gm/Documents"}` + +Walks the whole subtree under `path` looking for regenerable directories, sizes each one, and +answers a listing of the matches ranked heaviest first. The walk is the search walk's own +machinery with a different matcher: the same three-part answer, the same bounded slices, and a +result that is a listing like any other, so `window`, `thumb`, `trash` and every per-row +facility keep working unchanged. Each match is pushed as its path relative to `path`, and +`path` becomes the listing's base, exactly as a search's is. + +**A match is a directory whose base name is exactly one of a fixed set.** The set is build +systems' and package managers' regenerable trees — `node_modules`, `.next`, `.nuxt`, `.output`, +`.turbo`, `.parcel-cache`, `target`, `dist`, `build`, `out`, `.venv`, `venv`, `__pycache__`, +`.pytest_cache`, `.mypy_cache`, `.ruff_cache`, `.gradle`, `.terraform`, `.dart_tool`, +`.sass-cache`, `coverage` — matched case sensitively and by base name only. A match is reported +but never descended into: its own size already covers anything nested, and a monorepo's nested +`node_modules` would otherwise answer twice. There is no hidden flag on a reclaim — `.venv` and +`__pycache__` are the point, so dot directories are always descended. + +Every entry that is not a match and is a directory is descended, with one refusal: a directory +on another filesystem is never entered, so a scan rooted at a home directory stops at each +mount rather than reading a network share by accident. A symlink reports its own type, so a +link named like a target is never matched and never descended, and no loop is possible. An +unreadable directory is skipped in silence, the same way `search` skips one, and a `path` that +cannot be read at all finishes at once with nothing. + +Each match is sized with the same walk the `dirsize` answer uses, one match per slice so a +cancel is never behind more than one tree, and a size that ran past that walk's 2000 ms +deadline is marked partial: a floor, not a wrong exact number. The answers are also seeded into +the same answered-row cache `dirsize` reads, so a client that asks for the visible rows' sizes +after the scan is answered from that cache instead of walking every listed tree a second time. +Rows are ranked by bytes descending, ties by the listing's own name order, when the walk ran to +its end; a cancelled walk keeps its rows in discovery order, and the rows it never sized are +sized on demand by `dirsize` like any other directory row. + +A window on a reclaim listing carries the walk's own numbers: a directory row's `s` is the +measured bytes of its tree, not the directory's dirent size, which is what the scan's views read +and what the `dirsize` cache already answers with. A row the walk never sized — a cancelled +walk's tail — keeps its stat. + +The walk is answered in three parts, the same shapes a `search` answers: + +1. One `listed` with `n` of 0, immediately, because the client's old rows are gone the + moment the request is read. +2. `reclaiming` lines carrying the match count, the entries scanned and the bytes sized so + far, at most one every 100 ms while the walk runs. +3. One terminal `reclaimed` line, written after the ranking, carrying the final counts and + the bytes total the rows cover. + +`list`, `sort` and `search` each end a running reclaim before they touch the listing, and +answer their own lines after the reclaim's terminal `reclaimed`. + +### reclaimcancel + +`{"c":"reclaimcancel"}` + +Stops the running reclaim and answers one `reclaimed` line with `cancelled` true. Whatever the +walk already found stays in the listing, ranked when every row carried a size and in discovery +order otherwise, and stays windowable. One reclaim runs at a time, so a cancel can only mean +that one; a `reclaimcancel` with no walk running does nothing and answers nothing. + ### transfer `{"c":"transfer","op":"","paths":["",...],"dest":""}` diff --git a/keys.toml b/keys.toml index 6515c7ff..67c8067a 100644 --- a/keys.toml +++ b/keys.toml @@ -337,6 +337,20 @@ action = "filter" char = "f" action = "search" +# R scans the directory the pane stands in for regenerable trees — node_modules, target, the +# framework build directories — ranks them by size and lists them, where dd trashes a tree and z +# undoes. The capital-is-the-variant shape g/G and s/S already use: r renames a row, R weighs the +# directory. ui/js/Reclaim.js owns the scan. +[[text]] +char = "R" +action = "reclaim" + +# The scan's views: treemap, folders, sunburst, flame, bubbles, mind map, top sizes, age map. +# b means nothing outside reclaim results; ui/js/Focus.js drops it everywhere else. +[[text]] +char = "b" +action = "reclaimMap" + # Only means something on a search result, so ui/js/Focus.js drops it everywhere else and o stays type-ahead. [[text]] char = "o" @@ -537,11 +551,21 @@ keys = "r" action = "rename" label = "rename" +[[sheet]] +keys = "R" +action = "reclaim" +label = "find regenerable trees" + [[sheet]] keys = "dd" action = "trashArm" label = "trash" +[[sheet]] +keys = "R b" +action = "reclaim" +label = "scan trees, then views" + [[sheet]] keys = "z ^z" action = "undo" diff --git a/src/backend/mod.rs b/src/backend/mod.rs index 5c5b4046..a8b4f151 100644 --- a/src/backend/mod.rs +++ b/src/backend/mod.rs @@ -20,6 +20,8 @@ pub mod scan; pub mod fuzzy; pub mod search; pub mod searchreq; +pub mod reclaim; +pub mod reclaimreq; pub mod sort; pub mod state; pub mod mediaprobe; diff --git a/src/backend/proto.rs b/src/backend/proto.rs index 7bdf8a4a..2acf2b05 100644 --- a/src/backend/proto.rs +++ b/src/backend/proto.rs @@ -10,6 +10,9 @@ pub enum Request { Search { path: String, query: String, hidden: bool }, // Unlike thumbcancel there is no rows form: one walk runs at a time, so a cancel can only mean that one. SearchCancel, + // The regenerable-directory walk behind the client's cleanup entry point, one at a time like a search. + Reclaim { path: String }, + ReclaimCancel, Thumb { rows: Vec }, ThumbCancel { rows: Vec }, DirSize { rows: Vec }, @@ -66,6 +69,8 @@ pub fn parse_request(line: &str) -> Request { hidden: field_bool(line, "hidden"), }, Some("searchcancel") => Request::SearchCancel, + Some("reclaim") => Request::Reclaim { path: field_str(line, "path").unwrap_or_default() }, + Some("reclaimcancel") => Request::ReclaimCancel, Some("thumb") => Request::Thumb { rows: field_usize_array(line, "rows") }, Some("thumbcancel") => Request::ThumbCancel { rows: field_usize_array(line, "rows") }, Some("dirsize") => Request::DirSize { rows: field_usize_array(line, "rows") }, @@ -145,6 +150,20 @@ pub fn searched_line(n: usize, scanned: usize, ms: f64, cancelled: bool) -> Stri ) } +// The streaming progress of a reclaim: matches found, entries scanned and bytes sized so far. +pub fn reclaiming_line(n: usize, scanned: usize, bytes: u64, ms: f64) -> String { + format!(r#"{{"t":"reclaiming","n":{},"scanned":{},"bytes":{},"ms":{:.3}}}"#, n, scanned, bytes, ms) +} + +// The terminal line of a reclaim: n is the row count, bytes the total the rows cover, and +// cancelled is true when the client stopped the walk or a new listing replaced it. +pub fn reclaimed_line(n: usize, scanned: usize, bytes: u64, ms: f64, cancelled: bool) -> String { + format!( + r#"{{"t":"reclaimed","n":{},"scanned":{},"bytes":{},"ms":{:.3},"cancelled":{}}}"#, + n, scanned, bytes, ms, cancelled + ) +} + // The file is empty rather than absent on failure, so a client never waits forever for a row that will not arrive. pub fn thumbed_line(row: usize, file: &str, ms: f64) -> String { format!(r#"{{"t":"thumbed","row":{},"file":"{}","ms":{:.3}}}"#, row, escape(file), ms) diff --git a/src/backend/reclaim.rs b/src/backend/reclaim.rs new file mode 100644 index 00000000..d3e8bbce --- /dev/null +++ b/src/backend/reclaim.rs @@ -0,0 +1,374 @@ +// The subtree walk behind a reclaim request: find regenerable directories, size each one, rank +// by size. The walk is the search walk's shape — a slice of directories or one sized match per +// step, so a cancel is never behind more than one tree — and the ranking is the same span +// permutation a search performs, by bytes instead of score. +use crate::backend::dirsize; +use crate::backend::listing::Listing; +use std::os::unix::fs::MetadataExt; +use std::path::PathBuf; +use std::time::Instant; + +// One tick reads this many directories or sizes this many matches, the same bound the search +// walk and the dirsize walker run under. +const DIRS_PER_TICK: usize = 4; + +// The exact base names a reclaim reports, case sensitive. The list is build systems and package +// managers' regenerable trees: everything here is weight a package manager or a build can put +// back, and moving one to the trash is recoverable twice over — flea's undo, then a rebuild. +pub const TARGETS: &[&str] = &[ + // JavaScript and its frameworks. + "node_modules", + ".next", + ".nuxt", + ".output", + ".turbo", + ".parcel-cache", + // Rust, Go-style build outputs, and the generic two. + "target", + "dist", + "build", + "out", + // Python and its tooling. + ".venv", + "venv", + "__pycache__", + ".pytest_cache", + ".mypy_cache", + ".ruff_cache", + // JVM, Terraform, Dart, Sass, coverage runners. + ".gradle", + ".terraform", + ".dart_tool", + ".sass-cache", + "coverage", +]; + +#[derive(Clone, Copy, PartialEq)] +enum Phase { + // Names only, d_type and one lstat per directory for the device check. + Discover, + // One matched tree per step, through the same walker the dirsize row answers use. + Size, +} + +pub struct Reclaim { + root: PathBuf, + // The root's own device. A directory on any other filesystem is never descended into, so a + // scan rooted at home stops at every mount rather than reading a network share by accident. + dev: u64, + // Directories still to read, each a path relative to root; the empty string is root itself. + pending: Vec, + phase: Phase, + // Each match's path relative to root, in push order. Push order is row order until rank() + // permutes the spans, and sizes rides beside it under the same rule. + found: Vec, + // (bytes, partial) per match, in push order. partial is dirsize's own 2000 ms floor mark. + sizes: Vec<(u64, bool)>, + // How many matches are sized so far, one per step of the Size phase. + sized: usize, + pub scanned: usize, + pub bytes: u64, + pub started: Instant, +} + +impl Reclaim { + pub fn new(root: &str) -> Reclaim { + Reclaim { + root: PathBuf::from(root), + dev: std::fs::metadata(root).map(|m| m.dev()).unwrap_or(0), + pending: vec![String::new()], + phase: Phase::Discover, + found: Vec::new(), + sizes: Vec::new(), + sized: 0, + scanned: 0, + bytes: 0, + started: Instant::now(), + } + } + + #[cfg(test)] + pub fn matched(&self) -> usize { + self.found.len() + } + + #[cfg(test)] + pub fn sized(&self) -> usize { + self.sized + } + + // Sizes in the listing's current row order: push order before rank, ranked order after. + pub fn sizes(&self) -> &[(u64, bool)] { + &self.sizes + } + + // Returns true when the walk is finished; the caller then writes the terminal line. + pub fn step(&mut self, listing: &mut Listing) -> bool { + if self.phase == Phase::Discover { + for _ in 0..DIRS_PER_TICK { + match self.pending.pop() { + Some(rel) => self.read_one(&rel, listing), + None => { + self.phase = Phase::Size; + break; + } + } + } + if self.phase == Phase::Discover { + return false; + } + } + // The Size phase: one matched tree per call, the same one-at-a-time rule the dirsize + // walker runs under, so a cancel is never behind more than one subtree. + if self.sized < self.found.len() { + let path = self.root.join(&self.found[self.sized]); + let result = dirsize::walk(&path); + self.sizes.push((result.bytes, result.partial)); + self.bytes += result.bytes; + self.sized += 1; + } + self.sized >= self.found.len() + } + + fn read_one(&mut self, rel: &str, listing: &mut Listing) { + let dir = if rel.is_empty() { self.root.clone() } else { self.root.join(rel) }; + // corner: an unreadable directory is skipped in silence, exactly as scan.rs's phase one + // skips an unreadable entry and search's walk skips an unreadable subtree. + let rd = match std::fs::read_dir(&dir) { + Ok(rd) => rd, + Err(_) => return, + }; + for entry in rd.flatten() { + let name = entry.file_name(); + let name = name.to_string_lossy(); + self.scanned += 1; + // d_type is free and answers is_dir with no stat, matching scan.rs's phase one. A + // symlink reports its own type, so a link named like a target is never matched and + // never descended, and no loop is possible. + if !entry.file_type().map(|f| f.is_dir()).unwrap_or(false) { + continue; + } + let child = if rel.is_empty() { name.to_string() } else { format!("{}/{}", rel, name) }; + // corner: there is no hidden flag on a reclaim — .venv and __pycache__ are the point. + if TARGETS.contains(&name.as_ref()) { + // A match is reported but never descended into: its own size already covers + // anything nested, and a monorepo's nested node_modules would answer twice. + listing.push(&child, true); + self.found.push(child); + continue; + } + let entry_dev = entry.metadata().map(|m| m.dev()).unwrap_or(0); + if self.dev != 0 && entry_dev != 0 && entry_dev != self.dev { + continue; + } + self.pending.push(child); + } + } + + // The walk appends in discovery order, so ranking is one permutation of the spans at the end + // and the name arena never moves. Answers whether the row order changed, because a new order + // invalidates every outstanding row index the same way a sort does. The size table is + // permuted by the same order, so sizes() keeps naming rows by index afterwards. + pub fn rank(&mut self, listing: &mut Listing) -> bool { + // A walk cancelled mid-discovery or mid-sizing leaves rows without sizes; they stay in + // discovery order and the client's dirsize requests size them on demand instead. + if self.sizes.len() != listing.len() || listing.len() < 2 { + return false; + } + // Take the buffer out so the comparator can borrow it while spans are moved, as + // search's rank and sort.rs both do. + let names = std::mem::take(&mut listing.names); + let spans = &listing.spans; + let sizes = &self.sizes; + let mut order: Vec = (0..spans.len()).collect(); + // Heaviest first, which is the whole point of the ranking. The tie is the raw byte + // compare, so one walk over one tree answers in exactly one order. + order.sort_by(|&a, &b| { + sizes[b].0.cmp(&sizes[a].0).then_with(|| { + let an = &names[spans[a].off as usize..(spans[a].off + spans[a].len) as usize]; + let bn = &names[spans[b].off as usize..(spans[b].off + spans[b].len) as usize]; + an.cmp(bn) + }) + }); + let ranked: Vec<_> = order.iter().map(|&i| spans[i]).collect(); + let sized: Vec<_> = order.iter().map(|&i| sizes[i]).collect(); + listing.spans = ranked; + listing.names = names; + self.sizes = sized; + true + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::backend::testdir::TestDir; + use std::os::unix::fs::PermissionsExt; + + // Hard rule 9: every path below is inside a sandbox TestDir made and removes itself. + fn root(d: &TestDir) -> &str { + d.path().to_str().expect("the sandbox path is utf-8") + } + + fn walk_all(root: &str) -> (Listing, Reclaim) { + let mut r = Reclaim::new(root); + let mut l = Listing::new(); + while !r.step(&mut l) {} + r.rank(&mut l); + (l, r) + } + + // Ranked order, which is the order the client is answered in. + fn names(l: &Listing) -> Vec { + (0..l.len()).map(|i| l.name(i).to_string()).collect() + } + + fn sorted_names(l: &Listing) -> Vec { + let mut v = names(l); + v.sort(); + v + } + + #[test] + fn regenerable_directories_match_by_exact_base_name() { + let d = TestDir::new("reclaim"); + d.dir("proj/node_modules/left-pad"); + d.file("proj/node_modules/left-pad/index.js", "x"); + d.dir("rust/target/debug"); + d.file("rust/target/debug/app", "x"); + d.dir(".venv/lib"); + d.dir("node_module"); + d.file("node_module/readme", ""); + d.dir("builds"); + + let (l, _) = walk_all(root(&d)); + // The singular and the plural are decoys, and a match's inside is never listed. + assert_eq!(sorted_names(&l), [".venv", "proj/node_modules", "rust/target"]); + for i in 0..l.len() { + assert!(l.is_dir(i), "{} keeps the directory bit", l.name(i)); + } + } + + #[test] + fn a_match_reports_one_row_for_the_whole_tree_it_covers() { + let d = TestDir::new("cover"); + d.dir("node_modules/a/b"); + d.file("node_modules/a/b/big.bin", &"x".repeat(10_000)); + d.dir("node_modules/nested/node_modules"); + + let (l, r) = walk_all(root(&d)); + // The nested target is not a second row: the outer one's size already covers it. + assert_eq!(names(&l), ["node_modules"]); + assert_eq!(r.sizes().len(), 1); + assert!(r.sizes()[0].0 >= 10_000, "the size is the whole tree, got {}", r.sizes()[0].0); + assert!(!r.sizes()[0].1); + } + + #[test] + fn results_arrive_ranked_heaviest_first() { + let d = TestDir::new("rank"); + d.dir("small/node_modules"); + d.dir("big/target"); + d.file("big/target/libbig.a", &"x".repeat(100_000)); + + let (l, _) = walk_all(root(&d)); + assert_eq!(names(&l), ["big/target", "small/node_modules"]); + } + + #[test] + fn a_symlink_named_like_a_target_is_never_matched_nor_descended() { + let d = TestDir::new("loop"); + d.dir("real/node_modules"); + d.file("real/node_modules/x", ""); + std::os::unix::fs::symlink(d.join("real"), d.join("node_modules")).unwrap(); + + let (l, _) = walk_all(root(&d)); + assert_eq!(sorted_names(&l), ["real/node_modules"]); + } + + #[test] + fn a_directory_on_another_filesystem_is_never_descended_into() { + let d = TestDir::new("cross"); + d.dir("proj/target"); + d.file("proj/target/app", "x"); + d.dir("proj/src"); + d.file("proj/src/main.rs", "fn main() {}"); + + let mut r = Reclaim::new(root(&d)); + // Pretend every directory read out of the root lives on another device, the way a + // mounted share reports; nothing below the root may be descended into. + r.dev = r.dev.wrapping_add(1); + let mut l = Listing::new(); + while !r.step(&mut l) {} + assert_eq!(l.len(), 0); + assert!(r.scanned > 0, "the root itself was still read"); + } + + #[test] + fn a_missing_root_finishes_with_nothing_rather_than_failing() { + let mut r = Reclaim::new("/definitely/not/here"); + let mut l = Listing::new(); + assert!(r.step(&mut l)); + assert_eq!(l.len(), 0); + assert_eq!(r.scanned, 0); + } + + #[test] + fn a_step_reads_a_bounded_slice_so_a_cancel_is_never_blocked() { + let d = TestDir::new("slice"); + for i in 0..DIRS_PER_TICK + 3 { + d.dir(&format!("d{}", i)); + } + let mut r = Reclaim::new(root(&d)); + let mut l = Listing::new(); + // The root read queues every child, so the first step cannot also drain them. + assert!(!r.step(&mut l)); + } + + #[test] + fn the_size_phase_walks_one_match_per_step() { + let d = TestDir::new("pace"); + for name in ["node_modules", "target", ".venv"] { + d.dir(&format!("{}/pkg", name)); + d.file(&format!("{}/pkg/index.js", name), "x"); + } + let mut r = Reclaim::new(root(&d)); + let mut l = Listing::new(); + // The root read finds all three and the phase flips in the same tick, so the first step + // already sizes one match and each later step sizes exactly one more. + assert!(!r.step(&mut l)); + assert_eq!(r.matched(), 3); + assert_eq!(r.sized(), 1); + assert!(!r.step(&mut l)); + assert_eq!(r.sized(), 2); + assert!(r.step(&mut l)); + assert_eq!(r.sized(), 3); + assert_eq!(l.len(), 3); + } + + #[test] + fn an_unreadable_subtree_is_skipped_and_marks_its_size_partial() { + let d = TestDir::new("denied"); + d.dir("node_modules/locked"); + d.file("node_modules/locked/x", ""); + d.dir("target"); + std::fs::set_permissions( + d.join("node_modules/locked"), + std::fs::Permissions::from_mode(0o000), + ) + .unwrap(); + + let (l, r) = walk_all(root(&d)); + std::fs::set_permissions( + d.join("node_modules/locked"), + std::fs::Permissions::from_mode(0o755), + ) + .unwrap(); + // The match is still reported, its size is what was readable, and the client is told it + // is a floor rather than an exact number, which is dirsize's own rule. + assert_eq!(sorted_names(&l), ["node_modules", "target"]); + let sizes = r.sizes(); + let locked = sizes.iter().find(|&&(b, _)| b > 0).expect("the readable tree was sized"); + assert!(locked.1, "the denied subtree marks the size partial"); + } +} diff --git a/src/backend/reclaimreq.rs b/src/backend/reclaimreq.rs new file mode 100644 index 00000000..342b34d4 --- /dev/null +++ b/src/backend/reclaimreq.rs @@ -0,0 +1,61 @@ +// The wire side of a reclaim: the loop hands it a slice of walking and it decides what to say, +// the way searchreq answers search and dirsizereq answers dirsize. +use crate::backend::proto::{reclaimed_line, reclaiming_line}; +use crate::backend::run::{forget_rows, since}; +use crate::backend::state::State; +use crate::backend::thumbs::Pool; +use std::io::{self, BufWriter, Write}; +use std::time::{Duration, Instant}; + +// A running reclaim announces its counts no more often than this, so a fast walk cannot flood +// the client's parser, the same throttle a search's stream runs on. +const RECLAIM_REPORT: Duration = Duration::from_millis(100); + +// One tick of the walk: a discovery slice, or one sized match in the Size phase. +pub fn step_reclaim(out: &mut BufWriter, st: &mut State, pool: &Pool) { + let done = match st.reclaim.as_mut() { + Some(r) => r.step(&mut st.listing), + None => return, + }; + if done { + end_reclaim(out, st, pool, false); + return; + } + if st.reclaim_reported.elapsed() >= RECLAIM_REPORT { + st.reclaim_reported = Instant::now(); + let (scanned, bytes, ms) = match st.reclaim.as_ref() { + Some(r) => (r.scanned, r.bytes, since(r.started)), + None => return, + }; + writeln!(out, "{}", reclaiming_line(st.listing.len(), scanned, bytes, ms)).ok(); + out.flush().ok(); + } +} + +// The terminal line, and the cache seeding only the caller of a finished walk can do honestly. +// The rows are ranked before the line goes out, so every row index the client is still holding +// names a different file the moment this arrives, the same rule a search's terminal line follows. +// A cancelled walk keeps whatever it found, in discovery order when its sizes never covered every +// row, and the client's own dirsize requests size the rest on demand. Seeds the dirsize cache in +// row order afterwards, so a settled row's Size answer comes from the cache instead of walking +// again a tree this walk already paid for. +pub fn end_reclaim(out: &mut BufWriter, st: &mut State, pool: &Pool, cancelled: bool) -> bool { + let mut r = match st.reclaim.take() { + Some(r) => r, + None => return false, + }; + let reordered = r.rank(&mut st.listing); + let ms = since(r.started); + writeln!(out, "{}", reclaimed_line(st.listing.len(), r.scanned, r.bytes, ms, cancelled)).ok(); + out.flush().ok(); + if reordered { + forget_rows(st, pool); + } + for (row, &(bytes, partial)) in r.sizes().iter().enumerate() { + st.dirsizes.insert(row, (bytes, partial)); + } + // From here every window's directory rows carry these bytes in s, until the next list or + // search replaces the listing with one the walk did not measure. + st.reclaim_sizes = true; + reordered +} diff --git a/src/backend/run.rs b/src/backend/run.rs index 7c3d4153..fface639 100644 --- a/src/backend/run.rs +++ b/src/backend/run.rs @@ -22,6 +22,8 @@ use crate::backend::listing::Listing; use crate::backend::search::Search; use crate::backend::state::{State, Tables}; use crate::backend::searchreq::{finish_search, step_search}; +use crate::backend::reclaim::Reclaim; +use crate::backend::reclaimreq::{end_reclaim, step_reclaim}; use crate::backend::sort::{parse_sort_by, sort_by_name, sort_listing}; use crate::backend::thumbcache::{default_root, Cache}; use crate::backend::thumbreq::{cancel_row, forget_one, report_done, thumb_rows}; @@ -126,6 +128,9 @@ pub fn run() -> i32 { dirsize_queue: Vec::new(), search: None, search_reported: Instant::now(), + reclaim: None, + reclaim_reported: Instant::now(), + reclaim_sizes: false, }; let (tx, rx) = channel::(); @@ -145,7 +150,7 @@ pub fn run() -> i32 { spawn_reader(tx); loop { // Idle (nothing queued and no walk running) this is exactly the old blocking recv, see docs/protocol.md "dirsize". - let event = if st.dirsize_queue.is_empty() && st.search.is_none() { + let event = if st.dirsize_queue.is_empty() && st.search.is_none() && st.reclaim.is_none() { match rx.recv() { Ok(e) => e, Err(_) => break, @@ -200,6 +205,7 @@ fn handle_line( match parse_request(line) { Request::List { path, first, hidden } => { // A new listing replaces whatever the walk was filling, so the walk ends before the scan starts. + end_reclaim(out, st, pool, true); if finish_search(out, st, true) { forget_rows(st, pool); } @@ -210,6 +216,7 @@ fn handle_line( st.base = PathBuf::from(&path); st.listing = l; forget_rows(st, pool); + st.reclaim_sizes = false; writeln!(out, "{}", listed_line(st.listing.len(), read_ms, sort_ms, dev_of(&st.base))).ok(); // Rides along unasked: asking costs a 60 ms round trip at first paint. write_window(out, st, 0, first, tb); @@ -229,12 +236,15 @@ fn handle_line( out.flush().ok(); } Request::Search { path, query, hidden } => { + // Either walk owns the listing a search replaces, so both end before it starts. + end_reclaim(out, st, pool, true); if finish_search(out, st, true) { forget_rows(st, pool); } st.base = PathBuf::from(&path); st.listing = Listing::new(); forget_rows(st, pool); + st.reclaim_sizes = false; // The client is told at once that its old rows are gone, then the count grows as matches arrive. writeln!(out, "{}", listed_line(0, 0.0, 0.0, dev_of(&st.base))).ok(); st.search = Some(Search::new(&path, &query, hidden)); @@ -246,8 +256,30 @@ fn handle_line( forget_rows(st, pool); } } + Request::Reclaim { path } => { + // Either walk owns the listing a reclaim replaces, so both end before it starts. + end_reclaim(out, st, pool, true); + if finish_search(out, st, true) { + forget_rows(st, pool); + } + st.base = PathBuf::from(&path); + st.listing = Listing::new(); + forget_rows(st, pool); + // The same three-part answer a search gives: the client's old rows are gone at once, + // the count grows as matches are found and sized, and the terminal line carries the + // total itself. + writeln!(out, "{}", listed_line(0, 0.0, 0.0, dev_of(&st.base))).ok(); + st.reclaim = Some(Reclaim::new(&path)); + st.reclaim_reported = Instant::now(); + out.flush().ok(); + } + Request::ReclaimCancel => { + // Whatever was found stays listed; rows the sizes never covered are sized on demand. + end_reclaim(out, st, pool, true); + } Request::Sort { by, desc } => { // The walk owns the listing sort would reorder, so it ends first rather than racing it. + end_reclaim(out, st, pool, true); if finish_search(out, st, true) { forget_rows(st, pool); } @@ -338,12 +370,17 @@ pub fn forget_rows(st: &mut State, pool: &Pool) { st.dirsize_queue.clear(); } -// dirsize first: its rows are on screen now, while a search walk is work the client asked for and can wait a tick. +// dirsize first: its rows are on screen now, while a subtree walk is work the client asked for +// and can wait a tick. At most one of the two walks runs: each request ends the other. fn tick_walkers(out: &mut BufWriter, st: &mut State, pool: &Pool) { if !st.dirsize_queue.is_empty() { walk_one_dirsize(out, st); return; } + if st.reclaim.is_some() { + step_reclaim(out, st, pool); + return; + } // A finished walk hands back its rows in ranked order, which renames every outstanding index. if step_search(out, st) { forget_rows(st, pool); @@ -390,8 +427,18 @@ pub fn since(t: Instant) -> f64 { } pub fn write_window(out: &mut impl Write, st: &State, start: usize, count: usize, tb: &Tables) { - let (metas, ms) = stat_range(&st.base, &st.listing, start, count); + let (mut metas, ms) = stat_range(&st.base, &st.listing, start, count); let start = start.min(st.listing.len()); + // On a reclaim listing the walk has already measured every row's tree, so s carries those + // bytes instead of the directory's own dirent size; that is what the scan's views read. A row + // the walk never sized (a cancelled walk) keeps its stat. + if st.reclaim_sizes { + for (i, meta) in metas.iter_mut().enumerate() { + if let Some(&(bytes, _)) = st.dirsizes.get(&(start + i)) { + meta.size = bytes; + } + } + } let mut kinds = tb.kinds.borrow_mut(); let line = rows_line(&st.listing, &metas, start, ms, &tb.mime, &tb.icons, &tb.aliases, &tb.thumbs, &mut kinds); writeln!(out, "{}", line).ok(); diff --git a/src/backend/state.rs b/src/backend/state.rs index 73a80daf..c24137bb 100644 --- a/src/backend/state.rs +++ b/src/backend/state.rs @@ -6,6 +6,7 @@ use crate::backend::kind::Kinds; use crate::backend::listing::Listing; use crate::backend::mime::Db; use crate::backend::search::Search; +use crate::backend::reclaim::Reclaim; use crate::backend::thumbspec::Thumbnailers; use std::cell::RefCell; use std::collections::HashMap; @@ -40,5 +41,13 @@ pub struct State { pub search: Option, // When the running walk last announced its count, so SEARCH_REPORT can throttle the stream. pub search_reported: Instant, + // The reclaim walk the loop ticks; None means no reclaim is running. At most one of the two + // walks exists: each request ends the other before it touches the listing. + pub reclaim: Option, + // When the running reclaim last announced its progress, throttled the same way a search's is. + pub reclaim_reported: Instant, + // True from a reclaim's terminal line until the next list or search: a window's directory rows + // then report the walk's measured bytes in s, the one reading every view of the scan needs. + pub reclaim_sizes: bool, } diff --git a/tests/js/keymap.js b/tests/js/keymap.js index ebab226a..b8a7900f 100644 --- a/tests/js/keymap.js +++ b/tests/js/keymap.js @@ -138,7 +138,7 @@ function run(check) { Keymap.SHEET.map(sheetAction).join("|"), Keymap.SHEET.map(function (row) { return row.action }).join("|")) check("the sheet is not empty, so the check above has a denominator", - Keymap.SHEET.length, 29) + Keymap.SHEET.length, 31) // A chord shares the row of the key it doubles, so every caret token must resolve to that row's // own action, or the sheet advertises a chord bound to something else. check("every chord the sheet draws is bound to the action of its own row", diff --git a/tests/protocol.sh b/tests/protocol.sh index 986a1ff1..1bafde3e 100755 --- a/tests/protocol.sh +++ b/tests/protocol.sh @@ -385,6 +385,43 @@ second_bytes=$(echo "$out" | grep -oE '"bytes":[0-9]+' | sed -n 2p | cut -d: -f2 check "row 0's answer after the sort is zzz's larger size, not aaa's stale cache entry" "0" "$?" sandbox_remove "$SZ_SB"; sandbox_remove "$DZ_SB" +# reclaim: the regenerable-tree walk; see docs/protocol.md "reclaim". A staged run, because the +# window must be asked for after the terminal line that ranked the rows. +RC_SB="$FIXTURE_ROOT/flea-reclaim-test-$$" +RC="$RC_SB/tree" +sandbox_make "$RC_SB" +mkdir -p "$RC/proj/node_modules/left-pad" "$RC/rust/target/debug" "$RC/plain" +printf 'x' > "$RC/proj/node_modules/left-pad/index.js" +printf '%080d' 0 > "$RC/rust/target/debug/app" +: > "$RC/plain/notes.txt" +out=$(dirsize_run "$(printf '{"c":"reclaim","path":"%s"}\n' "$RC")" '{"c":"window","start":0,"count":10}') +check "a reclaim opens with a listed line of nothing" "0" "$(echo "$out" | head -1 | grep -oE '"n":[0-9]+' | cut -d: -f2)" +check "exactly one terminal reclaimed line answers" "1" "$(echo "$out" | grep -c '"t":"reclaimed"')" +check "the terminal line counts both matches" "2" "$(echo "$out" | grep '"t":"reclaimed"' | grep -oE '"n":[0-9]+' | cut -d: -f2)" +win=$(echo "$out" | grep '"t":"rows"' | head -1) +check "the rows arrive ranked heaviest first" "rust/target" "$(echo "$win" | grep -oE '"n":"[^"]+"' | head -1 | cut -d'"' -f4)" +check "and a directory that matches nothing is absent from the listing" "0" "$(echo "$win" | grep -c 'plain')" +sz=$(echo "$win" | grep -oE '"n":"rust/target","d":true,"s":[0-9]+' | head -1 | grep -oE '[0-9]+$') +[ -n "$sz" ] && [ "$sz" -ge 80 ] 2>/dev/null +check "the rows object carries the walk's measured bytes in s" "0" "$?" +# The walk seeded the dirsize cache, so a settled row answers from it instead of walking again. +out=$(dirsize_run "$(printf '{"c":"reclaim","path":"%s"}\n' "$RC")" '{"c":"dirsize","rows":[0]}') +check "a row the walk already sized answers from the cache" "1" "$(echo "$out" | grep -c '"t":"dirsized"')" +cached=$(echo "$out" | grep '"t":"dirsized"' | grep -oE '"bytes":[0-9]+' | cut -d: -f2) +[ -n "$cached" ] && [ "$cached" -ge 80 ] 2>/dev/null +check "and the cached answer is the tree's whole size, not its dirent" "0" "$?" +# A cancel at once still answers the terminal line, marked cancelled, with whatever was found. +out=$(printf '{"c":"reclaim","path":"%s"}\n{"c":"reclaimcancel"}\n{"c":"quit"}\n' "$RC" | $BIN --backend) +check "a reclaim cancelled at once answers reclaimed" "1" "$(echo "$out" | grep -c '"t":"reclaimed"')" +check "and that terminal line says it was cancelled" "1" "$(echo "$out" | grep -c '"cancelled":true')" + +# A reclaim on a path that cannot be read finishes at once with nothing rather than failing. Staged +# like the runs above: a quit sent at once is read before the loop's first tick of the walk. +out=$(dirsize_run '{"c":"reclaim","path":"/definitely/not/here"}') +check "a reclaim on a missing root answers reclaimed" "reclaimed" "$(echo "$out" | grep '"t":"reclaimed"' | head -1 | grep -oE '"t":"[a-z]+"' | cut -d'"' -f4)" +check "and it scanned nothing" "0" "$(echo "$out" | grep '"t":"reclaimed"' | head -1 | grep -oE '"scanned":[0-9]+' | cut -d: -f2)" +sandbox_remove "$RC_SB" + # A new folder: one mkdir(2), answered like rename and journaled so z removes it; see docs/protocol.md "mkdir". MK_SB="$FIXTURE_ROOT/flea-mkdir-test-$$" MK="$MK_SB/tree" diff --git a/ui/Backend.qml b/ui/Backend.qml index ee2bd722..6d292b2c 100644 --- a/ui/Backend.qml +++ b/ui/Backend.qml @@ -16,6 +16,8 @@ Item { signal dirSized(int row, real bytes, bool partial) signal searching(int total, int scanned, real ms) signal searched(int total, int scanned, real ms, bool cancelled) + signal reclaiming(int total, int scanned, real bytes, real ms) + signal reclaimed(int total, int scanned, real bytes, real ms, bool cancelled) // The write operations, see docs/protocol.md; every one of them is reversible with undo. signal transferStarted(int id, int n, bool moving) signal transferProgress(int id, int index, string name, real bytes, real total) @@ -111,6 +113,17 @@ Item { root.send({ c: "searchcancel" }) } + // The regenerable-tree walk replaces the current listing with its matches, each named + // relative to path; see docs/protocol.md "reclaim". + function reclaim(path) { + root.send({ c: "reclaim", path: path }) + } + + // No rows form, one walk at a time, the same rule searchcancel runs on. + function reclaimcancel() { + root.send({ c: "reclaimcancel" }) + } + // rows, not paths: a client can only build a path for a row inside the window it holds, and a // selection can be wider than that; the backend resolves them at request time, see docs/protocol.md. function transfer(op, rows, dest) { @@ -239,6 +252,8 @@ Item { // Sample input: {"t":"thumbed","row":2,"file":"/home/gm/.cache/thumbnails/large/b98fa4.png","ms":75.823} // Sample input: {"t":"dirsized","row":4,"bytes":1048576,"partial":false,"ms":12.500} // Sample input: {"t":"searching","n":812,"scanned":41200,"ms":300.114} + // Sample input: {"t":"reclaiming","n":812,"scanned":41200,"bytes":3241000,"ms":300.114} + // Sample input: {"t":"reclaimed","n":9,"scanned":51234,"bytes":12884901888,"ms":41230.500,"cancelled":false} // Sample input: {"t":"transferstarted","id":12,"n":2,"moving":true} // Sample input: {"t":"transferprogress","id":12,"index":0,"name":"a.txt","bytes":40000000,"total":120000000} // Sample input: {"t":"transferitem","id":12,"index":1,"name":"photos","ok":false,"err":"permission denied"} @@ -272,6 +287,10 @@ Item { root.searching(message.n, message.scanned, message.ms) } else if (message.t === "searched") { root.searched(message.n, message.scanned, message.ms, message.cancelled) + } else if (message.t === "reclaiming") { + root.reclaiming(message.n, message.scanned, message.bytes, message.ms) + } else if (message.t === "reclaimed") { + root.reclaimed(message.n, message.scanned, message.bytes, message.ms, message.cancelled) } else if (message.t === "transferstarted") { root.transferStarted(message.id, message.n, message.moving) } else if (message.t === "transferprogress") { diff --git a/ui/Pane.qml b/ui/Pane.qml index 969ef674..a0ff0e68 100644 --- a/ui/Pane.qml +++ b/ui/Pane.qml @@ -47,6 +47,10 @@ FocusScope { property bool filterTyping: false property int searchScanned: 0 property real searchMs: 0 + // True when the RESULTS walk is a reclaim scan (R); ui/js/Reclaim.js owns every transition, and b asks shell for the views. + property bool reclaimWalk: false + property real reclaimBytes: 0 + signal reclaimMapRequested() property bool listInFlight: false property bool listedSeen: false readonly property bool menuVisible: menu.opened diff --git a/ui/PaneWire.qml b/ui/PaneWire.qml index 4fc96a9c..4b836e81 100644 --- a/ui/PaneWire.qml +++ b/ui/PaneWire.qml @@ -4,6 +4,7 @@ import "js/DirSizes.js" as DirSizes import "js/Errors.js" as Errors import "js/Nav.js" as Nav import "js/Ops.js" as Ops +import "js/Reclaim.js" as Reclaim import "js/Search.js" as Search import "js/Tabs.js" as Tabs import "js/Thumbs.js" as Thumbs @@ -135,6 +136,39 @@ Item { pane.listArea.restartSettle() } + // Sample input: {"t":"reclaiming","n":3,"scanned":8021,"bytes":2040000,"ms":410.200} + function onReclaiming(total, scanned, bytes, ms) { + if (!pane.reclaimWalk) { + return + } + pane.total = total + pane.searchScanned = scanned + pane.searchMs = ms + pane.reclaimBytes = bytes + pane.listingState = Search.listingState(pane, total) + // A reclaim rides no first screenful either: the count grows, so the window is asked for as it does. + pane.listArea.restartCoalesce() + pane.listArea.restartSettle() + } + + // Sample input: {"t":"reclaimed","n":9,"scanned":51234,"bytes":12884901888,"ms":41230.500,"cancelled":false} + function onReclaimed(total, scanned, bytes, ms, cancelled) { + if (!pane.reclaimWalk) { + return + } + pane.searchRunning = false + pane.searchCancelled = cancelled + pane.total = total + pane.searchScanned = scanned + pane.searchMs = ms + pane.reclaimBytes = bytes + pane.listingState = Search.listingState(pane, total) + // The rows were ranked immediately before this line, exactly as a search's are, and the + // backend seeded its dirsize cache beside the ranking, so the settle's asks answer from it. + Reclaim.settled(pane) + pane.listArea.restartSettle() + } + // A thumbed line for the previous listing is still in the pipe when open() clears the map. function onThumbed(row, file) { if (!pane.listInFlight) diff --git a/ui/ReclaimMap.qml b/ui/ReclaimMap.qml new file mode 100644 index 00000000..2fe0ca43 --- /dev/null +++ b/ui/ReclaimMap.qml @@ -0,0 +1,188 @@ +import QtQuick +import qs.Commons +import "." as Flea +import "js/ReclaimTree.js" as ReclaimTree +import "js/ReclaimPaint.js" as ReclaimPaint +import "js/Ops.js" as Ops +import "js/Format.js" as Format + +// One scan, eight readings: the reclaim results drawn as a treemap, folder cards, a sunburst, a +// flame, bubbles, a mind map, ranked bars and an age map. b opens this over the results and +// cycles the readings once it is up; a click on anything lands the cursor on the row it drew, so +// dd and the strip's Quick Wins stage the very tree the operator is looking at, with undo behind +// both. +Rectangle { + id: root + + property var pane: null + property string mode: "treemap" + property var pickedNode: null + // Rebuilt beside every rows reply: one tree, and the mode strip is only a choice of reading. + readonly property var model: pane ? ReclaimTree.build(pane.rows, pane.held, pane.path) : null + readonly property int treeCount: model ? ReclaimTree.leaves(model).length : 0 + property var shapes: [] + readonly property var palette: ({ + accent: String(Theme.color.accent), + bg: String(Theme.color.background), + fg: String(Theme.color.foreground), + muted: String(Theme.color.muted), + onAccent: String(Theme.color.background), + small: Math.round(Theme.font.caption) + "px \"" + Theme.font.family + "\"" + }) + + color: Theme.color.background + + // b on an open map cycles to the next reading; the strip's buttons reach any of them directly. + function cycle() { + var at = ReclaimTree.MODES.indexOf(root.mode) + root.mode = ReclaimTree.MODES[(at + 1) % ReclaimTree.MODES.length] + canvas.requestPaint() + } + + function hit(x, y) { + for (var i = root.shapes.length - 1; i >= 0; i--) { + var s = root.shapes[i] + if (!s.node || s.node.row < 0) { + continue + } + if (s.k === "rect") { + if (x >= s.x && x <= s.x + s.w && y >= s.y && y <= s.y + s.h) + return s.node + } else if (s.k === "dot" || s.k === "disc") { + var dx = x - s.cx + var dy = y - s.cy + if (dx * dx + dy * dy <= s.r * s.r) + return s.node + } else if (s.k === "arc") { + var ax = x - s.cx + var ay = y - s.cy + var d2 = ax * ax + ay * ay + if (d2 >= s.r0 * s.r0 && d2 <= s.r1 * s.r1) { + var a = Math.atan2(ay, ax) + while (a < s.a0) a += Math.PI * 2 + if (a <= s.a1) return s.node + } + } else if (s.k === "clabel") { + if (Math.abs(x - s.cx) <= s.w / 2 && Math.abs(y - s.cy) <= 8) + return s.node + } + } + return null + } + + // The strip: eight readings on the left, the one destructive shortcut on the right. Quick Wins + // selects every tree the scan listed and hands them to the trash together; undo is the same z + // it always is, and nothing is deleted that the listing does not already name. + Item { + id: strip + anchors.top: parent.top + anchors.left: parent.left + anchors.right: parent.right + height: Theme.chromeHeight + + Row { + id: modes + anchors.left: parent.left + anchors.leftMargin: Theme.spacing.rowPaddingX + anchors.verticalCenter: parent.verticalCenter + spacing: Theme.spacing.gap + + Repeater { + model: ReclaimTree.MODES + + delegate: Text { + required property var modelData + readonly property bool active: root.mode === modelData + text: ReclaimTree.LABELS[modelData] + color: active ? Theme.color.accent : Theme.color.muted + font.family: Theme.font.family + font.pixelSize: Theme.font.caption + textFormat: Text.PlainText + + HoverHandler { cursorShape: Qt.PointingHandCursor } + + TapHandler { + acceptedButtons: Qt.LeftButton + onSingleTapped: root.mode = modelData + } + } + } + } + + Rectangle { + id: quickWins + anchors.right: parent.right + anchors.rightMargin: Theme.spacing.rowPaddingX + anchors.verticalCenter: parent.verticalCenter + radius: 4 + border.width: 1 + border.color: Theme.color.accent + color: qwTap.pressed ? Theme.color.accent : "transparent" + width: qwLabel.width + 16 + height: Theme.chromeHeight - 6 + + Text { + id: qwLabel + anchors.centerIn: parent + // The button is the scan's own answer: everything it found, staged together. + text: "Quick Wins · trash " + root.treeCount + " trees (" + (root.model ? Format.size(root.model.bytes) : "0 B") + ")" + color: qwTap.pressed ? Theme.color.background : Theme.color.accent + font.family: Theme.font.family + font.pixelSize: Theme.font.caption + textFormat: Text.PlainText + } + + HoverHandler { cursorShape: Qt.PointingHandCursor } + + TapHandler { + id: qwTap + acceptedButtons: Qt.LeftButton + onSingleTapped: { + if (root.treeCount > 0 && root.pane) { + root.pane.selectAll() + Ops.trash(root.pane) + } + } + } + } + } + + Canvas { + id: canvas + anchors.top: strip.bottom + anchors.left: parent.left + anchors.right: parent.right + anchors.bottom: parent.bottom + property var shapes: [] + + onPaint: { + if (!root.pane || !root.model) { + return + } + var ctx = canvas.getContext("2d") + var drawn = ReclaimTree.layout(root.mode, root.model, width, height, Date.now() / 1000) + canvas.shapes = drawn + ReclaimPaint.draw(ctx, drawn, root.palette, root.pickedNode, width, height) + } + + MouseArea { + anchors.fill: parent + onClicked: function (mouse) { + var node = root.hit(mouse.x, mouse.y) + if (node && root.pane) { + root.pickedNode = node + root.pane.setCursor(node.row) + } + } + } + } + + Connections { + target: root + + function onModelChanged() { canvas.requestPaint() } + function onModeChanged() { canvas.requestPaint() } + function onPickedNodeChanged() { canvas.requestPaint() } + + Component.onCompleted: canvas.requestPaint() +} diff --git a/ui/js/Focus.js b/ui/js/Focus.js index 53ca0746..b8cde810 100644 --- a/ui/js/Focus.js +++ b/ui/js/Focus.js @@ -6,6 +6,7 @@ .import "Ops.js" as Ops .import "PreviewKeys.js" as PreviewKeys .import "RailKeys.js" as RailKeys +.import "Reclaim.js" as Reclaim .import "Search.js" as Search .import "Sort.js" as Sort .import "Trash.js" as Trash @@ -58,6 +59,9 @@ function lookup(event, root) { // reveal only means something on a search result, so o is discarded everywhere else. if (action === "reveal" && root.searchMode !== Search.RESULTS) return "" + // The views draw a scan's results, so b means nothing outside reclaim results. + if (action === "reclaimMap" && !root.reclaimWalk) + return "" // A sort ends the running walk in the backend and the search strip hides the mark that would // show it happening, so both sort keys go quiet for as long as a search owns the header. if (action === "sortNext" || action === "sortReverse") @@ -84,10 +88,12 @@ function act(action, root) { case "open": root.openCursor(); return case "parent": root.openParent(); return case "toggleHidden": root.toggleHidden(); return - // Esc unwinds one thing at a time, and the least destructive first: a running walk, then the - // search, then the filter (which loses nothing), then the selection, then the transient line. + // Esc unwinds one thing at a time, and the least destructive first: a running reclaim scan, + // then a running search walk, then the search itself, then the filter (which loses nothing), + // then the selection, then the transient line. case "escape": - if (root.searchMode.length > 0) Search.cancel(root) + if (root.reclaimWalk) Reclaim.cancel(root) + else if (root.searchMode.length > 0) Search.cancel(root) else if (root.filterTyping || root.filterQuery.length > 0) Filter.close(root) else root.escapePressed() return @@ -99,6 +105,10 @@ function act(action, root) { // A walk replaces the listing the filter was narrowing, so the filter goes before the query line // does: leaving it up would hide every result that did not happen to match it. case "search": Filter.close(root); Search.start(root); return + // R weighs the directory the pane stands in: the scan replaces the listing with the + // regenerable trees it found, heaviest first, and dd on a row trashes that whole tree. + case "reclaim": Filter.close(root); Reclaim.start(root); return + case "reclaimMap": root.reclaimMapRequested(); return case "filter": Filter.start(root); return case "reveal": Search.reveal(root); return // The write operations; every one of them is reversible with undo, so none of them confirms. diff --git a/ui/js/Keymap.js b/ui/js/Keymap.js index 88c112e9..6fd61947 100644 --- a/ui/js/Keymap.js +++ b/ui/js/Keymap.js @@ -106,6 +106,8 @@ function lookup(key, text, modifiers) { case "h": return "parent" case "/": return "filter" case "f": return "search" + case "R": return "reclaim" + case "b": return "reclaimMap" case "o": return "reveal" case ":": return "pathBar" case "t": return "tabNew" @@ -165,6 +167,8 @@ var HINTS = { "paste": "p", "pathBar": ":", "preview": "space", + "reclaim": "R", + "reclaimMap": "b", "rename": "r", "reveal": "o", "search": "f", @@ -205,7 +209,9 @@ var SHEET = [ { keys: "x ^x", action: "cut", label: "cut" }, { keys: "p ^v", action: "paste", label: "paste" }, { keys: "r", action: "rename", label: "rename" }, + { keys: "R", action: "reclaim", label: "find regenerable trees" }, { keys: "dd", action: "trashArm", label: "trash" }, + { keys: "R b", action: "reclaim", label: "scan trees, then views" }, { keys: "z ^z", action: "undo", label: "undo" }, { keys: "^N", action: "newFolder", label: "new folder" }, { keys: "v", action: "toggleSelect", label: "select" }, diff --git a/ui/js/Nav.js b/ui/js/Nav.js index b3a02b12..2f22e756 100644 --- a/ui/js/Nav.js +++ b/ui/js/Nav.js @@ -75,6 +75,9 @@ function openWithoutHistory(pane, newPath) { pane.dirSizeState = DirSizes.empty() pane.cursorIndex = 0 pane.trashArmedAt = 0 + // A reclaim's results were the listing being replaced, so the flag goes with them: the new + // listing is an ordinary directory, and a stale terminal line must not re-rank it. + pane.reclaimWalk = false // The row the editor sat on belongs to the listing being replaced, so the rename goes with it: // leaving the index set opened an empty editor over whatever file arrived at that row instead. pane.renamingIndex = -1 diff --git a/ui/js/Reclaim.js b/ui/js/Reclaim.js new file mode 100644 index 00000000..03d351ef --- /dev/null +++ b/ui/js/Reclaim.js @@ -0,0 +1,74 @@ +.pragma library + +.import "Format.js" as Format +.import "Search.js" as Search + +// The reclaim scan's behaviour, taking ui/Pane.qml's root the way Search.js does. Deliberately +// the search state machine's own fields: there is one walk, one header and one set of guards, so +// a reclaim is searchMode RESULTS with reclaimWalk naming which walk it is. The wire is +// docs/protocol.md "reclaim"; R starts a scan on the directory the pane stands in. + +// A scan asked for on a search's results closes that search first, which returns the pane to +// where the search began, and the scan runs there: one searchFrom and one walk, the same rule a +// second search already answers to. Rows arrive ranked heaviest first, and dd on a row trashes +// that whole tree with undo behind it. +function start(root) { + if (root.searchMode === Search.RESULTS && !root.reclaimWalk) { + Search.close(root) + } + var scope = root.path + if (root.searchFrom.length === 0) { + root.searchFrom = scope + } + root.searchMode = Search.RESULTS + root.reclaimWalk = true + root.searchRunning = true + root.searchScanned = 0 + root.searchCancelled = false + root.reclaimBytes = 0 + root.total = 0 + root.held = 0 + root.rows = [] + root.kindNames = [] + root.cursorIndex = 0 + root.listingState = "loading" + root.clearSelection() + root.backend.reclaim(scope) +} + +// Esc stops a running scan and leaves what it found listed; a second Esc returns to the listing +// the scan began from, which is Search.close's own move once the flag names no walk. +function cancel(root) { + if (root.searchRunning) { + root.backend.reclaimcancel() + return + } + close(root) +} + +function close(root) { + root.reclaimWalk = false + Search.close(root) +} + +// The terminal reclaimed line answers with the same five moves a search's terminal line does: the +// rows were ranked on the wire before it arrived, so the client empties its per-row maps and +// re-reads its window. The backend seeded its own dirsize cache in the same statement, so the +// settle's size asks answer from that cache instead of walking every listed tree a second time. +function settled(root) { + Search.ranked(root) +} + +// The status bar's left half while the scan owns it: the running scan line, then the tally. +function statusLine(running, total, scanned, bytes, ms) { + if (running) { + return "Reclaiming, " + Search.grouped(scanned) + " scanned" + } + return Search.grouped(total) + (total === 1 ? " tree, " : " trees, ") + Format.size(bytes) + " in " + (ms / 1000).toFixed(1) + " s" +} + +function statusKeys(running) { + return running + ? "esc cancels, dd trashes a tree, z undoes" + : "esc returns to the listing" +} diff --git a/ui/js/ReclaimPaint.js b/ui/js/ReclaimPaint.js new file mode 100644 index 00000000..d4acac05 --- /dev/null +++ b/ui/js/ReclaimPaint.js @@ -0,0 +1,105 @@ +.pragma library + +.import "Format.js" as Format + +// The one drawer behind every reclaim view. It takes the shapes ui/js/ReclaimTree.js's layouts +// answer and puts them on a Canvas context, in the palette ui/ReclaimMap.qml builds from the +// theme: accent fills scaled by depth, the foreground for text, nothing hard-coded. Every draw +// clears its box first, because a Canvas keeps the previous view's pixels otherwise. + +function draw(ctx, shapes, pal, picked, w, h) { + ctx.clearRect(0, 0, w, h) + for (var i = 0; i < shapes.length; i++) { + var s = shapes[i] + switch (s.k) { + case "rect": + // The scan root is the container, so it takes the background; every real tree takes + // the accent at its depth's strength, and a background hairline separates the cells. + ctx.fillStyle = (s.depth || 0) === 0 ? pal.bg : pal.accent + ctx.globalAlpha = (s.depth || 0) === 0 ? 1 : 1 - Math.min(0.55, ((s.depth || 0) - 1) * 0.22) + ctx.fillRect(s.x, s.y, s.w, s.h) + ctx.globalAlpha = 1 + ctx.strokeStyle = pal.bg + ctx.lineWidth = 1 + ctx.strokeRect(s.x + 0.5, s.y + 0.5, Math.max(1, s.w - 1), Math.max(1, s.h - 1)) + break + case "arc": + ctx.globalAlpha = 1 - Math.min(0.55, (s.depth || 0) * 0.22) + ctx.fillStyle = pal.accent + ctx.beginPath() + ctx.arc(s.cx, s.cy, s.r1, s.a0, s.a1, false) + ctx.arc(s.cx, s.cy, s.r0, s.a1, s.a0, true) + ctx.closePath() + ctx.fill() + ctx.globalAlpha = 1 + break + case "disc": + ctx.fillStyle = pal.accent + ctx.beginPath() + ctx.arc(s.cx, s.cy, s.r, 0, Math.PI * 2, false) + ctx.fill() + break + case "dot": + ctx.globalAlpha = 1 - Math.min(0.6, (s.depth || 0) * 0.28) + ctx.fillStyle = pal.accent + ctx.beginPath() + ctx.arc(s.cx, s.cy, s.r, 0, Math.PI * 2, false) + ctx.fill() + ctx.globalAlpha = 1 + break + case "line": + ctx.strokeStyle = pal.muted + ctx.lineWidth = 1 + ctx.beginPath() + ctx.moveTo(s.x1, s.y1) + ctx.lineTo(s.x2, s.y2) + ctx.stroke() + break + case "label": + ctx.fillStyle = s.node.depth === 0 ? pal.fg : pal.onAccent + ctx.font = pal.small + clipText(ctx, s.node.name, s.x, s.y, s.w) + break + case "clabel": + ctx.fillStyle = pal.fg + ctx.font = pal.small + ctx.textAlign = "center" + clipText(ctx, s.node.name, s.cx - s.w / 2, s.cy + 3, s.w) + ctx.textAlign = "left" + break + case "slabel": + ctx.fillStyle = pal.fg + ctx.font = pal.small + clipText(ctx, s.node.name + " · " + Format.size(s.node.bytes), s.x, s.y, s.w) + break + case "text": + ctx.fillStyle = pal.muted + ctx.font = pal.small + ctx.fillText(s.text, s.x, s.y) + break + } + // The picked row's own shapes take the foreground outline, so a click reads back. + if (s.node && s.node === picked) { + ctx.strokeStyle = pal.fg + ctx.lineWidth = 1.5 + if (s.k === "rect") { + ctx.strokeRect(s.x + 0.5, s.y + 0.5, Math.max(1, s.w - 1), Math.max(1, s.h - 1)) + } else if (s.k === "dot" || s.k === "disc") { + ctx.beginPath() + ctx.arc(s.cx, s.cy, s.r + 1, 0, Math.PI * 2, false) + ctx.stroke() + } + } + } + ctx.globalAlpha = 1 +} + +// A name never draws past its shape: cut by characters, the cheap way that needs no measure pass. +function clipText(ctx, text, x, y, w) { + var max = Math.floor(w / 6.5) + var t = String(text) + if (t.length > max) { + t = max > 1 ? t.substring(0, max - 1) + "…" : "" + } + ctx.fillText(t, x, y) +} diff --git a/ui/js/ReclaimTree.js b/ui/js/ReclaimTree.js new file mode 100644 index 00000000..186577a4 --- /dev/null +++ b/ui/js/ReclaimTree.js @@ -0,0 +1,385 @@ +.pragma library + +// One scan, eight readings. The reclaim listing's rows — each a regenerable tree with its +// measured bytes in s and its mtime in m — build one tree, and every view in ui/ReclaimMap.qml +// renders that tree. The layouts here are pure: they take the tree and a box and answer shapes, +// and ui/js/ReclaimPaint.js is the only thing that draws. A shape carries the listing row it was +// built from, so a click on any view lands the cursor on the row it drew. +// +// The views read the window the pane holds, which is the whole listing whenever a scan fits in +// one, because a window is what the viewport rule lets the client hold. + +var MONTH = 2592000 +var YEAR = 31536000 +var MODES = ["treemap", "folders", "sunburst", "flame", "bubbles", "mindmap", "topsizes", "agemap"] +var LABELS = { + "treemap": "Treemap", + "folders": "Folders", + "sunburst": "Sunburst", + "flame": "Flame", + "bubbles": "Bubbles", + "mindmap": "Mind Map", + "topsizes": "Top Sizes", + "agemap": "Age Map" +} + +// One node per path segment, so a monorepo's apps/web/node_modules nests three deep and its +// aggregates come free: an internal node's bytes are the sum of what is under it. +function build(rows, held, rootPath) { + var root = { name: leafOf(rootPath), path: "", bytes: 0, m: 0, children: [], leaf: false, row: -1, depth: 0 } + for (var i = 0; i < rows.length; i++) { + var r = rows[i] + if (!r.d) { + continue + } + var segs = String(r.n).split("/") + var node = root + var acc = "" + for (var s = 0; s < segs.length; s++) { + acc = acc.length === 0 ? segs[s] : acc + "/" + segs[s] + var next = null + for (var c = 0; c < node.children.length; c++) { + if (node.children[c].name === segs[s]) { + next = node.children[c] + break + } + } + if (next === null) { + next = { name: segs[s], path: acc, bytes: 0, m: 0, children: [], leaf: false, row: -1, depth: s + 1 } + node.children.push(next) + } + node = next + } + node.leaf = true + node.bytes = r.s > 0 ? r.s : 0 + node.m = r.m || 0 + node.row = held + i + } + total(root) + return root +} + +function leafOf(path) { + var text = String(path) + var cut = text.lastIndexOf("/") + return cut < 0 || cut === text.length - 1 ? text : text.substring(cut + 1) +} + +function total(node) { + if (node.leaf) { + return node.bytes + } + var t = 0 + for (var c = 0; c < node.children.length; c++) { + t += total(node.children[c]) + } + node.bytes = t + return t +} + +function leaves(node, out) { + if (out === undefined) { + out = [] + } + if (node.leaf) { + out.push(node) + return out + } + for (var c = 0; c < node.children.length; c++) { + leaves(node.children[c], out) + } + return out +} + +function layout(mode, root, w, h, now) { + if (mode === "treemap") return treemap(root, 0, 0, w, h, []) + if (mode === "sunburst") return sunburst(root, w / 2, h / 2, Math.min(w, h) / 2 - 6, []) + if (mode === "flame") return flame(root, 0, 4, w, 18, 3, []) + if (mode === "bubbles") return bubbles(root, w, h, []) + if (mode === "mindmap") return mindmap(root, w / 2, h / 2, Math.min(w, h) / 2 - 10, []) + if (mode === "topsizes") return topsizes(root, 8, 6, w - 16, h - 12, []) + if (mode === "folders") return folders(root, 6, 6, w - 12, h - 12, []) + if (mode === "agemap") return agemap(root, 10, 6, w - 20, h - 12, now, []) + return [] +} + +// Squarified, one header strip per level, three levels deep: a scan nests deeper than that only +// through monorepos, and the listing itself is one click away for the rest. +function treemap(node, x, y, w, h, out) { + out.push({ k: "rect", x: x, y: y, w: w, h: h, node: node, depth: node.depth }) + if (node.depth < 3 && node.children.length > 0) { + var kids = node.children.slice().sort(function (a, b) { return b.bytes - a.bytes }) + squarify(kids, x + 1, y + 14, w - 2, h - 15, out) + } + out.push({ k: "label", x: x + 4, y: y + 11, w: Math.max(0, w - 8), node: node }) + return out +} + +function squarify(items, x, y, w, h, out) { + var total = 0 + for (var i = 0; i < items.length; i++) { + total += items[i].bytes + } + if (total <= 0 || w <= 2 || h <= 2) { + return + } + var rest = items.slice() + while (rest.length > 0) { + var side = Math.min(w, h) + var row = [rest.shift()] + var rowBytes = row[0].bytes + while (rest.length > 0 && rowBytes > 0) { + if (worst(row, rowBytes, side, total) < worst(row.concat([rest[0]]), rowBytes + rest[0].bytes, side, total)) { + break + } + rowBytes += rest[0].bytes + row.push(rest.shift()) + } + var frac = rowBytes / total + var across = w >= h + var rw = across ? w * frac : w + var rh = across ? h : h * frac + var off = 0 + for (var r = 0; r < row.length; r++) { + var share = rowBytes > 0 ? row[r].bytes / rowBytes : 0 + var ix = across ? x : x + off + var iy = across ? y + off : y + var iw = across ? rw : rw * share + var ih = across ? rh * share : rh + treemap(row[r], ix, iy, Math.max(1, iw), Math.max(1, ih), out) + off += across ? ih : iw + } + if (across) { + x += rw + w -= rw + } else { + y += rh + h -= rh + } + total -= rowBytes + } +} + +function worst(row, rowBytes, side, total) { + if (rowBytes <= 0) { + return Infinity + } + var t = side * (rowBytes / total) + if (t <= 0) { + return Infinity + } + var worst = 1 + for (var i = 0; i < row.length; i++) { + var d = side * (row[i].bytes / rowBytes) + if (d <= 0) { + return Infinity + } + var ratio = d > t ? d / t : t / d + if (ratio > worst) { + worst = ratio + } + } + return worst +} + +// Concentric rings, one per depth, angle by bytes. The root is the disc at the middle. +function sunburst(node, cx, cy, rmax, out) { + out.push({ k: "disc", cx: cx, cy: cy, r: 13, node: node }) + ring(node, cx, cy, 18, (rmax - 18) / 3, 0, 3, -Math.PI / 2, Math.PI * 2, out) + return out +} + +function ring(node, cx, cy, r0, thick, depth, maxDepth, a0, span, out) { + if (depth >= maxDepth || node.children.length === 0) { + return + } + var a = a0 + for (var c = 0; c < node.children.length; c++) { + var ch = node.children[c] + var s = node.bytes > 0 ? span * (ch.bytes / node.bytes) : 0 + if (s > 0.004) { + out.push({ k: "arc", cx: cx, cy: cy, r0: r0, r1: r0 + thick, a0: a, a1: a + s, node: ch, depth: depth }) + ring(ch, cx, cy, r0 + thick, thick, depth + 1, maxDepth, a, s, out) + } + a += s + } +} + +// Depth stacks downward, width by bytes inside the parent's own width: the scan's call stack. +function flame(node, x, y, w, rowH, maxDepth, out) { + out.push({ k: "rect", x: x, y: y, w: Math.max(1, w), h: rowH - 2, node: node, depth: node.depth === 0 ? 1 : node.depth }) + out.push({ k: "label", x: x + 4, y: y + 11, w: Math.max(0, w - 8), node: node }) + if (node.depth >= maxDepth || node.children.length === 0) { + return + } + var off = 0 + for (var c = 0; c < node.children.length; c++) { + var ch = node.children[c] + var cw = node.bytes > 0 ? w * (ch.bytes / node.bytes) : 0 + if (cw >= 2) { + flame(ch, x + off, y + rowH, cw, rowH, maxDepth, out) + } + off += cw + } + return out +} + +// Circles sized by area, biggest first, packed in rows so two big trees can never overlap. +function bubbles(root, w, h, out) { + var items = leaves(root).slice().sort(function (a, b) { return b.bytes - a.bytes }) + var total = root.bytes + var x = 10, y = 10, rowH = 0 + var cap = Math.max(24, Math.min(w, h) / 3) + for (var i = 0; i < items.length; i++) { + var it = items[i] + var r = total > 0 ? cap * Math.sqrt(it.bytes / total) : 3 + if (r < 3) { + r = 3 + } + if (x + 2 * r > w - 10 && x > 10) { + x = 10 + y += rowH + 10 + rowH = 0 + } + if (y + 2 * r > h - 10) { + r = Math.max(3, (h - 10 - y) / 2) + } + out.push({ k: "dot", cx: x + r, cy: y + r, r: r, node: it, depth: 0 }) + if (r >= 11) { + out.push({ k: "clabel", cx: x + r, cy: y + r, w: r * 1.7, node: it }) + } + x += 2 * r + 8 + if (2 * r > rowH) { + rowH = 2 * r + } + } + return out +} +// The root at the middle, its trees around it, each heavy tree's own biggest children beyond it. +function mindmap(root, cx, cy, R, out) { + out.push({ k: "disc", cx: cx, cy: cy, r: 15, node: root }) + out.push({ k: "clabel", cx: cx, cy: cy, w: 64, node: root }) + var kids = root.children.slice().sort(function (a, b) { return b.bytes - a.bytes }) + for (var i = 0; i < kids.length; i++) { + var a = (i / kids.length) * Math.PI * 2 - Math.PI / 2 + var x = cx + Math.cos(a) * R * 0.6 + var y = cy + Math.sin(a) * R * 0.6 + out.push({ k: "line", x1: cx, y1: cy, x2: x, y2: y }) + out.push({ k: "dot", cx: x, cy: y, r: bubbleR(kids[i], root, 16), node: kids[i], depth: 1 }) + out.push({ k: "clabel", cx: x, cy: y, w: 84, node: kids[i] }) + var grand = kids[i].children.slice().sort(function (a, b) { return b.bytes - a.bytes }).slice(0, 3) + for (var g = 0; g < grand.length; g++) { + var ga = a + (g - (grand.length - 1) / 2) * 0.4 + var gx = cx + Math.cos(ga) * R + var gy = cy + Math.sin(ga) * R + out.push({ k: "line", x1: x, y1: y, x2: gx, y2: gy }) + out.push({ k: "dot", cx: gx, cy: gy, r: 3, node: grand[g], depth: 2 }) + } + } + return out +} + +function bubbleR(node, parent, cap) { + return parent.bytes > 0 ? Math.max(3, Math.min(cap, 16 * Math.sqrt(node.bytes / parent.bytes))) : 3 +} + +// The ranked listing drawn as bars: heaviest at the top, the number at the end of each bar. +function topsizes(root, x, y, w, h, out) { + var items = leaves(root) + var max = 0 + for (var i = 0; i < items.length; i++) { + if (items[i].bytes > max) { + max = items[i].bytes + } + } + var n = Math.min(items.length, Math.max(1, Math.floor(h / 20))) + for (var b = 0; b < n; b++) { + var it = items[b] + var bw = max > 0 ? Math.max(2, w * 0.55 * (it.bytes / max)) : 2 + out.push({ k: "rect", x: x, y: y + b * 20, w: bw, h: 14, node: it, depth: 1 }) + out.push({ k: "slabel", x: x + bw + 6, y: y + b * 20 + 11, w: Math.max(0, w - bw - 6), node: it }) + } + return out +} + +// Folder cards in a grid, name and bytes, the listing's own order. +function folders(root, x, y, w, h, out) { + var items = leaves(root) + var cols = Math.max(1, Math.floor(w / 170)) + var cw = w / cols + var ch = Math.min(56, Math.max(40, h / Math.max(1, Math.ceil(items.length / cols)) - 6)) + for (var i = 0; i < items.length; i++) { + var cx = x + (i % cols) * cw + var cy = y + Math.floor(i / cols) * (ch + 6) + if (cy + ch > y + h) { + break + } + out.push({ k: "rect", x: cx, y: cy, w: cw - 8, h: ch, node: items[i], depth: 1 }) + out.push({ k: "clabel", cx: cx + (cw - 8) / 2, cy: cy + ch / 2 - 5, w: cw - 18, node: items[i] }) + out.push({ k: "slabel", x: cx + 8, y: cy + ch - 8, w: cw - 16, node: items[i] }) + } + return out +} + +// Bytes by how long ago the tree was last touched, in twelve buckets across the scan's age span, +// then the big and untouched — over a year old — listed under the chart. +function agemap(root, x, y, w, h, now, out) { + var items = leaves(root) + var maxAge = 0 + for (var i = 0; i < items.length; i++) { + var age = now - items[i].m + if (age > maxAge) { + maxAge = age + } + } + var span = Math.min(Math.max(maxAge, MONTH), YEAR * 2) * 1.05 + var buckets = 12 + var sums = [] + for (var b = 0; b < buckets; b++) { + sums.push(0) + } + var maxBytes = 0 + for (var i2 = 0; i2 < items.length; i2++) { + var bi = Math.floor((now - items[i2].m) / span * buckets) + if (bi < 0) { + bi = 0 + } + if (bi >= buckets) { + bi = buckets - 1 + } + sums[bi] += items[i2].bytes + if (sums[bi] > maxBytes) { + maxBytes = sums[bi] + } + } + var chartH = h * 0.45 + var bw = w / buckets + for (var b3 = 0; b3 < buckets; b3++) { + var bh = maxBytes > 0 ? chartH * (sums[b3] / maxBytes) : 0 + out.push({ k: "rect", x: x + b3 * bw + 2, y: y + chartH - bh, w: bw - 4, h: Math.max(bh, 1), node: null, depth: b3 + 1 }) + out.push({ k: "text", x: x + b3 * bw + 2, y: y + chartH + 11, text: ageLabel(span * (b3 + 0.5)) }) + } + var untouched = [] + for (var u = 0; u < items.length; u++) { + if (now - items[u].m > YEAR) { + untouched.push(items[u]) + } + } + untouched.sort(function (a, b) { return b.bytes - a.bytes }) + var lines = Math.min(untouched.length, Math.max(0, Math.floor((h - chartH - 30) / 16))) + if (untouched.length > 0) { + out.push({ k: "text", x: x, y: y + chartH + 28, text: "Big & untouched — over a year old:" }) + } + for (var v = 0; v < lines; v++) { + out.push({ k: "slabel", x: x, y: y + chartH + 44 + v * 16, w: w, node: untouched[v] }) + } + return out +} + +function ageLabel(seconds) { + var days = Math.round(seconds / 86400) + if (days < 31) { + return days + "d" + } + return Math.round(days / 30) + "mo" +} diff --git a/ui/shell.qml b/ui/shell.qml index 11067447..a603dcde 100644 --- a/ui/shell.qml +++ b/ui/shell.qml @@ -13,8 +13,12 @@ import "js/Nav.js" as Nav import "js/Ops.js" as Ops import "js/Renderer.js" as Renderer import "js/Search.js" as Search +import "js/Reclaim.js" as Reclaim ShellRoot { + // The reclaim views overlay, b-toggled over the pane's results; hidden again by navigation, + // which ends the walk the views draw. + property bool reclaimMapOn: false FloatingWindow { id: fleaWindow title: "Flea" @@ -151,6 +155,13 @@ ShellRoot { // Issue 9. ViewState persists the stop and Theme derives its own tokens from it, so // the whole window follows without any surface reading the chord itself. onTextSizeRequested: function (direction) { fleaWindow.applyTextSize(direction) } + onReclaimMapRequested: { + if (!reclaimMapOn) { + reclaimMapOn = true + } else { + reclaimMap.cycle() + } + } onOpened: function (path) { shareBrowser.close() } } @@ -168,12 +179,26 @@ ShellRoot { fsFree: pane.fsFree searchRunning: pane.searchRunning searchLine: pane.searchMode === "results" - ? Search.statusLine(pane.searchRunning, pane.total, pane.searchScanned, pane.searchMs) + ? (pane.reclaimWalk + ? Reclaim.statusLine(pane.searchRunning, pane.total, pane.searchScanned, pane.reclaimBytes, pane.searchMs) + : Search.statusLine(pane.searchRunning, pane.total, pane.searchScanned, pane.searchMs)) : "" - searchKeys: Search.statusKeys(pane.searchRunning) + searchKeys: pane.reclaimWalk ? Reclaim.statusKeys(pane.searchRunning) : Search.statusKeys(pane.searchRunning) onTransferCancelRequested: function (id) { backend.transfercancel(id) } } + // Explicit geometry, not anchors.fill: the component's own pane property shadows the + // pane id in that binding and the fill never lands. + Flea.ReclaimMap { + id: reclaimMap + x: pane.x + y: pane.y + width: pane.width + height: pane.height + visible: pane.reclaimWalk && reclaimMapOn + pane: pane + } + Flea.Preview { id: preview; pane: pane } Flea.ConvertDialog { From 97cf3e0e1d27a6990721786e48c32e55e8ab0956 Mon Sep 17 00:00:00 2001 From: Dzulfikar Ats Tsauri Date: Sun, 6 Sep 2026 18:45:23 +0700 Subject: [PATCH 2/4] reclaim: a chrome door and a staging Quick Wins The chrome bar gains a broom button beside the views it draws: one press scans where the pane stands and opens the views over it, a press on an open overlay toggles back to the listing, and the button lights while the views stand. It is the pointer's half of the R key. Quick Wins no longer deletes anything. It selects every tree the scan found and steps the overlay aside, so the listing comes back with the trees highlighted and the seeing comes before the dd; z and esc back out of the selection, undo stays behind the trash that follows. --- ui/ChromeBar.qml | 16 ++++++++++++++-- ui/ReclaimMap.qml | 22 ++++++++++++---------- ui/js/Icons.js | 3 +++ ui/shell.qml | 14 +++++++++++--- 4 files changed, 40 insertions(+), 15 deletions(-) diff --git a/ui/ChromeBar.qml b/ui/ChromeBar.qml index 4863c04f..e8fa924a 100644 --- a/ui/ChromeBar.qml +++ b/ui/ChromeBar.qml @@ -17,10 +17,15 @@ Item { property string viewMode: "list" // Read by the path bar alone, so a Tab on a dotted leaf peeks the way the listing is set to look. property bool showHidden: false + // Lit while the reclaim views overlay stands open over the pane's results. + property bool reclaimActive: false signal backRequested() - signal upRequested() signal searchRequested() + signal upRequested() + // The reclaim button: one press scans where the pane stands and opens the views over it; a + // press on an open views overlay toggles it back to the listing. See ui/js/Reclaim.js. + signal reclaimRequested() signal viewChosen(string mode) // The path bar's four. ui/shell.qml navigates, hands the keyboard back, runs the peek behind Tab // and carries what the bar says to the status line, because this file draws the chrome and knows @@ -380,8 +385,15 @@ Item { } } + // The reclaim scan's own door, beside the views it draws: a scan is a way of looking at a + // directory, and this is the pointer's half of the R key. + Flea.ChromeButton { + glyph: "broom" + active: root.reclaimActive + onActivated: root.reclaimRequested() + } + // The Settings board draws the sliders button at the right end, past a rule that separates - // it from the three view buttons: it changes the window, not the way the listing is drawn. // Row lays its own children out, so the rule takes the strip's height rather than anchoring. Rectangle { width: Theme.spacing.hairline diff --git a/ui/ReclaimMap.qml b/ui/ReclaimMap.qml index 2fe0ca43..bbf39cdd 100644 --- a/ui/ReclaimMap.qml +++ b/ui/ReclaimMap.qml @@ -3,20 +3,22 @@ import qs.Commons import "." as Flea import "js/ReclaimTree.js" as ReclaimTree import "js/ReclaimPaint.js" as ReclaimPaint -import "js/Ops.js" as Ops import "js/Format.js" as Format // One scan, eight readings: the reclaim results drawn as a treemap, folder cards, a sunburst, a // flame, bubbles, a mind map, ranked bars and an age map. b opens this over the results and -// cycles the readings once it is up; a click on anything lands the cursor on the row it drew, so -// dd and the strip's Quick Wins stage the very tree the operator is looking at, with undo behind -// both. +// cycles the readings once it is up; a click on anything lands the cursor on the row it drew. +// The strip's Quick Wins never deletes: it selects every tree the scan found and steps the +// overlay aside, so the seeing comes before the dd, and undo stays behind both. Rectangle { id: root property var pane: null property string mode: "treemap" property var pickedNode: null + // Fired by the strip's Quick Wins: everything the scan found is selected and the overlay + // steps aside, so the operator sees the trees highlighted in the listing before acting. + signal stageRequested() // Rebuilt beside every rows reply: one tree, and the mode strip is only a choice of reading. readonly property var model: pane ? ReclaimTree.build(pane.rows, pane.held, pane.path) : null readonly property int treeCount: model ? ReclaimTree.leaves(model).length : 0 @@ -70,9 +72,7 @@ Rectangle { return null } - // The strip: eight readings on the left, the one destructive shortcut on the right. Quick Wins - // selects every tree the scan listed and hands them to the trash together; undo is the same z - // it always is, and nothing is deleted that the listing does not already name. + // The strip: eight readings on the left, the staging shortcut on the right. Item { id: strip anchors.top: parent.top @@ -124,8 +124,9 @@ Rectangle { Text { id: qwLabel anchors.centerIn: parent - // The button is the scan's own answer: everything it found, staged together. - text: "Quick Wins · trash " + root.treeCount + " trees (" + (root.model ? Format.size(root.model.bytes) : "0 B") + ")" + // Staging, not deleting: the press selects everything the scan found and hands + // the listing back with the trees highlighted, so the seeing comes before the dd. + text: "Quick Wins · stage " + root.treeCount + " trees (" + (root.model ? Format.size(root.model.bytes) : "0 B") + ")" color: qwTap.pressed ? Theme.color.background : Theme.color.accent font.family: Theme.font.family font.pixelSize: Theme.font.caption @@ -140,7 +141,7 @@ Rectangle { onSingleTapped: { if (root.treeCount > 0 && root.pane) { root.pane.selectAll() - Ops.trash(root.pane) + root.stageRequested() } } } @@ -183,6 +184,7 @@ Rectangle { function onModelChanged() { canvas.requestPaint() } function onModeChanged() { canvas.requestPaint() } function onPickedNodeChanged() { canvas.requestPaint() } + } Component.onCompleted: canvas.requestPaint() } diff --git a/ui/js/Icons.js b/ui/js/Icons.js index 429336c7..39df213e 100644 --- a/ui/js/Icons.js +++ b/ui/js/Icons.js @@ -111,6 +111,9 @@ var PATHS = { // lucide's own folder-plus is the folder body byte for byte plus these two strokes, so the recut // body is reused verbatim and only the plus is new; the specimen sheet's 5 unit plus is not it. "folder-plus": "M2 20V3h6l2 3h12v14H2z M12 10v6 M9 13h6", + // A broom at rest: handle down from the top right, bristle fan along the bottom left. + "broom": "M21 3l-8 8 M13 11L4 20 M4 20v-4 M4 20h4 M8 12l4 4", + "trash": "M3 6h18 M8 6V3h8v3 M6 6l1.2 15h9.6L18 6 M10 10v7 M14 10v7", // A diamond eye with a square pupil: lucide's own eye is two arcs meeting at points, which the cut squares off. "eye": "M12 5 22 12 12 19 2 12z M10 10h4v4h-4z", diff --git a/ui/shell.qml b/ui/shell.qml index a603dcde..76cddd6c 100644 --- a/ui/shell.qml +++ b/ui/shell.qml @@ -105,12 +105,19 @@ ShellRoot { canGoUp: pane.canGoUp viewMode: pane.viewMode showHidden: pane.showHidden - onBackRequested: pane.goBack() + onViewChosen: function (mode) { pane.viewMode = mode } + reclaimActive: pane.reclaimWalk && reclaimMapOn + onReclaimRequested: { + if (!pane.reclaimWalk) { + pane.act("reclaim") + reclaimMapOn = true + } else { + reclaimMapOn = !reclaimMapOn + } + } onUpRequested: pane.openParent() onSearchRequested: pane.act("search") - onViewChosen: function (mode) { pane.viewMode = mode } // The path bar's four. The pane navigates and answers for the keyboard exactly as it - // does for every other route in, so a path typed and a row opened end the same way. onPathEntered: function (path) { pane.open(path) } onEditClosed: pane.forceActiveFocus() // Tab reads the directory with the same peek the columns view makes of an ancestor, @@ -197,6 +204,7 @@ ShellRoot { height: pane.height visible: pane.reclaimWalk && reclaimMapOn pane: pane + onStageRequested: reclaimMapOn = false } Flea.Preview { id: preview; pane: pane } From ce317336c347b5313c8a8a70e5dfa3087170ec0b Mon Sep 17 00:00:00 2001 From: Dzulfikar Ats Tsauri Date: Sun, 6 Sep 2026 19:04:46 +0700 Subject: [PATCH 3/4] review fixes: reclaim edges found by CodeRabbit - ListPaths ends a running reclaim (and its measured-bytes flag) before the picker's listing replaces it, exactly as list and search do. - The device check now refuses a foreign-filesystem entry before the target match, so a mount named node_modules is neither listed nor sized across the boundary; the cross-filesystem test grows the target-named child. - Top Sizes sorts its bars descending: tree discovery order is readdir order, not the rank the listing answers in. - A backend failure clears a running scan's flag, so esc closes instead of cancelling a walk nothing is walking. - The views overlay stores its painted shapes where click hit-testing reads them, so a click lands the cursor on the row it drew. - Quick Wins staging waits for the terminal line: a mid-walk rank would reshuffle the selection it hands back. - protocol.md states a cancelled walk's bytes total covers the trees sized before the cancel. --- docs/protocol.md | 3 ++- src/backend/reclaim.rs | 13 +++++++++---- src/backend/run.rs | 9 +++++++-- ui/PaneWire.qml | 3 +++ ui/ReclaimMap.qml | 5 ++++- ui/js/ReclaimTree.js | 2 +- 6 files changed, 26 insertions(+), 9 deletions(-) diff --git a/docs/protocol.md b/docs/protocol.md index 5ac031ed..ba7e8444 100644 --- a/docs/protocol.md +++ b/docs/protocol.md @@ -342,7 +342,8 @@ The walk is answered in three parts, the same shapes a `search` answers: 2. `reclaiming` lines carrying the match count, the entries scanned and the bytes sized so far, at most one every 100 ms while the walk runs. 3. One terminal `reclaimed` line, written after the ranking, carrying the final counts and - the bytes total the rows cover. + the bytes total of the rows the walk sized; a walk cancelled mid-sizing reports the bytes + it completed, not the total across rows it never sized. `list`, `sort` and `search` each end a running reclaim before they touch the listing, and answer their own lines after the reclaim's terminal `reclaimed`. diff --git a/src/backend/reclaim.rs b/src/backend/reclaim.rs index d3e8bbce..d75fca5f 100644 --- a/src/backend/reclaim.rs +++ b/src/backend/reclaim.rs @@ -149,6 +149,13 @@ impl Reclaim { continue; } let child = if rel.is_empty() { name.to_string() } else { format!("{}/{}", rel, name) }; + // A directory on another filesystem is refused before anything else about it is + // considered: a mount that happens to be named like a target would otherwise be + // listed and then sized across the boundary, network share included. + let entry_dev = entry.metadata().map(|m| m.dev()).unwrap_or(0); + if self.dev != 0 && entry_dev != 0 && entry_dev != self.dev { + continue; + } // corner: there is no hidden flag on a reclaim — .venv and __pycache__ are the point. if TARGETS.contains(&name.as_ref()) { // A match is reported but never descended into: its own size already covers @@ -157,10 +164,6 @@ impl Reclaim { self.found.push(child); continue; } - let entry_dev = entry.metadata().map(|m| m.dev()).unwrap_or(0); - if self.dev != 0 && entry_dev != 0 && entry_dev != self.dev { - continue; - } self.pending.push(child); } } @@ -291,6 +294,8 @@ mod tests { let d = TestDir::new("cross"); d.dir("proj/target"); d.file("proj/target/app", "x"); + d.dir("proj/node_modules/left-pad"); + d.file("proj/node_modules/left-pad/index.js", "x"); d.dir("proj/src"); d.file("proj/src/main.rs", "fn main() {}"); diff --git a/src/backend/run.rs b/src/backend/run.rs index fface639..00e5bc44 100644 --- a/src/backend/run.rs +++ b/src/backend/run.rs @@ -229,8 +229,13 @@ fn handle_line( } out.flush().ok(); } - Request::ListPaths { paths, first } => - listpaths::answer(out, st, pool, tb, &paths, first), + Request::ListPaths { paths, first } => { + // The picker's listing is the walk's to fill no more than any other: the reclaim ends + // before it, and its measured-bytes flag goes with the listing it measured. + end_reclaim(out, st, pool, true); + st.reclaim_sizes = false; + listpaths::answer(out, st, pool, tb, &paths, first); + } Request::Window { start, count } => { write_window(out, st, start, count, tb); out.flush().ok(); diff --git a/ui/PaneWire.qml b/ui/PaneWire.qml index 4b836e81..701c442f 100644 --- a/ui/PaneWire.qml +++ b/ui/PaneWire.qml @@ -300,6 +300,9 @@ Item { pane.held = 0 pane.rows = [] pane.cursorIndex = 0 + // A dead backend answers no terminal walk line either, so a running scan's flag + // goes with it: otherwise esc would keep cancelling a walk nothing is walking. + pane.searchRunning = false // No transferdone is coming from a backend that is gone, and nothing else ends a // running transfer, so the card would crawl over a dead child until the app closed. pane.transfer = Ops.emptyTransfer() diff --git a/ui/ReclaimMap.qml b/ui/ReclaimMap.qml index bbf39cdd..7988817d 100644 --- a/ui/ReclaimMap.qml +++ b/ui/ReclaimMap.qml @@ -139,7 +139,9 @@ Rectangle { id: qwTap acceptedButtons: Qt.LeftButton onSingleTapped: { - if (root.treeCount > 0 && root.pane) { + // Staging waits for the scan to finish: a walk still running would rank its + // rows once more and reshuffle the selection being handed back. + if (root.treeCount > 0 && root.pane && !root.pane.searchRunning) { root.pane.selectAll() root.stageRequested() } @@ -163,6 +165,7 @@ Rectangle { var ctx = canvas.getContext("2d") var drawn = ReclaimTree.layout(root.mode, root.model, width, height, Date.now() / 1000) canvas.shapes = drawn + root.shapes = drawn ReclaimPaint.draw(ctx, drawn, root.palette, root.pickedNode, width, height) } diff --git a/ui/js/ReclaimTree.js b/ui/js/ReclaimTree.js index 186577a4..475f1983 100644 --- a/ui/js/ReclaimTree.js +++ b/ui/js/ReclaimTree.js @@ -285,7 +285,7 @@ function bubbleR(node, parent, cap) { // The ranked listing drawn as bars: heaviest at the top, the number at the end of each bar. function topsizes(root, x, y, w, h, out) { - var items = leaves(root) + var items = leaves(root).slice().sort(function (a, b) { return b.bytes - a.bytes }) var max = 0 for (var i = 0; i < items.length; i++) { if (items[i].bytes > max) { From 54f4eb8db8cd19e3469fba7059582af76b4add99 Mon Sep 17 00:00:00 2001 From: Dzulfikar Ats Tsauri Date: Sun, 6 Sep 2026 19:10:22 +0700 Subject: [PATCH 4/4] shell: restore the back arrow the reclaim wiring dropped The chrome edit that added the reclaim button replaced the line the back arrow's handler lived on, so a click on the arrow stopped going back. onBackRequested is back on the pane, next to its up sibling. --- ui/shell.qml | 1 + 1 file changed, 1 insertion(+) diff --git a/ui/shell.qml b/ui/shell.qml index 76cddd6c..3dc657a8 100644 --- a/ui/shell.qml +++ b/ui/shell.qml @@ -115,6 +115,7 @@ ShellRoot { reclaimMapOn = !reclaimMapOn } } + onBackRequested: pane.goBack() onUpRequested: pane.openParent() onSearchRequested: pane.act("search") // The path bar's four. The pane navigates and answers for the keyboard exactly as it