[doc] Move cleanup part from README to documentation#1469
Conversation
BenjaminPelletier
left a comment
There was a problem hiding this comment.
Thanks, I think this is very timely as I expect there will be a lot of demand for this documentation in the near future. I have a few substantive tweaks and a number of typo-level suggestions.
| - Ensure a backup of the data is available. | ||
| - Double-check the TTL values passed to `--rid_ttl` and `--scd_ttl`. | ||
|
|
||
| Expiration of entities is preferably determined through their end times. When end times are not available, the last update times are used instead. |
There was a problem hiding this comment.
I think end times should never not be available -- although the SCD schema seems to not have end_time as NOT NULL, it probably should be NOT NULL as no unspecified end times are allowed
| Expiration of entities is preferably determined through their end times. When end times are not available, the last update times are used instead. | |
| Expiration of entities is preferably determined through their end times. In the unusual event that an end time is not available, the last update time is used instead. |
There was a problem hiding this comment.
Yes, that coming from old documentation. Should we track it as an issue to have time not nullables?
There was a problem hiding this comment.
Maybe as a P3. I think the important thing here is noting that it would be unusual/unexpected.
|
|
||
| Beyond running `db-manager evict` manually, the DSS deployment tooling can schedule the cleanup as a recurring Kubernetes `CronJob`. Three deployment paths expose the same set of evict knobs, but will always run with the `--delete` flat set. | ||
|
|
||
| Shared default: RID cleanup is enabled by default (`*/30 * * * *`, `ttl = 30m`); SCD cleanup is disabled by default (suggested schedule `0 2 * * *`, `ttl = 2688h` - i.e. 2 x 56 days). |
There was a problem hiding this comment.
@mickmis could you double-check that my suggestion is accurate? I think the reason the defaults are the way they are is that each DSS instance is supposed to be responsible for cleaning up its own RID stuff (and deletion is limited to entities created by the DSS instance doing the deletion, as determined by locality), but SCD cleanup is a global activity and therefore DSS instance operators would want to at least coordinate to avoid cleaning up more than necessary (or perhaps designated a single USS as responsible for cleanup).
| Shared default: RID cleanup is enabled by default (`*/30 * * * *`, `ttl = 30m`); SCD cleanup is disabled by default (suggested schedule `0 2 * * *`, `ttl = 2688h` - i.e. 2 x 56 days). | |
| Shared default: RID cleanup is locality-limited and therefore is enabled by default (`*/30 * * * *`, `ttl = 30m`); SCD cleanup is not locality-limited and therefore disabled by default (suggested schedule `0 2 * * *`, `ttl = 2688h` - i.e. 2 x 56 days). |
Co-authored-by: Benjamin Pelletier <BenjaminPelletier@users.noreply.github.com>
Co-authored-by: Benjamin Pelletier <BenjaminPelletier@users.noreply.github.com>
Co-authored-by: Benjamin Pelletier <BenjaminPelletier@users.noreply.github.com>
Co-authored-by: Benjamin Pelletier <BenjaminPelletier@users.noreply.github.com>
Co-authored-by: Benjamin Pelletier <BenjaminPelletier@users.noreply.github.com>
Co-authored-by: Benjamin Pelletier <BenjaminPelletier@users.noreply.github.com>
Co-authored-by: Benjamin Pelletier <BenjaminPelletier@users.noreply.github.com>
Co-authored-by: Benjamin Pelletier <BenjaminPelletier@users.noreply.github.com>
Co-authored-by: Benjamin Pelletier <BenjaminPelletier@users.noreply.github.com>
Improve documentation by moving and expending on cleanup's README into documentation.
Partially LLM-assisted ^^'