Skip to content

Refactor/figures#202

Open
clairemerker wants to merge 14 commits into
paper-figuresfrom
refactor/figures
Open

Refactor/figures#202
clairemerker wants to merge 14 commits into
paper-figuresfrom
refactor/figures

Conversation

@clairemerker

Copy link
Copy Markdown
Contributor

AI assisted implementation!

This PR is an attempt to improve the plotting workflow for publication to be able to generate them from the snakemake workflow as well as interactively.

A documentation is currently available in docs/publication_figures.md

I only tested creating figures using both workflows, never triggered the whole pipeline incl. results creation... So not sure if the whole workflow is still ok, but I think it should...

@clairemerker clairemerker requested a review from jonasbhend June 26, 2026 11:42

@jonasbhend jonasbhend left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking really good. Thanks for seeing this through. One question. The current workflow still overwrites results based on stations and analysis, as both the manifest and the output paths are truth-hash-free (or experiment-hash-free). I think we should change that, such that we can run station and analysis-based workflows, without overwriting?

@clairemerker

Copy link
Copy Markdown
Contributor Author

@jonasbhend Good point!

Do you mean the snakemake workflow only or also when runing the standalone plotting?
For the snakemake workflow I don't see an issue, I can make the manifest and the ouput folder hash-dependent.
But then the standalone workflow needs to find the right manifest without knowing the hash... Not sure how to deal with that... Any idea?

And generally, is it enough to use the truth-hash, or do we need to hash the config?

@jonasbhend

Copy link
Copy Markdown
Contributor

@jonasbhend Good point!

Do you mean the snakemake workflow only or also when runing the standalone plotting? For the snakemake workflow I don't see an issue, I can make the manifest and the ouput folder hash-dependent. But then the standalone workflow needs to find the right manifest without knowing the hash... Not sure how to deal with that... Any idea?

And generally, is it enough to use the truth-hash, or do we need to hash the config?

Well, we've done this before and it didn't work. But I would probably just use the truth label to denote the manifests. That way it is easy to operate in standalone mode and 'reasonably' fail-safe. If you trigger a full rerun, this will anyway be recomputed if the config changes and we probably don't want to keep old manifests lying around.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants