Doppio live processing requirements - #790
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
… resubscribe to transport object
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #790 +/- ##
==========================================
- Coverage 54.71% 54.67% -0.04%
==========================================
Files 104 104
Lines 11185 11262 +77
Branches 1481 1485 +4
==========================================
+ Hits 6120 6158 +38
- Misses 4740 4774 +34
- Partials 325 330 +5 🚀 New features to boost your workflow:
|
|
I've had a good look over and I feel pretty happy with it! I had forgotten how many changes had been made overall. As discussed on Friday the logic for the paths in processing_params.py will need looking at, and further testing for the job reservation system... I'll have another look at this soon. |
| core = Path("/") / Path(*parts[: visit_idx + 1]) | ||
| ppath = Path("/") / Path(*parts) |
There was a problem hiding this comment.
From the limited testing I've done, it looks like Path(*parts...) will give you the same result as Path("/") / Path(*parts...), so this could actually be simplified.
| if murfey.server._transport_object: | ||
| murfey.server._transport_object.send( | ||
| murfey.server._transport_object.feedback_queue, zocalo_message | ||
| ) | ||
| else: | ||
| logger.error( | ||
| f"Pre-processing was requested for {sanitise(str(session_id))} " | ||
| "but no Zocalo transport object was found" | ||
| ) | ||
| return proc_params |
There was a problem hiding this comment.
If I'm reading this correctly, it will return None when successful and proc_params if unsucessful. Is this intended behaviour?
| session_id: MurfeySessionID, | ||
| dcg_params: DCGroupParameters, | ||
| db=murfey_db, | ||
| db: Session = murfey_db, |
There was a problem hiding this comment.
You appear to be using murfey.util.db.Session as the type hint, which is an SQLModel table. I'm not sure how the tests have passed, but this should be pointing to sqlmodel.Session instead.
This would be where I'd usually alias the SQLModel Session and Murfey DB Session so that they are clearly distinguishable.
e.g.
from sqlmodel import Session as SQLModelSession
import murfey.util.db as MurfeyDB
|
|
||
| @router.get("/sessions/{session_id}/rsyncers", response_model=List[RsyncInstance]) | ||
| def get_rsyncers_for_client(session_id: MurfeySessionID, db=murfey_db): | ||
| def get_rsyncers_for_client(session_id: MurfeySessionID, db: Session = murfey_db): |
There was a problem hiding this comment.
Same error in this module. Session is referring to murfey.util.db.Session, which is an SQLModel table. The type hint should be pointing to sqlmodel.Session.
| SPARelionParameters.pj_id == collected_ids[2].id | ||
| ) | ||
| ).one() | ||
| ) |
There was a problem hiding this comment.
This is an SQLModel table, so it's apparently possible to use table.model_dump() to convert this into a dictionary structure.
proc_params = db.exec(
select(SPARelionParameters).where(
SPARelionParameters.pj_id == collected_ids[2].id
)
).one_or_none()
proc_params_dict = proc_params.model_dump() if proc_params else None
4 participants
Various changes are needed for doppio-live to work.
These are primarily connected to the job number assigned, as we need to be able to vary these if the default pipeline file is being changed by a user. For eBIC operations, the behaviour should be as before, but for doppio-live the jobs are assigned properly using ccpem-pipeliner.
Changes are also needed for the output motion corrected mrc file. The output path for doppio uses the pipeline star file to find a suitable job number. It currently does not include the GridSquare folder so may need changing again in future. The behaviour at eBIC should be unchanged.
Other bits:
picker_murfey_idfrom the database tablesmurfey.serverand not the_transport_objectdirectly