Skip to content

Add grid module, .grid_finder, and .grid_monitor tools - #169

Open
Baharis wants to merge 12 commits into
instamatic-dev:mainfrom
Baharis:grid
Open

Add grid module, .grid_finder, and .grid_monitor tools#169
Baharis wants to merge 12 commits into
instamatic-dev:mainfrom
Baharis:grid

Conversation

@Baharis

@Baharis Baharis commented Sep 1, 2026

Copy link
Copy Markdown
Member

Context

Instamatic has already some tools to deal with the "grid" i.e. the copper scaffolding used to hold thin carbon film with the samples. Most of the routines deal with its presence in a occasional rather than exhaustive manner: by looking at the grid in imaging mode, one can quickly realize that some part of detector is shadowed and simply ignore it from further analysis. However, the grid is never treated as an object of investigation itself, as none of the methods currently present require this.

This PR introduces a new small module, instamatic.grid. It exists to collect all functionality, where the grid rather than a crystal sample is a target of investigation itself. The new module stores functionality used to: find grid position (currently using two methods: beam scanning/shadowing and manually), refine grid parameters against points on the edge, read or write its geometry from file, draw current grid model, predict available area and new window positions, and many more.

instamatic.grid_finder

New functionality is available via two scripts: instamatic.grid_finder and .grid_monitor. The finder can be ran in manual or auto mode. instamatic.grid_finder manual opens a VideoStreamFrame and asks user to travel around the window edge while clicking left mouse button. Each click adds current stage position to the list of "intercepts". In auto mode, the same "intercepts" are found by traveling from the origin in star shape and binary-searching the edge based on total counts on the camera. The grid is then fit to said "intercepts". Everything is saved live in a --file, by default grid.yaml:

grid_type: square
geometry: {x: 20666.375407617892, y: 18128.3077938044, t: 0.38165546590960825, w: 87736.63246809371,
  h: null, s: 38631.338330394705}
intercepts:
  0: [[65444.0, 27.0], [-19793.0, 34353.0], [-14847.0, -25784.0], [48881.0, 62388.0],
    [-23288.0, 4247.0], [32901.0, -26104.0], [59302.0, 32444.0], [14535.0, 62559.0],
    [-23484.0, 20414.0], [-22902.0, -13034.0], [9030.0, -26100.0], [65626.0, -22599.0]]
  1: [[191283.0, 27395.0], [102469.0, 41264.0], [103015.0, -8876.0], [160380.0, 63341.0],
    [103751.0, 16223.0], [171311.0, -25319.0], [191163.0, 56144.0], [130454.0, 63141.0],
    [102428.0, 29005.0], [102898.0, 3347.0], [137893.0, -25476.0], [191446.0, -3681.0]]

This grid file can be reused by other experiments, further improved by running instamatic.grid_finder, or plotted on some provided ax object using plot_grid from instamatic.grid.artist, which for the file above produces:

grid

instamatic.grid_monitor

Since monitoring the quality of grid determination live can be useful not only in an complex pipeline, but also by itself, the plotting function is granted to the user also directly via a simple GUI, where a path to grid --file can be given:

image

In this particular case, the grid geometry is fitted against intercept points from four windows, indexed 0 to 3. Position of window 0 has been determined manually by traversing along the stage edge while regularly adding new points. Windows 1-3 were fit from 2nd-order 3-arms star search (3+3 points per window), with some-angular offset per window.

The new functionality may find some use by itself, but it is predominantly great as a prerequisite for long automated experiments that are a matter of ongoing development. Since the automated grid geometry determination relies on scanning with a beam only, it can be performed by a experiment that has no information about "imaging" config. The continuous-serial electron diffraction routine implemented in incoming patches uses this information to easily define areas where it can scan for new data, without any need to change detector config or any imaging data.

Additions

  • docs/programs.md: Added new programs to list, added new programs' documentation;
  • camera_serval.py: Fixed a small config bug introduced in Make get_movie setup eager, acquisition lazy; optimize serval #166;
  • grid/__init__.py: A few common type hints and utils used in the module;
  • grid/artist.py: Function plot_grid that draws grid details onto provided axes.
  • grid/finder.py: High-level tools to locate, read, write grid & intercept information;
  • grid/grid.py: Definitions, registry, fitting of grids: square, rectangular, hexagonal;
  • grid/monitor.py: GUI utility, standalone or embed-able, auto-plots grid geometry;
  • grid/sweeping.py: Sweepers used to find grid edges within the star_sweep function;
  • grid/window.py: Definition of individual windows used by higher-level grid.py;
  • pyproject.toml: Added instamatic.grid_finder and .grid_monitor entry points;
  • tests/test_grid.py: Test suite covering 70% of new functionality (not GUI, scripts).

Notes

This is a direct prerequisite to a new experiment routine that is already ready, functional, tested, proved to give good 3D ED data. In order to merge this new routine before October, I'd like to merge this PR before the end of the next week.

@Baharis
Baharis requested a review from stefsmeets September 1, 2026 18:54
@Baharis Baharis self-assigned this Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant