Skip to content

Add bottles linked to compounds and storage places; new add-bottle page - #4

Open
rcb1409 wants to merge 26 commits into
mainfrom
feature/storage-and-compounds
Open

Add bottles linked to compounds and storage places; new add-bottle page#4
rcb1409 wants to merge 26 commits into
mainfrom
feature/storage-and-compounds

Conversation

@rcb1409

@rcb1409 rcb1409 commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator

Why

Bottles in the ELN now point to shared compounds (name, structure, hazards) and to storage places (eLabFTW 6 containers: place + amount + unit), instead of 16 free-text fields per bottle. The data on the live ELN was moved over in September (225 bottles placed, 213 compound links). This PR updates the app to work the same way.

What changes

Adding a bottle: new page /add_bottle_interface (the old address redirects)

  • Pick the chemical from a dropdown (filter by name or CAS), or find it in PubChem by CAS and add it. Existing compounds are never overwritten, and a deleted one can be restored.
  • Pick Room → Place, or add a new place, with "Did you mean…?" for near-duplicates.
  • Amount + unit (units follow the State, no conversion) and number of bottles (1–20, one id and label each).
  • Create links the compound(s), adds the storage entry, the structure image, and tags: GHS hazards and "Peroxide former: A–D" (from the EPA CompTox lists in automations/peroxides).
  • If bottles from the same lot are already in the ELN, it asks whether this bottle already has a label.
  • Then Print labels.

Hazards come straight from PubChem's GHS data: the EU official classification first, otherwise ECHA's main summary, otherwise the other sources. eLabFTW's lookup by PubChem ID drops the hazard flags.

Scanner page

  • "Change Location" is back: STORAGE=<id> QR codes move the bottle's storage entry, and moving no longer marks a bottle as opened.
  • "Mark Empty" also sets the amount to 0.
  • The broken SUBLOC= command is removed.

Removed

  • Autofill (route, timer, settings).
  • The old add form, with /template, /search, /add_resource, /add_option, fill_info, search_process and pubchempy.
  • /change_location and /get_locations.
  • scripts/add_all_compounds.py, which used eLabFTW's upserting import that overwrote compound #80, and scripts/associate_compounds.py, which used a link path that doesn't exist.

Smaller fixes: /print and /create_label build the PDF in memory instead of a shared static/print.pdf, and answer 400/401 instead of 500.

Testing

  • 187 offline tests and 9 live tests pass, run in the Docker image.
  • Every step was tried against a local copy of the ELN (eLabFTW 6.0.5 restored from the nightly dump): creating bottles, PubChem import and restore, places, tags, the same-lot question, printing, change location, and mark empty.

Deploy notes

  • git pull and docker compose up -d --build in deploy/.
  • Run client/install.sh to remove the old eln-autofill.timer.
  • The container now calls pubchem.ncbi.nlm.nih.gov directly.
  • Still to do in follow-ups: the peroxide reminder (reads Storage, before Nov 1), labels with hazard pictograms, filling in hazards for the older compounds, and trimming the templates after the deploy.

🤖 Generated with Claude Code

rtg-1409 and others added 26 commits October 1, 2026 19:13
@rtg-1409 @claude
Add routes listing existing compounds and storage places
c22f4c6 
The add-resource page will offer dropdowns of what already exists in the
ELN instead of free-typed fields: /compounds_list returns every compound as
{id, name, cas, formula} sorted by name, /storage_tree every storage place
as {id, name, parent_id, full_path} so the page can build Room -> Place
dropdowns. Both act as the caller's API key, like /categories.

get_compounds() now returns all compounds (it was capped at 1000) and raises
on HTTP errors; get_storage_units() is new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Import compounds from PubChem without overwriting existing ones
92fa382 
eLabFTW's compound create (both the PubChem "duplicate" import and a plain
POST) upserts: if CAS, InChIKey or PubChem ID matches any compound, even a
deleted one, it overwrites that compound and returns its id. In the lab's
ELN that turned Hexitol (#80) into Mannitol and restored deleted #8.

New eln_common/compound_import.py:
- find_existing() checks a PubChem result against every compound (all
  states) on PubChem ID, InChIKey and the CAS that would be saved
- create_compound_safely() refuses a clash (CompoundClash), saves the
  user's CAS rather than PubChem's, and stops if eLabFTW still returns an
  existing id

Routes for the add-resource page:
- GET /compounds/pubchem?cas=|name=  read-only preview, flags existing
- POST /compounds {cid, cas}  re-fetches by PubChem ID and creates safely;
  201 new, 409 with the existing compound, 404 not in PubChem

Tried on a local 6.0.5 copy: new compound created, repeat refused (409),
deleted compound #2 refused and left deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Remove autofill
2b3eca0 
Autofill copied PubChem details (full name, SMILES, molecular weight,
PubChem and hazards links) into every resource's extra fields, creating
the fields when missing, every 10 minutes and after each new resource.
Resources are now linked to a shared compound that holds those details, so
the copies are redundant, they would put the old fields back on trimmed
templates, and the re-saves flooded the changelog.

Removed: automations/autofill.py, POST /api/autofill, the background
autofill after /add_resource, the eln-autofill systemd units and the
"autofill" timer task, and the legacy auto_upload_labels setting (labels
are generated on the fly). install.sh now takes the old autofill timer off
hosts that still have it.

Kept for the new add-resource flow: automations/image_generator.py (the
structure image will be drawn once when a resource is added) and the
chemical_categories setting, reworded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Read compounds and their hazard pictograms straight from PubChem
a187838 
eLabFTW's PubChem lookup by PubChem ID leaves out the hazard flags, so
compounds added through the app had no hazard pictograms. The app now asks
PubChem's REST API directly for the details, CAS and GHS pictograms, and
eLabFTW only stores the compound. Each pictogram (GHS01-09) turns on its
eLabFTW hazard flag; POST /compounds reports the pictograms found, or null
when PubChem has no GHS classification (check the supplier's SDS).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Offer to restore a deleted compound instead of refusing it
d2f3562 
When the only compound sharing a new one's PubChem ID, InChIKey or CAS is
a deleted one, POST /compounds answers 409 with can_restore, and sending
{cid, cas, restore: <id>} brings that compound back with fresh details
from PubChem, the user's CAS and its hazard flags. A live match is still
refused, since creating would overwrite it and the bottles linked to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Add storage places from the app without near-duplicates
cd9b648 
POST /storage_units {name, parent_id, confirm} adds a place inside an
existing room or place. eLabFTW accepts any name, so the app first compares
it with the places in the same parent: the same name (ignoring case and
spaces) is refused with the existing place, and a similar one ("Flamable
cabinet") needs confirm: true. Names that differ in their numbers ("Shelf 1",
"Shelf 2") are not similar. New rooms are left to admins in eLabFTW.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Add bottles linked to their compound and storage place
d0f4d24 
POST /resources {category, title, compounds, storage: {place_id, amount,
unit}, fields} creates the bottle from its template, keeps only the fields
about the bottle itself (place and amount go to Storage, chemical details to
the linked compounds; polymers keep theirs), links the compounds, adds the
storage container and the structure image. The request is checked first so
a bad one creates nothing; once the bottle exists, a later step that fails
is reported in "problems" instead of deleting it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Tag new bottles with their compounds' hazards and peroxide class
a0ee261 
Right after the compounds are linked, the bottle gets one tag per GHS hazard
flag set on them (Flammable, Corrosive, Health hazard...) and, when a
compound's CAS or InChIKey is on the EPA CompTox peroxide-former lists in
automations/peroxides, "Peroxide former: A-D". add_tag now raises on failure
so a tag that could not be added is reported in "problems".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Warn before adding a bottle that is already in the ELN
2dcf155 
Before creating, POST /resources looks for current bottles with the same
lot number and manufacturer (ignoring case) holding the same chemical: a
shared linked compound, or the same CAS field for older, unlinked bottles.
If any match, it answers 409 with links to them and creates nothing;
confirm_duplicate: true adds the bottle anyway. No lot number, no check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Ask about the label instead of blocking same-lot bottles; add several…
51b18de 
… at once

Bottles with the same lot, supplier and chemical are often different bottles
of one order, so a match is no longer treated as a duplicate: the 409 lists
the bottles from that lot and asks whether this bottle already has an ELN
label (confirm_same_lot: true adds it). POST /resources also takes "count"
(1-20) to add identical bottles of an order in one go, each with its own id;
the reply is now {bottles: [{id, url, tags, problems}], problems}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
New add-bottle page, part 1: category, compounds and the bottle's own…
5088935 
… fields

/add_bottle_interface is a new page next to the old form. GET /bottle_form
gives it each bottle category (templates with a State field), the compounds
it needs (one chemical; a solution's chemical and solvent; none for polymers)
and the fields that stay on the bottle. Picking from the compound dropdown
(filter by name or CAS) shows the hazards and peroxide class the bottle will
be tagged with (now part of /compounds_list) and fills in the name and CAS.
For now the page shows the request "Create" will send.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Add-bottle page, part 2: room and place dropdowns, add a new place
7b71ec0 
The Room dropdown lists the top of the storage tree; Place then lists only
that room's places (at any depth, without the room in the name) and ends
with "+ Add a new place…". That opens a small form (inside the room or one
of its places) that calls POST /storage_units: a new place is added and
selected, the same name selects the existing place, and a similar name asks
"Did you mean …?" with "Use it" or "Add anyway".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Add-bottle page, part 3: amount, unit and number of bottles
7fb119f 
"3. How much" holds the State field (moved up from the bottle's fields),
the amount with a unit dropdown, and the number of identical bottles (1-20).
The State's usual units come first (Liquid: μL mL L, Solid: μg mg g kg,
Gas: bar L g), then eLabFTW's other units; changing the State switches to
its usual unit unless the chosen one still fits. Nothing is converted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Add-bottle page, part 4: the Create button
5605e84 
Create checks the form first (pick from the lists, a room and place,
amount, unit, name and the template's required fields), outlines what is
missing and sends nothing until it is complete. The button is disabled while
POST /resources runs, so a double click can't make two bottles. 201 shows
each new bottle's number (linked to eLabFTW) with its tags and any step to
finish by hand; 409 lists the bottles from the same lot and asks whether
this bottle already has a label, with "It's another bottle: add it".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Add-bottle page, part 5: find a missing compound in PubChem
4b5b1ee 
Under each compound picker, "Not in the list? Search PubChem" looks the CAS
number (or name) up with GET /compounds/pubchem. Each match says what can be
done: a compound already in the ELN (maybe under another name or CAS) is
offered with "Use #id"; a deleted one with "Restore #id"; a new one with "Add
this compound to the ELN" (POST /compounds). The added or restored compound
is then picked for the bottle, with a note when PubChem has no hazard data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Choose hazard pictograms the way an SDS would, not from any source
6cb44a0 
Taking a pictogram from any PubChem source over-warned: water got "Health
hazard" from a small ECHA group that is really a polymer, and hydrogen
peroxide got Japan's pictograms for concentrated grades. Now the EU's
official classification (Regulation 1272/2008) is used when there is one;
otherwise ECHA's main summary (most company reports), with no pictograms if
most reports say not hazardous; otherwise the other sources combined.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Add-bottle page, part 6: print the new bottles' labels
e0543d8 
After creating, "Print label(s)" sends the new ids to POST /print and opens
the PDF in a new tab (opened on the click, since browsers block tabs opened
after a wait; a link is shown if pop-ups are blocked). /print now makes the
PDF in memory instead of writing the shared static/print.pdf, so two people
printing at once can't get each other's labels, and answers 400 for bad ids
and 401 without an API key instead of failing with 500.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Remove the old add-resource form and the code only it used
1b8e04b 
The add-bottle page replaces the 16-field form, so this removes
add_resource.html, render_form.js, the /template, /search, /add_resource and
/add_option routes, search_process.py and fill_info.py (check_if_cas moves
to pubchem.py), and pubchempy. Also removed: scripts/add_all_compounds.py,
which used eLabFTW's upserting PubChem import, and associate_compounds.py,
which used a link path eLabFTW does not have, with the Resource_Manager
methods only they and the old form used. The start page's button opens the
new page and the old address redirects there. /create_label now makes its
PDF in memory like /print.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Scanner: change location moves the bottle's storage entry
09ea181 
"Change Location" is back on the scanner page. Its popup shows a QR code
(STORAGE=<id>) for every storage place from /storage_tree; scanning one
after the bottles calls POST /move_to_storage, which moves each bottle's
storage entry there (eLabFTW logs the move) and no longer marks the bottle
as opened. Bottles with no storage entry or several are reported, not
guessed. This replaces /change_location (the Location text field),
/get_locations (the template's old Location options) and the SUBLOC=
command, whose route never existed. The label generator's "Location" QR
type now picks a storage place and encodes STORAGE=<id>.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Scanner: mark empty also sets the amount to 0
ffd0ad0 
Marking a bottle empty now sets its status to Empty and the amount in each
of its storage entries to 0 (eLabFTW logs the change), so the inventory no
longer counts a used-up bottle as full. The bottle and its place stay on
record. The scanner page shows any bottle that could not be emptied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Scanner page: "Resource Manager" heading
786f1f8 
The start page gets a "Resource Manager" heading and browser-tab title
(was "Inventory Manager"), and the change-location popup says what its QR
codes do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Scanner: routine-check steps on Mark Open, Tested and Mark Empty
5c2530b 
automations/routine_checks.yaml maps a bottle tag to a check (Peroxide
former A: test every 3 months, B and C: 6, D: 12). Mark Open now adds that
check as a step on the bottle, e.g. "Test for peroxides (class B), due
2027-04-06" with the same deadline (the date is also in the text because
eLabFTW clears the deadline when a step is ticked). The new "Tested" command
ticks the open check and adds the next one; Mark Empty closes it with a note
(ticked, not deleted). Mark Open moves into bottle_actions and reports
already-opened bottles per bottle instead of stopping at the first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Scanner page: let the command buttons scroll
c3fb02e 
The main area was overflow: hidden, so with the heading and the Tested
command the last row of QR buttons (Associate with Experiment, Custom Label)
was cut off on smaller screens. It now scrolls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Name the app "Toolkit" and add page tabs in the left column
2b0d91b 
The scanner page is now headed "Toolkit". Both pages share a left column
with two tabs, "Registry" (the scanner page, with the scanned bottles below
the tabs) and "Add resources" (the add-bottle page, which loses its Back
link). The tab styles live in static/tabs.css so a page such as Experiments
can be added later with one more tab.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Toolkit tabs: Registry shows every command, Resources the resource ac…
14a3adc 
…tions

The left column's tabs now filter the QR commands on the scanner page
instead of switching pages: Registry shows all of them, Resources only the
actions on bottles (Add resource, Change Location, Mark Open, Tested, Mark
Empty, Print, Open Page, Associate with Experiment), leaving out the general
Clear, Batch and Custom Label. Switching keeps the scanned list; the tab is
kept in the address (?tab=resources). The add-bottle page links back to
both tabs with Resources highlighted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rtg-1409 @claude
Name the app "ELN Toolkit"
3bbe133 
Heading, browser-tab titles and the settings page's back link now say
"ELN Toolkit" instead of "Toolkit".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@rcb1409 @rtg-1409