diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 7f2f38a..df3667a 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -17,6 +17,7 @@ attribution ledger. | --- | --- | --- | --- | --- | | Root repository router | Current `VeryMath/AI4Math-Optimization` repository | [Conan Xu](https://github.com/ConanXu-math), conanxu <1845830029@qq.com> | Root README, bilingual repository map, package routing layer, and public repository structure. | Preserved through repository commit history. | | `skills/osqp-solver` | New package prepared in the AI4Math Skill Library from official OSQP documentation | [Conan Xu](https://github.com/ConanXu-math) | Package commissioning, integration, review, and release stewardship; implementation was AI-assisted and independently tested. | Preserve through the upstream integration commit and pull request. | +| `skills/optskills` | [fujiwaranoM0kou/OptSkills](https://github.com/fujiwaranoM0kou/OptSkills) | [Haochen Yang](https://github.com/fujiwaranoM0kou) and the OptSkills paper authors | Released optimization problem-archetype skill cards; VeryMath adds standalone packaging, indexing, documentation, and update instructions. | Original MIT notice is preserved in `skills/optskills/LICENSE`; source scope is recorded in `skills/optskills/SOURCES.md`; import authorship is preserved in the import commit. | | Optimization skill packages | Repository history and imported optimization work | [ricercar77](https://github.com/ricercar77) | Optimization skill contributions and package content. | Preserved through GitHub contributor graph, repository history, and this ledger. | | `skills/linear-programming`, `skills/mixed-integer-programming`, `skills/second-order-cone-programming`, `skills/or-solver` | Current package metadata | 李爽夕 | LP, MIP, SOCP, solver-setup workflow content, examples, and benchmark materials. | Recorded in package metadata and this ledger. | diff --git a/README.md b/README.md index c387ed3..a34873d 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ manifold-constrained optimization. [中文说明](README.zh-CN.md) · [Contributors](CONTRIBUTORS.md) · [Skill packages](#skill-packages) · [Installation](#installation) · [Quick start](#quick-start) · [Security model](#security-and-scope) ![version](https://img.shields.io/badge/version-0.1.0-blue) -![skills](https://img.shields.io/badge/skills-7-2ea44f) +![skills](https://img.shields.io/badge/skills-8-2ea44f) ![license](https://img.shields.io/badge/license-MIT-green) @@ -39,6 +39,9 @@ package that matches the problem class. | [`second-order-cone-programming`](skills/second-order-cone-programming/) | SOCP modeling and cvxpy-based conic solver workflows. | [`README`](skills/second-order-cone-programming/README.md) · [`SKILL`](skills/second-order-cone-programming/SKILL.md) | | [`or-solver`](skills/or-solver/) | Shared solver detection, installation planning, license checks, and solver selection for OR skills. | [`README`](skills/or-solver/README.md) · [`SKILL`](skills/or-solver/SKILL.md) | | [`osqp-solver`](skills/osqp-solver/) | OSQP modeling, repeated solves, status gates, and independent verification for continuous convex QPs. | [`README`](skills/osqp-solver/README.md) · [`SKILL`](skills/osqp-solver/SKILL.md) | +| [`optskills`](skills/optskills/) | Standalone selection and use of 103 released OptSkills problem-archetype cards, with explicit upstream updates. | [`README`](skills/optskills/README.md) · [`SKILL`](skills/optskills/SKILL.md) | + +`optskills` is self-contained and does not require sibling packages. ## Installation @@ -57,6 +60,7 @@ Skill paths: - skills/second-order-cone-programming - skills/or-solver - skills/osqp-solver +- skills/optskills Steps: 1. Clone or update the repository locally. @@ -80,6 +84,7 @@ ln -s "$PWD/skills/mixed-integer-programming" ~/.codex/skills/mixed-integer-prog ln -s "$PWD/skills/second-order-cone-programming" ~/.codex/skills/second-order-cone-programming ln -s "$PWD/skills/or-solver" ~/.codex/skills/or-solver ln -s "$PWD/skills/osqp-solver" ~/.codex/skills/osqp-solver +ln -s "$PWD/skills/optskills" ~/.codex/skills/optskills ``` If your agent uses a different local Skill directory, replace `~/.codex/skills` with that configured path. @@ -112,6 +117,12 @@ For continuous convex QPs solved with OSQP, start with: skills/osqp-solver/SKILL.md ``` +For natural-language operations-research archetypes, start with: + +```text +skills/optskills/SKILL.md +``` + ## Repository Layout ```text @@ -126,6 +137,7 @@ AI4Math-Optimization/ ├── mixed-integer-programming/ ├── or-solver/ ├── osqp-solver/ + ├── optskills/ └── second-order-cone-programming/ ``` diff --git a/README.zh-CN.md b/README.zh-CN.md index 1bc6d25..6e2f521 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -8,7 +8,7 @@ [English](README.md) · [贡献者](CONTRIBUTORS.md) · [技能包](#技能包) · [安装](#安装) · [快速开始](#快速开始) · [安全边界](#安全边界) ![version](https://img.shields.io/badge/version-0.1.0-blue) -![skills](https://img.shields.io/badge/skills-7-2ea44f) +![skills](https://img.shields.io/badge/skills-8-2ea44f) ![license](https://img.shields.io/badge/license-MIT-green) @@ -35,6 +35,9 @@ | [`second-order-cone-programming`](skills/second-order-cone-programming/) | SOCP 建模和基于 cvxpy 的锥优化求解流程。 | [`README`](skills/second-order-cone-programming/README.md) · [`SKILL`](skills/second-order-cone-programming/SKILL.md) | | [`or-solver`](skills/or-solver/) | 为 OR skills 提供统一求解器检测、安装规划、license 检查和选择策略。 | [`README`](skills/or-solver/README.md) · [`SKILL`](skills/or-solver/SKILL.md) | | [`osqp-solver`](skills/osqp-solver/) | 面向连续凸 QP 的 OSQP 建模、重复求解、状态门槛与独立验证。 | [`README`](skills/osqp-solver/README.zh-CN.md) · [`SKILL`](skills/osqp-solver/SKILL.md) | +| [`optskills`](skills/optskills/) | 独立选择和使用 OptSkills 已发布的 103 个优化问题原型知识卡，并支持显式的上游更新。 | [`README`](skills/optskills/README.zh-CN.md) · [`SKILL`](skills/optskills/SKILL.md) | + +`optskills` 是自包含的，不依赖相邻技能包。 ## 安装 @@ -53,6 +56,7 @@ Skill 路径： - skills/second-order-cone-programming - skills/or-solver - skills/osqp-solver +- skills/optskills 请执行： 1. 本地 clone 或更新仓库。 @@ -76,6 +80,7 @@ ln -s "$PWD/skills/mixed-integer-programming" ~/.codex/skills/mixed-integer-prog ln -s "$PWD/skills/second-order-cone-programming" ~/.codex/skills/second-order-cone-programming ln -s "$PWD/skills/or-solver" ~/.codex/skills/or-solver ln -s "$PWD/skills/osqp-solver" ~/.codex/skills/osqp-solver +ln -s "$PWD/skills/optskills" ~/.codex/skills/optskills ``` 如果你的 agent 使用别的本地 Skill 目录，把 `~/.codex/skills` 替换成对应配置路径。 @@ -107,6 +112,12 @@ skills/cdopt-optimization/SKILL.md skills/osqp-solver/SKILL.md ``` +自然语言运筹优化问题原型从这里开始： + +```text +skills/optskills/SKILL.md +``` + ## 仓库结构 ```text @@ -121,6 +132,7 @@ AI4Math-Optimization/ ├── mixed-integer-programming/ ├── or-solver/ ├── osqp-solver/ + ├── optskills/ └── second-order-cone-programming/ ``` diff --git a/SKILL.md b/SKILL.md index add90d0..6c86ae8 100644 --- a/SKILL.md +++ b/SKILL.md @@ -20,6 +20,10 @@ Use this repository as a routing layer for optimization workflows. - `skills/or-solver/`: shared optimization solver setup and selection. - `skills/osqp-solver/`: continuous convex QP modeling, OSQP execution, status gates, and independent solution or infeasibility-certificate checks. +- `skills/optskills/`: standalone selection and use of released OptSkills + problem-archetype cards. Read its `SKILL.md`, combined + `skill_library/index.json`, and selected card files. It does not depend on + sibling skill packages. Prefer package-local instructions over this router when running a concrete workflow. diff --git a/skills/optskills/LICENSE b/skills/optskills/LICENSE new file mode 100644 index 0000000..b5e0785 --- /dev/null +++ b/skills/optskills/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Haochen Yang + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/skills/optskills/README.md b/skills/optskills/README.md new file mode 100644 index 0000000..2ca2005 --- /dev/null +++ b/skills/optskills/README.md @@ -0,0 +1,50 @@ +# OptSkills + +OptSkills is a standalone thin entrypoint to 103 released OptSkills +problem-archetype cards. It helps an agent select a card, formulate the +user's actual operations-research problem, solve it when the environment +permits, and report what was checked. + +## Source selection + +The library contains 93 NanoCO cards and 10 learned-only cards. It includes no +cluster duplicate and no `ingredients.json`. The selected cards and their +relative paths are listed in `skill_library/index.json`. + +## Installation + +Clone `VeryMath/AI4Math-Optimization`, then link only `skills/optskills` into +the agent's skill directory: + +```bash +git clone https://github.com/VeryMath/AI4Math-Optimization.git +cd AI4Math-Optimization +mkdir -p ~/.codex/skills +ln -s "$PWD/skills/optskills" ~/.codex/skills/optskills +``` + +## Quick start + +Use this prompt: + +> Read `SKILL.md`, select the relevant OptSkills cards through +> `skill_library/index.json`, formulate my problem, solve it if possible, and +> check the key constraints. + +## Updates + +Use this prompt only when an upstream update is wanted: + +> Update this standalone OptSkills package from the official upstream. Read `UPDATE.md`, report proposed changes first, and wait for approval before editing. + +## Boundaries + +Raw cards may contain placeholders or unverified examples. Being installed, +loaded, solved, and checked are different claims; report each state separately. +Normal use is package-local and does not call sibling skills or require the +upstream training, agent, chat, or embedding system. + +## License and sources + +See [SOURCES.md](SOURCES.md) for upstream attribution and [LICENSE](LICENSE) +for licensing information. diff --git a/skills/optskills/README.zh-CN.md b/skills/optskills/README.zh-CN.md new file mode 100644 index 0000000..3e1702c --- /dev/null +++ b/skills/optskills/README.zh-CN.md @@ -0,0 +1,43 @@ +# OptSkills + +OptSkills 是一个独立的薄入口，连接 103 张已发布的 OptSkills +问题原型卡片。它帮助智能体选择卡片、为用户的实际运筹问题建模、在环境允许时求解，并如实报告已检查的内容。 + +## 来源选择 + +该库包含 93 张 NanoCO 卡片和 10 张仅来自 learned 的卡片；不包含重复的 +cluster 卡片，也不包含 `ingredients.json`。选中的卡片及其相对路径列在 +`skill_library/index.json` 中。 + +## 安装 + +克隆 `VeryMath/AI4Math-Optimization`，然后只把 `skills/optskills` +链接到智能体的技能目录： + +```bash +git clone https://github.com/VeryMath/AI4Math-Optimization.git +cd AI4Math-Optimization +mkdir -p ~/.codex/skills +ln -s "$PWD/skills/optskills" ~/.codex/skills/optskills +``` + +## 快速开始 + +使用下面的提示： + +> 阅读 `SKILL.md`，通过 `skill_library/index.json` 选择相关的 OptSkills +> 卡片，为我的问题建模，在可以时求解，并检查关键约束。 + +## 更新 + +只有需要上游更新时才使用下面的提示： + +> Update this standalone OptSkills package from the official upstream. Read `UPDATE.md`, report proposed changes first, and wait for approval before editing. + +## 边界 + +原始卡片可能含有占位符或未经验证的示例。已安装、已加载、已求解和已检查是不同的声明；应分别报告。普通使用仅依赖本包，不调用同仓库的其他技能，也不要求上游的训练、智能体、聊天或嵌入系统。 + +## 许可与来源 + +上游署名见 [SOURCES.md](SOURCES.md)，许可信息见 [LICENSE](LICENSE)。 diff --git a/skills/optskills/SKILL.md b/skills/optskills/SKILL.md new file mode 100644 index 0000000..5a4c0d4 --- /dev/null +++ b/skills/optskills/SKILL.md @@ -0,0 +1,53 @@ +--- +name: optskills +description: Use when a coding agent must model or solve a natural-language operations-research problem by selecting from the released OptSkills problem-archetype cards, or when the user explicitly asks to update this standalone OptSkills library from its official upstream repository. +--- + +# OptSkills + +Use this package independently. Do not call sibling skills in the enclosing +repository and do not require the OptSkills training, agent, chat, or embedding +system. + +## Lightweight version check + +At the start of every normal use, read only the `Upstream snapshot commit` line +from `SOURCES.md` and run +`git ls-remote https://github.com/fujiwaranoM0kou/OptSkills.git HEAD`. Compare +only those two commits; do not fetch or inspect upstream cards. If they match, +continue silently. If they differ or the lookup fails, give at most one short +note and continue with the packaged cards without retrying. Read `UPDATE.md` +and inspect upstream content only when the user asks to update. + +## Normal use + +1. Restate the user's sets, parameters, decision variables, objective, + constraints, units, and assumptions. +2. Read `skill_library/index.json`. Compare the problem structure with each + entry's `name` and `description`, then select one to three candidates. +3. Read only the selected files using their relative `path` fields. Never + invent a path from a skill ID. +4. Choose the closest card by variables, objective, and constraint structure. + If none fits, say that no released card matched and continue with general + mathematical modeling without claiming a card was used. +5. Apply the card to the user's actual data. Card examples are guidance, not + proof that the new model is correct. +6. Use only solver libraries and licenses actually available in the current + environment. Ask before installing dependencies. +7. Check solver status before reading values. Recompute the objective and the + problem's key constraints whenever execution is possible. +8. Report separately: selected card, completed model, solver execution, + constraint checks, and unresolved limitations. + +## Explicit update requests + +Only when the user asks to update or synchronize OptSkills, read `UPDATE.md` +and follow it. Do not update during an ordinary modeling task. + +## Source boundary + +The files under `skill_library/` are released upstream material. Do not rewrite +them as part of normal use. If a card contains a placeholder, missing +dependency, inconsistent notation, or suspected error, correct the active task +explicitly, report the issue, and leave the card unchanged until an upstream +release replaces it. diff --git a/skills/optskills/SOURCES.md b/skills/optskills/SOURCES.md new file mode 100644 index 0000000..b4ee7b3 --- /dev/null +++ b/skills/optskills/SOURCES.md @@ -0,0 +1,31 @@ +# Sources and attribution + +This package provides a thin, standalone entrypoint to released optimization +skill cards from [OptSkills](https://github.com/fujiwaranoM0kou/OptSkills). + +Upstream snapshot commit: d9e14300df4b499529c74ea1981e2c1aba0628b8 + +Paper: *OptSkills: Learning Generalizable Optimization Skills from Problem +Archetypes via Cluster-Based Distillation*, arXiv:2605.29829. + +Paper authors: Haochen Yang, Ke Zhao, Mengyuan Ma, Xingyu Lu, Xiangfeng Wang, +and Hong Qian. + +## Included material + +- all 93 Markdown cards from `skill_library_nanoco_learned`; +- the 10 `skill_library_learned` cards whose `skill_id` is absent from NanoCO; +- the selected entries from the two upstream `index.json` files. + +The Markdown card bodies and selected index fields are kept as published +upstream. VeryMath adds only the standalone entrypoint, combined index, human +documentation, and reviewed update instructions. + +## Excluded material + +The OptSkills training, clustering, self-learning, trajectory, evaluation, +agent, LLM, embedding, dataset, and `ingredients.json` components are not part +of this package and are not runtime dependencies. + +The included upstream material is distributed under the MIT license in +`LICENSE`. Consult the source repository and paper for the original project. diff --git a/skills/optskills/UPDATE.md b/skills/optskills/UPDATE.md new file mode 100644 index 0000000..5c26b61 --- /dev/null +++ b/skills/optskills/UPDATE.md @@ -0,0 +1,53 @@ +# Updating the standalone OptSkills library + +Run this workflow only after the user explicitly asks for an upstream update. + +1. Read `SOURCES.md`, this file, and the current `skill_library/index.json`. +2. Fetch the official `fujiwaranoM0kou/OptSkills` repository. +3. Inspect the current NanoCO, learned, and cluster indexes. If their meaning, + fields, or directory structure changed, stop and explain the change before + applying the old selection rule. +4. Before any edit, preflight each selected upstream index and every file it + names. Within each index, require unique non-empty `skill_id` values and + unique `path` values. Accept a path only when it is a direct-child Markdown + filename under that source library. Reject absolute paths, separators or + nested paths, `.` or `..`, non-`.md` suffixes, missing files, and symlinks. + Never follow an upstream path outside its source library. +5. Form the new selected set in this order: NanoCO first, learned only for IDs + absent from NanoCO, and cluster only for IDs absent from both when cluster is + still published as directly usable cards. Require the selected set itself + to have unique non-empty IDs and unique paths. +6. Complete the comparison before writing. Compare the selected set with the + local package by `skill_id` and ordinary text review. Report every addition, + body change, source switch, path change, full upstream removal, path + collision, and license or attribution change. Never rely only on the index + `version` field. +7. Treat an ID moving from NanoCO to learned or cluster as a source switch, not + a removal. Ask again for every current ID removed from all three upstream + libraries. Resolve every approval before editing; if any removal is + declined, leave the entire package unchanged. +8. Compare every proposed path with all currently occupied local paths, + including paths owned by IDs that will move or disappear. A path occupied + by a different current ID remains a collision until that ID's move or + removal is approved. Stop on any unresolved collision, and never overwrite + a current card while its removal is unresolved. +9. Treat any upstream license, copyright-holder, or attribution change as a + separate stop. Do not import that release until redistribution compatibility + is confirmed and the user explicitly approves it. If approved, update + `LICENSE`, `SOURCES.md`, root `CONTRIBUTORS.md`, and any affected README + claims together. +10. After all comparisons and approvals are complete, copy selected card text + and index fields as published upstream. Follow each accepted upstream path; + when an ID moves, remove its obsolete path. Update every affected card plus + index, source, count wording, and the `SOURCES.md` upstream snapshot commit + in `index.json`, `SOURCES.md`, and any affected package or root README files + in the same reviewable Git change. +11. Validate every index path and run real problems for every affected card. + Report file updates separately from solver execution and checked results. + If final validation fails, do not publish the change; restore the pre-update + state through the repository's normal review/rollback workflow. Do not + create an old-content backup directory. + +Do not add a synchronization script, retain old card copies, rewrite card +content into a local style, modify sibling skill packages, or add integrity, +receipt, manifest, authorization, or execution-unlock mechanisms. diff --git a/skills/optskills/skill_library/2d_bin_packing_with_disjunctive_constraints.md b/skills/optskills/skill_library/2d_bin_packing_with_disjunctive_constraints.md new file mode 100644 index 0000000..9e14218 --- /dev/null +++ b/skills/optskills/skill_library/2d_bin_packing_with_disjunctive_constraints.md @@ -0,0 +1,306 @@ +--- +name: 2D Bin Packing with Disjunctive Constraints +description: | + Models and solves 2D rectangular packing problems with non-overlap, containment, and fixed-orientation constraints, minimizing the number of sheets used via either iterative feasibility or direct MIP optimization. + +--- +# Workflow 1 (Iterative Feasibility with CP-SAT) + +## Modeling stage + +### Strategy Overview +Use a constraint programming approach with Boolean indicator variables for pairwise non-overlap. Solve iteratively by checking feasibility for an increasing number of sheets, leveraging CP-SAT's strength in combinatorial feasibility problems. + +### Step 1 - Define Placement Variables +- For each item `i`, create integer variables `x[i]` and `y[i]` representing the bottom-left corner coordinates. +- Set variable domains from 0 to the sheet dimensions (width, height) to ensure feasibility. + +### Step 2 - Enforce Containment Constraints +- For each item `i` with width `w_i` and height `h_i`, add constraints: `x[i] + w_i <= sheet_width` and `y[i] + h_i <= sheet_height`. +- This ensures the entire item lies within the sheet boundaries. + +### Step 3 - Model Non-Overlap with Boolean Indicators +- For each pair of items `(i, j)` with `i < j`, introduce four Boolean variables: `left`, `right`, `below`, `above`. +- Enforce that at least one Boolean is true: `left + right + below + above >= 1`. +- Use `OnlyEnforceIf` to link each Boolean to the corresponding geometric inequality: + - `left`: `x[i] + w_i <= x[j]` + - `right`: `x[j] + w_j <= x[i]` + - `below`: `y[i] + h_i <= y[j]` + - `above`: `y[j] + h_j <= y[i]` + +### Step 4 - Multi-Sheet Assignment (for minimization) +- Introduce binary assignment variables `z[i][k]` indicating which sheet each item is assigned to. +- Add sheet-level containment and non-overlap constraints per sheet, using `OnlyEnforceIf(z[i][s])` to activate constraints only for the assigned sheet. +- Start with `k = ceil(total_item_area / sheet_area)` as a theoretical lower bound, then iterate over sheet count `k` until a feasible solution is found. + +### Formulation Template +```json +{ + "sets": ["I: items", "K: sheets (iterative)"], + "parameters": ["w_i: item width", "h_i: item height", "W: sheet width", "H: sheet height"], + "decision_variables": [ + "x_i: integer, bottom-left x coordinate", + "y_i: integer, bottom-left y coordinate", + "b_{ij}^1, b_{ij}^2, b_{ij}^3, b_{ij}^4: binary, non-overlap indicators", + "z_{ik}: binary, assignment of item i to sheet k" + ], + "objective": { + "sense": "min", + "expression": "minimize k (iterative feasibility check)" + }, + "constraints": [ + "x_i + w_i <= W, y_i + h_i <= H for all i", + "b_{ij}^1 + b_{ij}^2 + b_{ij}^3 + b_{ij}^4 >= 1 for all i= 1) + model.Add(x[i] + widths[i] <= x[j]).OnlyEnforceIf([left, z[i][s], z[j][s]]) + model.Add(x[j] + widths[j] <= x[i]).OnlyEnforceIf([right, z[i][s], z[j][s]]) + model.Add(y[i] + heights[i] <= y[j]).OnlyEnforceIf([below, z[i][s], z[j][s]]) + model.Add(y[j] + heights[j] <= y[i]).OnlyEnforceIf([above, z[i][s], z[j][s]]) + + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = time_limit + solver.parameters.num_search_workers = 8 + solver.parameters.random_seed = 42 + status = solver.Solve(model) + + if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + positions = [] + for i in range(n): + for s in range(k): + if solver.Value(z[i][s]) == 1: + positions.append({ + "x": solver.Value(x[i]), + "y": solver.Value(y[i]), + "w": widths[i], + "h": heights[i], + "sheet": s + }) + return {"status": "optimal", "sheets_used": k, "positions": positions} + return {"status": "failed", "reason": "infeasible_or_exceeded_max_sheets"} +``` + +### Common Pitfalls +- **Excessive tool calls for validation**: Do not run separate verification scripts after solver confirms feasibility; extract results directly. +- **Ignoring solver status codes**: Always check for `OPTIMAL` or `FEASIBLE` before reading variable values; handle `INFEASIBLE` or `UNKNOWN` gracefully. + +# Workflow 2 (Direct MIP with Pyomo) + +## Modeling stage + +### Strategy Overview +Formulate the packing problem as a Mixed-Integer Program (MIP) using Big-M linearization for disjunctive non-overlap constraints. Minimize the number of sheets directly using binary sheet-usage variables. + +### Step 1 - Define Continuous Placement Variables +- For each item `i`, define continuous variables `x[i]` and `y[i]` representing the bottom-left corner coordinates. +- Use `NonNegativeReals` domain with upper bounds set to sheet dimensions. + +### Step 2 - Enforce Containment Constraints +- Add explicit constraints: `x[i] + width[i] <= sheet_width` and `y[i] + height[i] <= sheet_height`. +- Do not rely solely on variable bounds; use explicit constraints for clarity and solver robustness. + +### Step 3 - Model Non-Overlap with Big-M +- For each pair `(i, j)` with `i < j`, introduce four binary variables: `left[i,j]`, `right[i,j]`, `below[i,j]`, `above[i,j]`. +- Enforce at least one holds: `left + right + below + above >= 1`. +- Use a sufficiently large `M` (e.g., `sheet_width + sheet_height`) to linearize: + - `left`: `x[i] + w_i <= x[j] + M * (1 - left[i,j])` + - `right`: `x[j] + w_j <= x[i] + M * (1 - right[i,j])` + - `below`: `y[i] + h_i <= y[j] + M * (1 - below[i,j])` + - `above`: `y[j] + h_j <= y[i] + M * (1 - above[i,j])` + +### Step 4 - Multi-Sheet Objective +- Introduce binary variable `u[s]` for each sheet `s` indicating if the sheet is used. +- Assign each item to exactly one sheet using binary assignment variables `z[i,s]`. +- Add sheet-specific containment and non-overlap constraints, activated via `z[i,s]`. +- Minimize `sum(u[s])` to find the minimum number of sheets. + +### Formulation Template +```json +{ + "sets": ["I: items", "S: sheets (predefined max)"], + "parameters": ["w_i: item width", "h_i: item height", "W: sheet width", "H: sheet height", "M: large constant"], + "decision_variables": [ + "x_i: continuous, bottom-left x coordinate", + "y_i: continuous, bottom-left y coordinate", + "b_{ij}^1, b_{ij}^2, b_{ij}^3, b_{ij}^4: binary, non-overlap indicators", + "z_{is}: binary, assignment of item i to sheet s", + "u_s: binary, whether sheet s is used" + ], + "objective": { + "sense": "min", + "expression": "sum_s u_s" + }, + "constraints": [ + "x_i + w_i <= W, y_i + h_i <= H for all i", + "b_{ij}^1 + b_{ij}^2 + b_{ij}^3 + b_{ij}^4 >= 1 for all i= 1) + m.left_con = pyo.Constraint(m.pairs, rule=lambda m, i, j: m.x[i] + widths[i] <= m.x[j] + M * (1 - m.left[i, j])) + m.right_con = pyo.Constraint(m.pairs, rule=lambda m, i, j: m.x[j] + widths[j] <= m.x[i] + M * (1 - m.right[i, j])) + m.below_con = pyo.Constraint(m.pairs, rule=lambda m, i, j: m.y[i] + heights[i] <= m.y[j] + M * (1 - m.below[i, j])) + m.above_con = pyo.Constraint(m.pairs, rule=lambda m, i, j: m.y[j] + heights[j] <= m.y[i] + M * (1 - m.above[i, j])) + + # Assignment and sheet usage + m.assign = pyo.Constraint(m.I, rule=lambda m, i: sum(m.z[i, s] for s in m.S) == 1) + m.sheet_use = pyo.Constraint(m.I, m.S, rule=lambda m, i, s: m.z[i, s] <= m.u[s]) + + # Objective + m.obj = pyo.Objective(expr=sum(m.u[s] for s in m.S), sense=pyo.minimize) + + solver = pyo.SolverFactory("highs") + solver.options["time_limit"] = time_limit + solver.options["mip_rel_gap"] = 0.01 + results = solver.solve(m, tee=False) + + if results.solver.status == SolverStatus.ok and results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}: + positions = [] + for i in items: + for s in sheets: + if pyo.value(m.z[i, s]) > 0.5: + positions.append({ + "x": pyo.value(m.x[i]), + "y": pyo.value(m.y[i]), + "w": widths[i], + "h": heights[i], + "sheet": s + }) + return {"status": "optimal", "objective": pyo.value(m.obj), "positions": positions} + else: + return {"status": "failed", "reason": "infeasible_or_error"} +``` + +### Common Pitfalls +- **Trusting non-zero return codes**: Always check solver status and termination condition explicitly; do not assume success from non-zero exit codes. +- **Outputting pseudo-numeric answers on failure**: When execution fails, output a clear failure message with reason; do not fabricate numeric results. diff --git a/skills/optskills/skill_library/assignment_problem_solver.md b/skills/optskills/skill_library/assignment_problem_solver.md new file mode 100644 index 0000000..156e117 --- /dev/null +++ b/skills/optskills/skill_library/assignment_problem_solver.md @@ -0,0 +1,249 @@ +--- +name: Assignment Problem Solver +description: | + Models and solves one-to-one assignment problems with binary decision variables, covering both MIP and CP-SAT solver backends. +--- + +# Workflow 1 (MIP with OR-Tools) + +## Modeling stage + +### Strategy Overview +Formulate the assignment problem as a mixed-integer linear program using binary variables, linear constraints for one-to-one matching, and a linear objective minimizing total cost. + +### Step 1 - Define Binary Assignment Variables +- Create a binary integer variable `x[i, j]` for each pair `(i, j)` where `i` indexes the first set (e.g., workers) and `j` indexes the second set (e.g., tasks). +- Use `solver.IntVar(0, 1, f"x_{i}_{j}")` to enforce the binary domain. + +### Step 2 - Enforce One-to-One Matching Constraints +- For each element `i` in the first set, add constraint: `sum_j x[i, j] == 1` to ensure exactly one assignment per entity. +- For each element `j` in the second set, add constraint: `sum_i x[i, j] == 1` to ensure exactly one assignment per entity. + +### Step 3 - Minimize Total Cost Objective +- Build the objective as `minimize sum_i sum_j cost[i][j] * x[i, j]`. +- Use `objective.SetCoefficient(x[i, j], cost[i][j])` for each variable, then call `objective.SetMinimization()`. + +### Formulation Template +```json +{ + "sets": ["I: first set entities", "J: second set entities"], + "parameters": ["c[i, j]: cost of assigning i to j"], + "decision_variables": ["x[i, j] ∈ {0, 1} for all i in I, j in J"], + "objective": { + "sense": "min", + "expression": "sum_{i in I} sum_{j in J} c[i, j] * x[i, j]" + }, + "constraints": [ + "sum_{j in J} x[i, j] == 1 for all i in I", + "sum_{i in I} x[i, j] == 1 for all j in J" + ] +} +``` + +### Common Pitfalls +- Forgetting to handle unbalanced instances: if `|I| != |J|`, add dummy entities with zero-cost assignments to create a square matrix. +- Using continuous variables instead of binary, which can lead to fractional assignments. + +## Solving stage + +### Strategy Overview +Solve the MIP formulation using OR-Tools' SCIP solver with time limits and parallel processing, then extract and validate the solution. + +### Step 1 - Initialize Solver and Configure Parameters +- Create solver instance with `pywraplp.Solver.CreateSolver("SCIP")`. +- **Check solver availability:** Verify the returned instance is not `None`. +- Set time limit: `solver.SetTimeLimit([TIME_LIMIT_MS])` (e.g., 30000 for 30 seconds). +- Enable parallelism: `solver.SetNumThreads([NUM_THREADS])` (e.g., 4). + +### Step 2 - Build and Solve Model +- Populate the variable dictionary `x = {}` with `solver.IntVar(0, 1, name)` for all `(i, j)` pairs. +- Add constraints using `solver.Add(sum(x[i, j] for j in range(num_tasks)) == 1)` for each worker, and similarly for tasks. +- Build objective with `objective.SetCoefficient()` and call `solver.Solve()`. + +### Step 3 - Extract and Validate Results +- Check solver status: `if status in [pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE]`. +- Extract assignments where `x[i, j].solution_value() > 0.5`. +- **Validate solution feasibility:** Verify the extracted assignment satisfies all one-to-one constraints. +- **Verify objective:** Manually sum costs of assigned pairs and compare to `solver.Objective().Value()` as a sanity check. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +def solve_assignment_mip(cost_matrix): + num_workers = len(cost_matrix) + num_tasks = len(cost_matrix[0]) + + solver = pywraplp.Solver.CreateSolver("SCIP") + if not solver: + return {"status": "failed", "error": "Solver creation failed"} + + solver.SetTimeLimit(30000) + solver.SetNumThreads(4) + + # Variables + x = {} + for i in range(num_workers): + for j in range(num_tasks): + x[i, j] = solver.IntVar(0, 1, f"x_{i}_{j}") + + # Constraints + for i in range(num_workers): + solver.Add(sum(x[i, j] for j in range(num_tasks)) == 1) + for j in range(num_tasks): + solver.Add(sum(x[i, j] for i in range(num_workers)) == 1) + + # Objective + objective = solver.Objective() + for i in range(num_workers): + for j in range(num_tasks): + objective.SetCoefficient(x[i, j], cost_matrix[i][j]) + objective.SetMinimization() + + status = solver.Solve() + + if status in (pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE): + assignments = [] + total_cost = 0 + for i in range(num_workers): + for j in range(num_tasks): + if x[i, j].solution_value() > 0.5: + assignments.append({"worker": i, "task": j}) + total_cost += cost_matrix[i][j] + # Sanity check: verify objective consistency + if abs(total_cost - solver.Objective().Value()) > 1e-6: + return {"status": "error", "error": "Objective mismatch"} + return { + "status": "optimal" if status == pywraplp.Solver.OPTIMAL else "feasible", + "objective": total_cost, + "assignments": assignments + } + else: + return {"status": "failed", "solver_status": status} +``` + +### Common Pitfalls +- Not checking if `CreateSolver` returns `None`, which indicates the solver backend is unavailable. +- Assuming `OPTIMAL` status when the time limit may cause early termination with `FEASIBLE`. + +# Workflow 2 (CP-SAT with OR-Tools) + +## Modeling stage + +### Strategy Overview +Model the assignment problem using constraint programming with Boolean variables, leveraging CP-SAT's efficient propagation for one-to-one matching constraints. + +### Step 1 - Define Boolean Assignment Variables +- Create a Boolean variable `x[i, j]` for each pair using `model.NewBoolVar(f"x_{i}_{j}")`. +- These variables implicitly enforce binary domain without explicit integer constraints. + +### Step 2 - Enforce One-to-One Matching Constraints +- For each entity `i` in the first set: `model.Add(sum(x[i, j] for j in range(num_tasks)) == 1)`. +- For each entity `j` in the second set: `model.Add(sum(x[i, j] for i in range(num_workers)) == 1)`. + +### Step 3 - Minimize Total Cost Objective +- Build a list of cost-weighted terms: `[cost_matrix[i][j] * x[i, j] for i in range(num_workers) for j in range(num_tasks)]`. +- Set objective: `model.Minimize(sum(objective_terms))`. +- **Note:** CP-SAT requires all objective coefficients to be integers; scale floating-point costs by a common factor if needed. + +### Formulation Template +```json +{ + "sets": ["I: first set entities", "J: second set entities"], + "parameters": ["c[i, j]: cost of assigning i to j"], + "decision_variables": ["x[i, j] ∈ {True, False} for all i in I, j in J"], + "objective": { + "sense": "min", + "expression": "sum_{i in I} sum_{j in J} c[i, j] * x[i, j]" + }, + "constraints": [ + "sum_{j in J} x[i, j] == 1 for all i in I", + "sum_{i in I} x[i, j] == 1 for all j in J" + ] +} +``` + +### Common Pitfalls +- Using `model.AddBoolOr()` or other logical constraints instead of linear sums, which can be less efficient for assignment problems. +- Forgetting that CP-SAT requires all coefficients to be integers; scale floating-point costs by a common factor if needed. + +## Solving stage + +### Strategy Overview +Solve using OR-Tools CP-SAT solver with parallel search and optimality guarantees, then extract assignments with proper status checking. + +### Step 1 - Configure CP-SAT Solver +- Instantiate `cp_model.CpSolver()`. +- Set parameters: `solver.parameters.max_time_in_seconds = [TIME_LIMIT]` (e.g., 30), `solver.parameters.num_search_workers = [NUM_WORKERS]` (e.g., 8), `solver.parameters.random_seed = [SEED]` (e.g., 42), `solver.parameters.relative_gap_limit = 0.0`. + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve(model)`. +- Check `if status in (cp_model.OPTIMAL, cp_model.FEASIBLE)` before extracting values. + +### Step 3 - Extract and Validate Solution +- Iterate over all `(i, j)` pairs and collect assignments where `solver.Value(x[i, j]) == 1`. +- **Validate solution feasibility:** Ensure the extracted assignment satisfies all one-to-one constraints. +- **Verify objective:** Manually sum costs of assigned pairs and compare to `solver.ObjectiveValue()` as a sanity check. +- Return structured output with status, objective value, and assignment list. + +### Code Usage +```python +from ortools.sat.python import cp_model + +def solve_assignment_cpsat(cost_matrix): + num_workers = len(cost_matrix) + num_tasks = len(cost_matrix[0]) + + model = cp_model.CpModel() + + # Variables + x = {} + for i in range(num_workers): + for j in range(num_tasks): + x[i, j] = model.NewBoolVar(f"x_{i}_{j}") + + # Constraints + for i in range(num_workers): + model.Add(sum(x[i, j] for j in range(num_tasks)) == 1) + for j in range(num_tasks): + model.Add(sum(x[i, j] for i in range(num_workers)) == 1) + + # Objective + objective_terms = [] + for i in range(num_workers): + for j in range(num_tasks): + objective_terms.append(cost_matrix[i][j] * x[i, j]) + model.Minimize(sum(objective_terms)) + + # Solve + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = 30 + solver.parameters.num_search_workers = 8 + solver.parameters.random_seed = 42 + solver.parameters.relative_gap_limit = 0.0 + + status = solver.Solve(model) + + if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + assignments = [] + total_cost = 0 + for i in range(num_workers): + for j in range(num_tasks): + if solver.Value(x[i, j]) == 1: + assignments.append({"worker": i, "task": j}) + total_cost += cost_matrix[i][j] + # Sanity check: verify objective consistency + if abs(total_cost - solver.ObjectiveValue()) > 1e-6: + return {"status": "error", "error": "Objective mismatch"} + return { + "status": "optimal" if status == cp_model.OPTIMAL else "feasible", + "objective": total_cost, + "assignments": assignments + } + else: + return {"status": "failed", "solver_status": status} +``` + +### Common Pitfalls +- Not setting `relative_gap_limit = 0.0` when proven optimality is required; otherwise CP-SAT may stop early with a feasible solution. +- Using `solver.Value()` without first checking the status, which can raise exceptions on infeasible models. diff --git a/skills/optskills/skill_library/assignment_problem_with_capacity_and_demand.md b/skills/optskills/skill_library/assignment_problem_with_capacity_and_demand.md new file mode 100644 index 0000000..adab131 --- /dev/null +++ b/skills/optskills/skill_library/assignment_problem_with_capacity_and_demand.md @@ -0,0 +1,249 @@ +--- +name: Assignment Problem with Capacity and Demand +description: | + Model and solve linear assignment problems with resource availability, task demand, and per-assignment capacity limits using continuous variables and linear programming solvers. +--- + +# Workflow 1 (Direct Solver API - OR-Tools) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a linear program using a direct solver API (e.g., OR-Tools). Define continuous assignment variables with built-in bounds to represent capacity limits. Structure data as indexed lists or dictionaries for clarity and efficient model construction. + +### Step 1 - Define Data Structures +- Organize problem parameters into clear, indexed data structures. +- Create lists for resource availability and task demand. +- Create a 2D structure (list-of-lists or dictionary) for assignment costs and capacity limits. + +### Step 2 - Instantiate Model and Variables +- Create a solver instance for linear programming (e.g., `GLOP`). +- Define continuous decision variables `x[i][j]` for assignment quantity from resource `i` to task `j`. +- Set variable bounds directly: lower bound `0`, upper bound `capacity[i][j]`. + +### Step 3 - Build Demand Satisfaction Constraints +- For each task `j`, create an equality constraint. +- Set the constraint's right-hand side to `demand[j]`. +- Add the coefficient `1` for each variable `x[i][j]` across all resources `i`. + +### Step 4 - Build Resource Availability Constraints +- For each resource `i`, create a less-than-or-equal constraint. +- Set the constraint's upper bound to `availability[i]`. +- Add the coefficient `1` for each variable `x[i][j]` across all tasks `j`. + +### Step 5 - Define the Objective Function +- Create a linear objective to minimize total cost. +- For each `(i, j)` pair, add the term `cost[i][j] * x[i][j]`. + +### Formulation Template +```json +{ + "sets": [ + "resources", + "tasks" + ], + "parameters": [ + "availability[resource]", + "demand[task]", + "cost[resource][task]", + "capacity[resource][task]" + ], + "decision_variables": [ + "x[resource][task] >= 0" + ], + "objective": { + "sense": "min", + "expression": "sum_over_resources_tasks(cost[i][j] * x[i][j])" + }, + "constraints": [ + "demand_satisfaction[task]: sum_over_resources(x[i][j]) == demand[j]", + "resource_availability[resource]: sum_over_tasks(x[i][j]) <= availability[i]", + "capacity_limit[resource][task]: x[i][j] <= capacity[i][j] (enforced via variable bounds)" + ] +} +``` + +### Common Pitfalls +- Forgetting to handle resources with zero availability, which can lead to unnecessary variables; consider explicitly fixing their assignments to zero. +- Using inconsistent indexing between parameters and variables, causing `KeyError` or incorrect model construction. +- Not setting variable upper bounds, requiring separate capacity constraints and increasing model size. + +## Solving stage + +### Strategy Overview +Solve the constructed LP model using the chosen solver. Implement robust solution status checking and post-solve verification of all constraints. Extract and report the solution in a clear, actionable format. + +### Step 1 - Solve and Check Status +- Call the solver's `Solve()` method. +- Check the result status (e.g., `OPTIMAL`, `FEASIBLE`). +- If not optimal or feasible, log an informative error and analyze infeasibility. + +### Step 2 - Extract and Store Solution +- If the status is acceptable, iterate over all variables to extract their solution values. +- Store assignments in a data structure mirroring the variable indexing. + +### Step 3 - Verify Solution Against Constraints +- Recompute totals per task and compare to demand within a small tolerance. +- Recompute totals per resource and compare to availability within a tolerance. +- Check each non-zero assignment against its capacity limit. +- Recompute the objective value from assignments and compare to the solver's reported value. + +### Step 4 - Report Results +- Print a summary of the total cost and solver status. +- Print a detailed report of non-zero assignments. +- Print per-resource and per-task utilization summaries. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('GLOP') +x = {} +for i in resources: + for j in tasks: + x[i, j] = solver.NumVar(0, capacity[i][j], f'x_{i}_{j}') + +# Demand constraints +for j in tasks: + ct = solver.Constraint(demand[j], demand[j]) + for i in resources: + ct.SetCoefficient(x[i, j], 1) + +# Availability constraints +for i in resources: + ct = solver.Constraint(0, availability[i]) + for j in tasks: + ct.SetCoefficient(x[i, j], 1) + +# Objective +objective = solver.Objective() +for i in resources: + for j in tasks: + objective.SetCoefficient(x[i, j], cost[i][j]) +objective.SetMinimization() + +# solve with status / termination checks +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + # Extract solution... +else: + print(f'Solver did not find optimal solution. Status: {status}') +``` + +### Common Pitfalls +- Extracting solution values without first checking the solver status, leading to errors. +- Using an absolute tolerance of zero for floating-point comparisons in verification, causing false failures. +- Not verifying the solution independently, potentially missing solver or model-building errors. + +# Workflow 2 (Modeling Framework - Pyomo) + +## Modeling stage + +### Strategy Overview +Formulate the problem using a modeling framework (e.g., Pyomo) to separate the model declaration from the solver interface. Define abstract sets and parameters for flexibility, and use constraint rules for clean, maintainable model construction. + +### Step 1 - Declare Abstract Sets and Parameters +- Define the abstract sets `resources` and `tasks`. +- Declare `Param` components for `availability`, `demand`, `cost`, and `capacity`, indexed appropriately. + +### Step 2 - Define Continuous Variables +- Declare a `Var` component `x` indexed over `resources` and `tasks`. +- Set the variable domain to `NonNegativeReals`. +- Optionally, set variable upper bounds using the `capacity` parameter within a rule or initialization. + +### Step 3 - Define Objective Rule +- Create a `Objective` component with `sense=minimize`. +- Define a rule that returns the sum of `cost[i,j] * x[i,j]` over all indices. + +### Step 4 - Define Constraint Rules +- Create a `Constraint` component for demand satisfaction, indexed by `tasks`. +- The rule for each task `j` returns the sum of `x[i,j]` over `resources` equals `demand[j]`. +- Create a `Constraint` component for resource availability, indexed by `resources`. +- The rule for each resource `i` returns the sum of `x[i,j]` over `tasks` less than or equal to `availability[i]`. + +### Step 5 - Handle Edge Cases Explicitly +- Add a preprocessing step or additional constraints to fix variables to zero for resources with zero availability, improving model clarity and presolve. + +### Formulation Template +```json +{ + "sets": [ + "resources", + "tasks" + ], + "parameters": [ + "availability[resource]", + "demand[task]", + "cost[resource][task]", + "capacity[resource][task]" + ], + "decision_variables": [ + "x[resource][task] in NonNegativeReals" + ], + "objective": { + "sense": "min", + "expression": "sum(cost[i,j] * x[i,j] for i in resources for j in tasks)" + }, + "constraints": [ + "demand_satisfaction[task]: sum(x[i,j] for i in resources) == demand[j]", + "resource_availability[resource]: sum(x[i,j] for j in tasks) <= availability[i]", + "capacity_limit[resource][task]: x[i,j] <= capacity[i,j] (optional, can be variable bound)" + ] +} +``` + +### Common Pitfalls +- Defining variable bounds within a constraint rule instead of the variable declaration, which can be less efficient for the solver. +- Using concrete data initialization with abstract sets, causing runtime errors. +- Not attaching parameter dictionaries to the model object, making them inaccessible during post-solve verification. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a compatible LP solver (e.g., HiGHS, CBC). Use a pattern that separates solving from solution loading to handle statuses robustly. Perform comprehensive numerical verification of the solution. + +### Step 1 - Instantiate Solver and Set Options +- Create a solver object using `SolverFactory`. +- Set essential options like `time_limit` and `threads`. Avoid over-specifying options. + +### Step 2 - Solve with Careful Solution Loading +- Execute `solve(model, load_solutions=False)` to get results without immediate loading. +- Check the solver termination condition (e.g., `optimal`, `feasible`). +- If termination is acceptable, manually load the solution into the model. + +### Step 3 - Verify All Constraints +- Iterate through all constraints, evaluating their expressions and comparing to bounds with a tolerance. +- Compute per-task and per-resource totals from the variable values to verify demand and availability. +- Check variable values against their upper bounds (capacity). + +### Step 4 - Compute and Report Metrics +- Recalculate the objective value from the loaded variable values. +- Generate a structured report showing assignment details, constraint slack, and total cost. +- Summarize resource utilization and task fulfillment. + +### Code Usage +```python +# build model from formulation +model = pyo.ConcreteModel() +model.resources = pyo.Set(initialize=resources) +model.tasks = pyo.Set(initialize=tasks) +# ... define parameters, variables, objective, and constraints ... + +# solve with status / termination checks +solver = pyo.SolverFactory('appsi_highs') +results = solver.solve(model, load_solutions=False) + +if results.solver.termination_condition == pyo.TerminationCondition.optimal: + model.solutions.load_from(results) + # Verification and reporting... +elif results.solver.termination_condition == pyo.TerminationCondition.feasible: + print('Feasible solution found, may not be optimal.') + model.solutions.load_from(results) + # Verification and reporting... +else: + print(f'Solve failed: {results.solver.termination_condition}') +``` + +### Common Pitfalls +- Loading solutions automatically without checking termination condition, which can load invalid results. +- Using a solver interface that requires specific option syntax; stick to documented, minimal options. +- Neglecting to use a tolerance (e.g., `1e-6`) when verifying equality constraints due to numerical precision. diff --git a/skills/optskills/skill_library/assignment_problem_with_surplus_resources.md b/skills/optskills/skill_library/assignment_problem_with_surplus_resources.md new file mode 100644 index 0000000..8f3c46b --- /dev/null +++ b/skills/optskills/skill_library/assignment_problem_with_surplus_resources.md @@ -0,0 +1,287 @@ +--- +name: Assignment Problem with Surplus Resources +description: | + Model and solve rectangular assignment problems (more agents than tasks) with binary assignment variables, cardinality limits, and a linear cost minimization objective using either a direct MIP formulation or a network flow transformation. +--- + +# Workflow 1 (Direct MIP Formulation) + +## Modeling stage + +### Strategy Overview +Model the problem directly as a Mixed-Integer Program (MIP) using binary decision variables for each potential assignment. This approach is intuitive, flexible, and directly encodes the cardinality constraints on both sets. + +### Step 1 - Define Sets and Parameters +- Define the set of agents `A` (e.g., workers, machines) and the set of tasks `T` (e.g., jobs, documents). The cardinality of `A` is greater than that of `T`. +- Define a cost parameter `c_{a,t}` representing the cost of assigning agent `a` to task `t`. Store this as a 2D array or dictionary. + +### Step 2 - Create Decision Variables +- Create a binary decision variable `x_{a,t}` for each agent `a` in `A` and each task `t` in `T`. +- `x_{a,t} = 1` indicates agent `a` is assigned to task `t`; `0` otherwise. + +### Step 3 - Formulate Task Coverage Constraints +- For each task `t` in `T`, add a constraint `∑_{a ∈ A} x_{a,t} = 1`. This ensures every task is assigned to exactly one agent. + +### Step 4 - Formulate Agent Capacity Constraints +- For each agent `a` in `A`, add a constraint `∑_{t ∈ T} x_{a,t} ≤ 1`. This enforces that each agent can be assigned to at most one task. + +### Step 5 - Define Objective Function +- Formulate the objective to minimize total assignment cost: `min ∑_{a ∈ A} ∑_{t ∈ T} c_{a,t} * x_{a,t}`. + +### Formulation Template +```json +{ + "sets": [ + "A: Set of agents (size n, where n > m).", + "T: Set of tasks (size m)." + ], + "parameters": [ + "c_{a,t}: Cost of assigning agent a to task t." + ], + "decision_variables": [ + "x_{a,t} ∈ {0, 1}: Binary assignment variable." + ], + "objective": { + "sense": "min", + "expression": "∑_{a ∈ A} ∑_{t ∈ T} c_{a,t} * x_{a,t}" + }, + "constraints": [ + "Task Coverage: ∑_{a ∈ A} x_{a,t} = 1, ∀ t ∈ T", + "Agent Capacity: ∑_{t ∈ T} x_{a,t} ≤ 1, ∀ a ∈ A" + ] +} +``` + +### Common Pitfalls +- Forgetting that the agent set must be larger than the task set for a valid rectangular assignment; otherwise, the `Agent Capacity` constraints may make the model infeasible. +- Using `== 1` for the agent capacity constraint, which would force all agents to be assigned, incorrectly modeling a perfect matching instead of a surplus resource scenario. + +## Solving stage + +### Strategy Overview +Solve the MIP model using a dedicated integer programming solver via a modeling library (e.g., OR-Tools, PuLP). Configure solver settings for performance and implement robust solution status checking and result extraction. + +### Step 1 - Initialize Solver and Model +- Instantiate a MIP solver (e.g., `SCIP`, `CBC` via OR-Tools' `pywraplp`). +- Optionally set performance parameters: time limit, relative optimality gap, and number of threads. + +### Step 2 - Instantiate Variables and Add Constraints +- Create binary variables using loops over sets `A` and `T`. +- Add the task coverage and agent capacity constraints using summation over the created variables. + +### Step 3 - Set Objective and Solve +- Build the linear objective function by setting coefficients for each variable. +- Call the solver's `Solve()` method. + +### Step 4 - Check Status and Extract Solution +- Check the solver status (`OPTIMAL`, `FEASIBLE`). Handle non-optimal statuses (e.g., `INFEASIBLE`, `UNBOUNDED`) with appropriate error messages. +- If optimal or feasible, extract the objective value. +- Iterate over all variables, collecting assignments where the solution value is greater than a threshold (e.g., 0.5). + +### Code Usage +```python +# Example using OR-Tools +from ortools.linear_solver import pywraplp + +# Data placeholders +agents = list(range(num_agents)) # n +tasks = list(range(num_tasks)) # m (n > m) +cost = {(a, t): cost_value for a in agents for t in tasks} # Define your cost matrix + +# 1. Initialize Solver +solver = pywraplp.Solver.CreateSolver("SCIP") +if not solver: + raise Exception("Solver not available.") + +# Optional: Set solver parameters +solver.SetTimeLimit(30000) # milliseconds +solver.SetNumThreads(4) + +# 2. Create Variables +x = {} +for a in agents: + for t in tasks: + x[a, t] = solver.IntVar(0, 1, f"x_{a}_{t}") + +# 3. Add Constraints +# Task coverage: each task assigned to exactly one agent +for t in tasks: + solver.Add(solver.Sum([x[a, t] for a in agents]) == 1) +# Agent capacity: each agent assigned to at most one task +for a in agents: + solver.Add(solver.Sum([x[a, t] for t in tasks]) <= 1) + +# 4. Set Objective +objective = solver.Objective() +for a in agents: + for t in tasks: + objective.SetCoefficient(x[a, t], cost[a, t]) +objective.SetMinimization() + +# 5. Solve +status = solver.Solve() + +# 6. Check Status and Extract Solution +if status in (solver.OPTIMAL, solver.FEASIBLE): + total_cost = objective.Value() + assignments = [] + for a in agents: + for t in tasks: + if x[a, t].solution_value() > 0.5: + assignments.append((a, t)) + print(f"Total cost: {total_cost}") + print(f"Assignments: {assignments}") +else: + print(f"Solver did not find a solution. Status: {status}") +``` + +### Common Pitfalls +- Not checking solver status before accessing solution values, which can cause runtime errors. +- Using a loose optimality gap (`mip_rel_gap`) when an exact optimal solution is required; set it to `0.0` for exact MIP solving. + +# Workflow 2 (Network Flow Transformation) + +## Modeling stage + +### Strategy Overview +Reformulate the assignment problem as a minimum-cost flow problem on a bipartite network. This leverages specialized, often more efficient, network flow algorithms and provides an alternative solving pathway. + +### Step 1 - Define Network Structure +- Model the problem as a directed graph with nodes: a source (`S`), agent nodes (`A`), task nodes (`T`), and a sink (`K`). +- Define arcs: `S → A` (capacity 1, cost 0), `A → T` (capacity 1, cost = `c_{a,t}`), `T → K` (capacity 1, cost 0). + +### Step 2 - Define Supplies and Demands +- Set the supply at the source node equal to the number of tasks `|T|` (or `m`). +- Set the demand at the sink node equal to `-|T|`. +- Set the supply/demand for all other nodes (agents and tasks) to 0. + +### Step 3 - Map Flow to Assignment +- The flow on arc `(a, t)` from the agent layer to the task layer corresponds to the binary assignment variable `x_{a,t}`. A flow of 1 indicates assignment. + +### Formulation Template +```json +{ + "sets": [ + "A: Set of agent nodes.", + "T: Set of task nodes." + ], + "parameters": [ + "c_{a,t}: Cost per unit flow on arc from agent a to task t." + ], + "decision_variables": [ + "f_{a,t}: Flow on arc from agent a to task t (integer, 0 or 1)." + ], + "objective": { + "sense": "min", + "expression": "∑_{a ∈ A} ∑_{t ∈ T} c_{a,t} * f_{a,t}" + }, + "constraints": [ + "Flow Conservation at Agent a: f_{S→a} = ∑_{t ∈ T} f_{a,t}, ∀ a ∈ A", + "Flow Conservation at Task t: ∑_{a ∈ A} f_{a,t} = f_{t→K}, ∀ t ∈ T", + "Source Supply: ∑_{a ∈ A} f_{S→a} = |T|", + "Sink Demand: ∑_{t ∈ T} f_{t→K} = |T|", + "Arc Capacities: 0 ≤ f_{a,t} ≤ 1, 0 ≤ f_{S→a} ≤ 1, 0 ≤ f_{t→K} ≤ 1" + ] +} +``` + +### Common Pitfalls +- Incorrectly setting node supplies/demands, which can lead to infeasibility. The total supply must equal total demand. +- Forgetting that the `S→A` and `T→K` arcs also have capacity 1, which enforces the agent and task cardinality limits. + +## Solving stage + +### Strategy Overview +Use a dedicated minimum-cost flow solver (e.g., OR-Tools `SimpleMinCostFlow`) to find the optimal flow. Construct the network by adding arcs with capacities and costs, set node supplies, and solve. + +### Step 1 - Initialize Flow Solver and Graph +- Create an instance of a min-cost flow solver. +- Prepare data structures to map your original agent/task indices to the solver's internal node indices. + +### Step 2 - Add Arcs and Set Parameters +- Add arcs in three layers: source-to-agents, agents-to-tasks, and tasks-to-sink. +- For each arc, specify its start node, end node, capacity, and unit cost. +- Set the supply for each node (positive for source, negative for sink, zero for others). + +### Step 3 - Solve and Check Optimality +- Invoke the solver's `Solve()` or equivalent method. +- Verify the solver status is `OPTIMAL`. Handle other statuses (e.g., `INFEASIBLE`) appropriately. + +### Step 4 - Extract Assignments from Flow +- Iterate over all arcs. Identify arcs that carry positive flow (flow > 0.5) and connect an agent node to a task node. +- Map these arcs back to the original agent and task indices to obtain the assignment pairs. +- Calculate the total cost from the solver's objective or by summing `cost * flow` for the assignment arcs. + +### Code Usage +```python +# Example using OR-Tools MinCostFlow +from ortools.graph.python import min_cost_flow + +# Data placeholders +agents = list(range(num_agents)) +tasks = list(range(num_tasks)) +cost = {(a, t): cost_value for a in agents for t in tasks} + +# 1. Initialize Solver +smcf = min_cost_flow.SimpleMinCostFlow() + +# Define node indices +source = 0 +sink = 1 +agent_nodes = {a: 2 + a for a in agents} # e.g., 2, 3, ... +task_nodes = {t: 2 + len(agents) + t for t in tasks} # e.g., 2+n, 2+n+1, ... + +# 2. Add Arcs and Set Supplies +# Arcs from source to each agent (capacity 1, cost 0) +for a in agents: + smcf.add_arc_with_capacity_and_unit_cost( + source, agent_nodes[a], 1, 0 + ) + +# Arcs from each agent to each task (capacity 1, cost = c[a,t]) +for a in agents: + for t in tasks: + smcf.add_arc_with_capacity_and_unit_cost( + agent_nodes[a], task_nodes[t], 1, cost[a, t] + ) + +# Arcs from each task to sink (capacity 1, cost 0) +for t in tasks: + smcf.add_arc_with_capacity_and_unit_cost( + task_nodes[t], sink, 1, 0 + ) + +# Set node supplies +smcf.set_node_supply(source, len(tasks)) # Supply = number of tasks +smcf.set_node_supply(sink, -len(tasks)) # Demand = - (number of tasks) +for a in agents: + smcf.set_node_supply(agent_nodes[a], 0) +for t in tasks: + smcf.set_node_supply(task_nodes[t], 0) + +# 3. Solve +status = smcf.solve() + +# 4. Check Status and Extract Solution +if status == smcf.OPTIMAL: + total_cost = smcf.optimal_cost() + assignments = [] + for arc in range(smcf.num_arcs()): + if smcf.flow(arc) > 0: + tail = smcf.tail(arc) + head = smcf.head(arc) + # Check if arc connects an agent node to a task node + if tail in agent_nodes.values() and head in task_nodes.values(): + # Map node indices back to original IDs + a = [k for k, v in agent_nodes.items() if v == tail][0] + t = [k for k, v in task_nodes.items() if v == head][0] + assignments.append((a, t)) + print(f"Total cost: {total_cost}") + print(f"Assignments: {assignments}") +else: + print(f"Solver returned non-optimal status: {status}") +``` + +### Common Pitfalls +- Mismatching node indices when adding arcs or setting supplies, leading to an incorrect network topology. +- Assuming all arcs with positive flow are assignment arcs; must filter for arcs connecting the agent and task layers only, ignoring source-agent and task-sink arcs. diff --git a/skills/optskills/skill_library/assignmentflowwithparticipationrequirements.md b/skills/optskills/skill_library/assignmentflowwithparticipationrequirements.md new file mode 100644 index 0000000..713222e --- /dev/null +++ b/skills/optskills/skill_library/assignmentflowwithparticipationrequirements.md @@ -0,0 +1,293 @@ +--- +name: AssignmentFlowWithParticipationRequirements +description: | + Model and solve mixed-integer linear programs for assignment-flow problems with minimum contributor counts and conditional minimum delivery amounts, using either direct solver APIs or algebraic modeling frameworks. + +--- +# Workflow 1 (Direct Solver API - OR-Tools) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) using a direct solver API. This approach involves manually constructing variables, constraints, and the objective using the solver's native objects, offering fine-grained control and immediate integration with the solver's features. + +### Step 1 - Define Data Structures +- Organize problem data into Python dictionaries or lists for easy indexing. +- Create sets for producers and contracts, and define parameters for capacity, demand, cost, minimum delivery amounts, and minimum contributor counts. + +### Step 2 - Create Decision Variables +- Instantiate continuous flow variables `x[p][c]` for the amount allocated from producer `p` to contract `c`. +- Instantiate binary assignment variables `y[p][c]` to indicate if producer `p` is assigned to contract `c`. + +### Step 3 - Formulate Capacity and Demand Constraints +- Add linear inequality constraints to ensure total flow from each producer does not exceed its capacity. +- Add linear inequality constraints to ensure total flow to each contract meets or exceeds its demand. + +### Step 4 - Enforce Minimum Contributor Counts +- For each contract, add a constraint that sums the binary assignment variables to be at least the required minimum number of contributors. + +### Step 5 - Link Assignment to Flow with Big-M Constraints +- Add a lower-bound linking constraint: `x[p][c] >= min_delivery[p] * y[p][c]`. This enforces a minimum contribution amount when a producer is active. +- Add an upper-bound linking constraint: `x[p][c] <= capacity[p] * y[p][c]`. This forces the flow to zero when the producer is inactive and provides a tight upper bound. + +### Step 6 - Define Linear Cost Objective +- Formulate the objective to minimize the total linear cost, summing `cost[p][c] * x[p][c]` over all producer-contract pairs. + +### Formulation Template +```json +{ + "sets": [ + "producers", + "contracts" + ], + "parameters": [ + "capacity[producers]", + "demand[contracts]", + "cost[producers][contracts]", + "min_delivery[producers]", + "min_contributors[contracts]" + ], + "decision_variables": [ + "x[producers][contracts] (continuous, >=0)", + "y[producers][contracts] (binary)" + ], + "objective": { + "sense": "min", + "expression": "sum(cost[p][c] * x[p][c] for p in producers for c in contracts)" + }, + "constraints": [ + "capacity_limit[p]: sum(x[p][c] for c in contracts) <= capacity[p]", + "demand_satisfaction[c]: sum(x[p][c] for p in producers) >= demand[c]", + "min_contributor_count[c]: sum(y[p][c] for p in producers) >= min_contributors[c]", + "min_delivery_link[p][c]: x[p][c] >= min_delivery[p] * y[p][c]", + "max_delivery_link[p][c]: x[p][c] <= capacity[p] * y[p][c]" + ] +} +``` + +### Common Pitfalls +- Using an arbitrarily large `M` value in the upper-bound linking constraint, which weakens the linear relaxation and harms solver performance. Use the smallest valid bound (e.g., producer capacity). +- Forgetting to enforce the logical link in both directions (lower and upper bounds), which can lead to solutions where `y=1` but `x=0`. +- Not verifying that the sum of minimum delivery amounts from the required minimum contributors can satisfy the contract demand, potentially causing infeasibility. + +## Solving stage + +### Strategy Overview +Solve the constructed MILP using an open-source solver like SCIP or CBC via the OR-Tools wrapper. Configure solver parameters for reproducibility and performance, then extract and rigorously validate the solution. + +### Step 1 - Initialize Solver and Set Parameters +- Create a solver instance (e.g., `SCIP` or `CBC`). +- Set a time limit, optimality gap tolerance, and number of threads for deterministic and efficient solving. + +### Step 2 - Build Model from Formulation +- Use the solver's methods to create variables and add constraints as defined in the modeling stage. + +### Step 3 - Solve and Check Status +- Invoke the solver's `Solve()` method. +- Check the result status (`OPTIMAL`, `FEASIBLE`, `INFEASIBLE`, etc.) to determine success. + +### Step 4 - Extract and Validate Solution +- If the solution is optimal or feasible, retrieve the objective value and variable values. +- Programmatically verify all constraints (capacity, demand, contributor counts, minimum deliveries) to ensure the solution is correct and meets all business rules. + +### Step 5 - Report Results +- Format the allocation details (active assignments and flows) in a human-readable structure. +- Output key metrics like total cost, capacity utilization, and constraint satisfaction status. + +### Code Usage +```python +# build model from formulation +from ortools.linear_solver import pywraplp + +solver = pywraplp.Solver.CreateSolver('SCIP') +if not solver: + raise Exception('Solver not available.') + +# Set parameters +solver.SetTimeLimit(60000) # milliseconds +solver.SetNumThreads(4) + +# Create variables +x = {} +y = {} +for p in producers: + for c in contracts: + x[p, c] = solver.NumVar(0, solver.infinity(), f'x_{p}_{c}') + y[p, c] = solver.IntVar(0, 1, f'y_{p}_{c}') + +# Add constraints (example: capacity) +for p in producers: + solver.Add(sum(x[p, c] for c in contracts) <= capacity[p]) + +# ... Add all other constraints (demand, min contributors, big-M links) + +# Set objective +objective = solver.Objective() +for p in producers: + for c in contracts: + objective.SetCoefficient(x[p, c], cost[p][c]) +objective.SetMinimization() + +# solve with status / termination checks +status = solver.Solve() + +if status in (solver.OPTIMAL, solver.FEASIBLE): + print(f'Objective value: {objective.Value()}') + # Extract and validate solution + solution_flows = {(p,c): x[p,c].solution_value() for p in producers for c in contracts if x[p,c].solution_value() > 1e-6} + solution_assignments = {(p,c): y[p,c].solution_value() for p in producers for c in contracts if y[p,c].solution_value() > 0.5} + # ... Perform validation checks +else: + print('No optimal or feasible solution found.') + # Handle infeasibility (e.g., analyze constraints) +``` + +### Common Pitfalls +- Accepting a `FEASIBLE` status without checking the optimality gap, which may lead to suboptimal solutions in time-limited runs. +- Not filtering near-zero values (e.g., `< 1e-6`) when extracting flows, resulting in cluttered output. +- Failing to implement post-solve validation, which can miss subtle constraint violations due to numerical tolerances. + +# Workflow 2 (Algebraic Modeling - Pyomo) + +## Modeling stage + +### Strategy Overview +Formulate the problem using an algebraic modeling language (Pyomo). This approach separates the abstract model definition from the solver interface, promoting readability, maintainability, and easier modification of the problem structure. + +### Step 1 - Declare Abstract Sets and Parameters +- Define Pyomo `Set` objects for producers and contracts. +- Define Pyomo `Param` objects for all numerical data, indexed by the appropriate sets. + +### Step 2 - Declare Decision Variables +- Declare a continuous, non-negative `Var` for flow amounts, indexed by producer and contract. +- Declare a binary `Var` for assignment decisions, indexed by producer and contract. + +### Step 3 - Construct Constraints via Rules +- Define Python functions (rules) that return constraint expressions for each index in a set. +- Implement capacity, demand, minimum contributor count, and Big-M linking constraints using these rules. + +### Step 4 - Define the Objective Function +- Construct the objective expression as the sum of cost-weighted flows. +- Declare it as a minimization objective. + +### Step 5 - Instantiate the Concrete Model +- Create a `ConcreteModel()` and populate it with the defined components. +- Load the specific data (sets and parameters) into the model instance. + +### Formulation Template +```json +{ + "sets": [ + "model.P (producers)", + "model.C (contracts)" + ], + "parameters": [ + "model.capacity[P]", + "model.demand[C]", + "model.cost[P, C]", + "model.min_delivery[P]", + "model.min_contributors[C]" + ], + "decision_variables": [ + "model.x[P, C] (NonNegativeReals)", + "model.y[P, C] (Binary)" + ], + "objective": { + "sense": "min", + "expression": "sum(model.cost[p, c] * model.x[p, c] for p in model.P for c in model.C)" + }, + "constraints": [ + "model.capacity_con[P]: sum(model.x[p, c] for c in model.C) <= model.capacity[p]", + "model.demand_con[C]: sum(model.x[p, c] for p in model.P) >= model.demand[c]", + "model.min_contributors_con[C]: sum(model.y[p, c] for p in model.P) >= model.min_contributors[c]", + "model.min_delivery_con[P, C]: model.x[p, c] >= model.min_delivery[p] * model.y[p, c]", + "model.max_delivery_con[P, C]: model.x[p, c] <= model.capacity[p] * model.y[p, c]" + ] +} +``` + +### Common Pitfalls +- Defining constraints with incorrect index ordering in rule functions, leading to `KeyError` or wrong constraint application. +- Using mutable default arguments (like lists) within Pyomo rule functions, which can cause unexpected behavior. +- Not leveraging Pyomo's ability to define sparse sets or parameters, which can unnecessarily increase model size for problems with many potential but inactive connections. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a solver factory, which provides a clean interface to various backends (e.g., CBC, GLPK, Gurobi). Configure solver options and implement robust checks for solution status and termination conditions. + +### Step 1 - Select and Configure Solver +- Use `SolverFactory` to instantiate a solver interface (e.g., `'cbc'`). +- Pass a dictionary of solver options (time limit, MIP gap, threads) to the `solve` call. + +### Step 2 - Execute Solve and Capture Results +- Call the solver's `solve` method on the model instance. +- Capture the returned `SolverResults` object for detailed inspection. + +### Step 3 - Verify Solution Status +- Check both the solver status (`results.solver.status`) and model termination condition (`results.solver.termination_condition`). +- Proceed only if the status is `ok` and termination is `optimal` or `feasible`. + +### Step 4 - Extract and Process Solution +- Load the solution into the model instance. +- Iterate over model variables to extract flow values and active assignments, applying a tolerance for binary variables (e.g., `> 0.5`). + +### Step 5 - Validate and Report +- Perform the same programmatic validation of constraints as in Workflow 1. +- Summarize the solution, including active assignments, flows, costs, and constraint slack. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo + +model = pyo.ConcreteModel() + +# Sets +model.P = pyo.Set(initialize=producers) +model.C = pyo.Set(initialize=contracts) + +# Parameters +model.capacity = pyo.Param(model.P, initialize=capacity_data) +model.demand = pyo.Param(model.C, initialize=demand_data) +model.cost = pyo.Param(model.P, model.C, initialize=cost_data) +model.min_delivery = pyo.Param(model.P, initialize=min_delivery_data) +model.min_contributors = pyo.Param(model.C, initialize=min_contributors_data) + +# Variables +model.x = pyo.Var(model.P, model.C, domain=pyo.NonNegativeReals) +model.y = pyo.Var(model.P, model.C, domain=pyo.Binary) + +# Objective +model.obj = pyo.Objective(expr=sum(model.cost[p,c] * model.x[p,c] for p in model.P for c in model.C), sense=pyo.minimize) + +# Constraints (example: capacity rule) +def capacity_rule(m, p): + return sum(m.x[p, c] for c in m.C) <= m.capacity[p] +model.capacity_con = pyo.Constraint(model.P, rule=capacity_rule) + +# ... Define all other constraint rules + +# solve with status / termination checks +solver = pyo.SolverFactory('cbc') +results = solver.solve(model, options={'seconds': 30, 'threads': 4}) + +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in (pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible)): + print(f'Objective value: {pyo.value(model.obj)}') + # Extract solution + for p in model.P: + for c in model.C: + if model.y[p,c].value > 0.5: + flow_val = model.x[p,c].value + # ... record active assignment and flow + # ... Perform validation checks +else: + print('Solver failed to find an optimal/feasible solution.') + print(f'Status: {results.solver.status}, Termination: {results.solver.termination_condition}') +``` + +### Common Pitfalls +- Assuming `pyo.value(model.obj)` is valid without first checking the solver status, which may raise an error if the model was not solved. +- Not setting a seed for the solver when using stochastic algorithms, leading to non-reproducible results. +- Overlooking the need to explicitly load the solution into the model with `model.solutions.load_from(results)` when using certain solver managers or for advanced post-processing. diff --git a/skills/optskills/skill_library/binary_arc_selection_with_flow_conservation.md b/skills/optskills/skill_library/binary_arc_selection_with_flow_conservation.md new file mode 100644 index 0000000..b5c3d58 --- /dev/null +++ b/skills/optskills/skill_library/binary_arc_selection_with_flow_conservation.md @@ -0,0 +1,260 @@ +--- +name: Binary Arc Selection with Flow Conservation +description: | + Model and solve network optimization problems where arcs are selected via binary variables and flow conservation enforces connectivity from source to sink, minimizing total cost. +--- + +# Workflow 1 (MIP with Pyomo) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Program (MIP) using Pyomo's algebraic modeling language. Define binary variables for arc selection and enforce flow conservation constraints to ensure a connected path from source to sink, with the objective of minimizing total fixed arc costs. + +### Step 1 - Define Sets and Parameters +- Define a set `N` for all nodes in the network. +- Define a set `A` for all directed arcs, as `(i,j)` tuples where `i != j`. +- Create a parameter `c` mapping each arc in `A` to its associated cost. +- Define scalar parameters: `source` node, `sink` node, `supply` (net flow from source, e.g., 1), `demand` (net flow into sink, e.g., 1). + +### Step 2 - Create Binary Decision Variables +- Create a binary variable `x[i,j]` for each arc in set `A`. A value of 1 indicates the arc is selected. + +### Step 3 - Formulate the Objective Function +- Define the objective to minimize the total cost of selected arcs: `min sum( c[i,j] * x[i,j] for (i,j) in A )`. + +### Step 4 - Implement Flow Conservation Constraints +- For the source node `s`, enforce net outflow equals the supply: `sum( x[s,j] for j in N if (s,j) in A ) - sum( x[j,s] for j in N if (j,s) in A ) == supply`. +- For the sink node `t`, enforce net inflow equals the demand: `sum( x[j,t] for j in N if (j,t) in A ) - sum( x[t,j] for j in N if (t,j) in A ) == demand`. +- For all intermediate nodes `k` (where `k != s` and `k != t`), enforce flow balance: `sum( x[j,k] for j in N if (j,k) in A ) == sum( x[k,j] for j in N if (k,j) in A )`. + +### Formulation Template +```json +{ + "sets": [ + "N: set of nodes", + "A: set of directed arcs (i,j) where i != j" + ], + "parameters": [ + "c[A]: cost for using arc (i,j)", + "source: source node index", + "sink: sink node index", + "supply: net flow from source (e.g., 1)", + "demand: net flow into sink (e.g., 1)" + ], + "decision_variables": [ + "x[A] ∈ {0, 1}: binary arc selection" + ], + "objective": { + "sense": "min", + "expression": "sum( c[i,j] * x[i,j] for (i,j) in A )" + }, + "constraints": [ + "source_flow: sum( x[source,j] for j in N if (source,j) in A ) - sum( x[j,source] for j in N if (j,source) in A ) == supply", + "sink_flow: sum( x[j,sink] for j in N if (j,sink) in A ) - sum( x[sink,j] for j in N if (sink,j) in A ) == demand", + "flow_conservation[k in N, k != source, k != sink]: sum( x[j,k] for j in N if (j,k) in A ) == sum( x[k,j] for j in N if (k,j) in A )" + ] +} +``` + +### Common Pitfalls +- Creating variables for non-existent arcs (e.g., self-loops). Always define the arc set `A` explicitly from available cost data. +- Incorrectly signing flow balance constraints for source/sink nodes, leading to infeasibility. Verify net outflow for source is positive and net inflow for sink is positive. +- Assuming binary variables can carry fractional flow in conservation equations. The formulation uses binary selection; flow quantity is implied by the unit supply/demand. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a MILP solver backend (e.g., Gurobi, CBC, HiGHS). Configure solver options for performance and reliability, then extract and verify the solution. + +### Step 1 - Instantiate Solver and Set Options +- Create a solver object using `SolverFactory("solver_name")`. +- Set key options: a time limit (`TimeLimit` or `seconds`), optimality gap (`MIPGap` or `ratio`), thread count for parallelism, and a random seed for reproducibility. + +### Step 2 - Solve and Check Status +- Execute `solver.solve(model, tee=False)`. +- Check the solver status (`SolverStatus.ok`) and termination condition (`TerminationCondition.optimal` or `.feasible`). Proceed only if both indicate a valid solution. + +### Step 3 - Extract and Verify Solution +- Retrieve the objective value via `pyo.value(model.obj)`. +- Extract selected arcs by iterating over `model.A` and checking `pyo.value(model.x[arc]) > 0.5`. +- Optionally, verify flow conservation by recalculating inflows/outflows for all nodes using the selected arcs. + +### Step 4 - Output Results +- Print the total cost in a standard format (e.g., `RESULT:{total_cost}`) for automated parsing. +- For failures, output a structured JSON payload with status details. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Build model (follow formulation steps) +model = pyo.ConcreteModel() +model.N = pyo.Set(initialize=nodes) +model.A = pyo.Set(initialize=arcs, dimen=2) +model.c = pyo.Param(model.A, initialize=cost_dict) +model.x = pyo.Var(model.A, domain=pyo.Binary) + +def obj_rule(m): + return sum(m.c[i,j] * m.x[i,j] for (i,j) in m.A) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + +# Add constraints +def source_flow_rule(m): + return (sum(m.x[i,j] for (i,j) in m.A if i == source) - + sum(m.x[i,j] for (i,j) in m.A if j == source)) == supply +model.source_flow = pyo.Constraint(rule=source_flow_rule) + +def sink_flow_rule(m): + return (sum(m.x[i,j] for (i,j) in m.A if j == sink) - + sum(m.x[i,j] for (i,j) in m.A if i == sink)) == demand +model.sink_flow = pyo.Constraint(rule=sink_flow_rule) + +def flow_conservation_rule(m, k): + if k == source or k == sink: + return pyo.Constraint.Skip + return (sum(m.x[i,j] for (i,j) in m.A if j == k) == + sum(m.x[i,j] for (i,j) in m.A if i == k)) +model.flow_conservation = pyo.Constraint(model.N, rule=flow_conservation_rule) + +# Solve with configured solver +solver = pyo.SolverFactory("gurobi") +solver.options["TimeLimit"] = 30 +solver.options["MIPGap"] = 0.0 +solver.options["Threads"] = 4 +solver.options["Seed"] = 42 + +results = solver.solve(model, tee=False) +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + total_cost = float(pyo.value(model.obj)) + selected_arcs = [(i,j) for (i,j) in model.A if pyo.value(model.x[i,j]) > 0.5] + print(f"RESULT:{total_cost}") +else: + import json + payload = {"status": str(status), "termination": str(term)} + print(f"RESULT_JSON:{json.dumps(payload)}") +``` + +### Common Pitfalls +- Not checking both solver status and termination condition, leading to attempts to read solutions from failed solves. +- Accessing variable values for indices not in the variable set, causing KeyErrors. Always iterate over the defined set `model.A`. +- Setting overly restrictive solver options (e.g., `MIPGap=0.0` on large problems) without a time limit, causing excessive runtime. + +# Workflow 2 (Specialized Network Solver with OR-Tools) + +## Modeling stage + +### Strategy Overview +Leverage a specialized network flow algorithm (OR-Tools MinCostFlow) which internally handles flow conservation. Model the problem as a min-cost flow with unit supply at the source, unit demand at the sink, and arc capacities of 1 to enforce binary selection. + +### Step 1 - Map Network Elements +- Map node identifiers to consecutive integer indices required by the solver API. +- Define all directed arcs, excluding self-loops. + +### Step 2 - Define Arc Capacities and Costs +- Assign a capacity of 1 to each arc to enforce binary usage. +- Assign the given cost as the unit cost for flow on each arc. + +### Step 3 - Set Node Supplies +- Set the supply for the source node to the required flow amount (e.g., 1). +- Set the supply for the sink node to the negative of the flow amount (e.g., -1). +- Set the supply for all intermediate nodes to 0. + +### Step 4 - Formulate the Optimization Problem +- The solver's internal formulation minimizes `sum( cost[arc] * flow[arc] )` subject to flow conservation at nodes and flow <= capacity on arcs. + +### Formulation Template +```json +{ + "sets": [ + "N: set of node indices (0..n-1)", + "A: list of directed arcs (tail, head)" + ], + "parameters": [ + "capacity[A]: 1 for all arcs (enforces binary selection)", + "unit_cost[A]: cost per unit flow on arc", + "supply[N]: net flow for each node (source: >0, sink: <0, others: 0)" + ], + "decision_variables": [ + "flow[A] ∈ [0, capacity]: continuous flow on each arc" + ], + "objective": { + "sense": "min", + "expression": "sum( unit_cost[a] * flow[a] for a in A )" + }, + "constraints": [ + "flow_conservation[i in N]: sum( flow[a] for a in A where a.head == i ) - sum( flow[a] for a in A where a.tail == i ) == supply[i]", + "capacity[a in A]: flow[a] <= capacity[a]" + ] +} +``` + +### Common Pitfalls +- Forgetting to set supplies for all nodes, leading to an undefined flow problem. +- Using arc capacities greater than 1, which allows fractional flow and may not enforce a single path. +- Mismatching node indices between arc definitions and supply assignments. + +## Solving stage + +### Strategy Overview +Use OR-Tools' `SimpleMinCostFlow` solver, which is optimized for linear min-cost flow problems. Build the network, solve, and extract the solution based on positive flow values. + +### Step 1 - Initialize Solver and Add Arcs +- Create a `SimpleMinCostFlow` object. +- For each arc, add it to the solver using `add_arc_with_capacity_and_unit_cost(tail, head, capacity, cost)`. + +### Step 2 - Set Node Supplies +- For each node index, call `set_node_supply(node_index, supply_value)`. + +### Step 3 - Solve and Check Optimality +- Call `solve()` and check if the return status equals `SimpleMinCostFlow.OPTIMAL`. + +### Step 4 - Extract Solution +- Retrieve the total cost via `optimal_cost()`. +- Iterate through all arc indices, using `flow(arc_index)` to get the flow value. Arcs with flow > 0.5 are selected. + +### Step 5 - Verify Path Connectivity +- Trace the path from source to sink using the selected arcs to ensure a simple path is formed. + +### Code Usage +```python +from ortools.graph.python import min_cost_flow + +# Initialize solver +smcf = min_cost_flow.SimpleMinCostFlow() + +# Add arcs (assuming arcs_list contains (tail, head, cost) tuples) +for tail, head, cost in arcs_list: + arc_index = smcf.add_arc_with_capacity_and_unit_cost(tail, head, 1, cost) + +# Set supplies +node_count = max(max(tail, head) for tail, head, _ in arcs_list) + 1 +for i in range(node_count): + if i == source_index: + smcf.set_node_supply(i, 1) + elif i == sink_index: + smcf.set_node_supply(i, -1) + else: + smcf.set_node_supply(i, 0) + +# Solve +status = smcf.solve() +if status == smcf.OPTIMAL: + total_cost = smcf.optimal_cost() + selected_arcs = [] + for arc_index in range(smcf.num_arcs()): + if smcf.flow(arc_index) > 0.5: # tolerance for binary flow + selected_arcs.append((smcf.tail(arc_index), smcf.head(arc_index))) + print(f"RESULT:{total_cost}") +else: + print(f"RESULT_JSON:{{\"status\": {status}}}") +``` + +### Common Pitfalls +- Assuming the solver status is boolean; it is an integer constant (`SimpleMinCostFlow.OPTIMAL`). +- Not handling cases where the solver returns `INFEASIBLE` or `UNBALANCED`; always check the status. +- Misinterpreting flow values due to numerical precision; use a tolerance (e.g., > 0.5) when identifying selected arcs. diff --git a/skills/optskills/skill_library/binary_coverage_maximization_with_budget.md b/skills/optskills/skill_library/binary_coverage_maximization_with_budget.md new file mode 100644 index 0000000..85dc079 --- /dev/null +++ b/skills/optskills/skill_library/binary_coverage_maximization_with_budget.md @@ -0,0 +1,236 @@ +--- +name: Binary Coverage Maximization with Budget +description: | + Model and solve binary selection problems to maximize weighted coverage subject to a budget constraint, using two-layer binary variables and activation constraints. +--- + +# Workflow 1 (OR-Tools MIP Solver) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Program (MIP) using the OR-Tools linear solver wrapper. The model uses two sets of binary variables to decouple selection decisions from coverage outcomes, linked via linear activation constraints. + +### Step 1 - Define Data Structures +- Define clear sets for selectable items (e.g., `items`) and coverage targets (e.g., `targets`). +- Store parameters as dictionaries: `cost[item]`, `weight[target]`, and `budget`. +- Represent coverage relationships as a mapping `coverage_map[target] = [list_of_items]`. + +### Step 2 - Create Binary Variables +- Create selection variables: `x[item] = solver.IntVar(0, 1, f'x_{item}')`. +- Create coverage indicator variables: `y[target] = solver.IntVar(0, 1, f'y_{target}')`. + +### Step 3 - Build Coverage Activation Constraints +- For each target `t`, create a constraint: `solver.Constraint(-solver.infinity(), 0)`. +- Set coefficient for `y[t]` to `1`. +- For each covering item `i` in `coverage_map[t]`, set coefficient for `x[i]` to `-1`. +- This enforces `y[t] <= sum(x[i] for i in coverage_map[t])`. + +### Step 4 - Add Budget Constraint +- Create a constraint: `solver.Constraint(-solver.infinity(), budget)`. +- For each item `i`, set coefficient `cost[i]` for variable `x[i]`. + +### Step 5 - Define Weighted Objective +- Set the objective to maximize: `solver.Maximize(solver.Sum(weight[t] * y[t] for t in targets))`. + +### Formulation Template +```json +{ + "sets": ["items", "targets"], + "parameters": { + "cost": {"item": "float"}, + "weight": {"target": "float"}, + "budget": "float", + "coverage_map": {"target": ["list_of_items"]} + }, + "decision_variables": { + "x": {"item": "binary"}, + "y": {"target": "binary"} + }, + "objective": { + "sense": "max", + "expression": "sum(weight[t] * y[t] for t in targets)" + }, + "constraints": [ + "y[t] <= sum(x[i] for i in coverage_map[t]) for each target t", + "sum(cost[i] * x[i] for i in items) <= budget" + ] +} +``` + +### Common Pitfalls +- Forgetting to set the coefficient for the coverage indicator variable `y[t]` to `1` in the activation constraint, resulting in an incorrect inequality. +- Using `==` instead of `<=` in coverage constraints, which forces coverage when an item is selected and may eliminate beneficial solutions. +- Not verifying that the `coverage_map` dictionary includes all targets, which can lead to missing constraints. + +## Solving stage + +### Strategy Overview +Solve the MIP using the OR-Tools wrapper for SCIP or CBC. Configure solver parameters for performance and implement robust solution extraction and validation. + +### Step 1 - Initialize Solver +- Create solver: `solver = pywraplp.Solver.CreateSolver('SCIP')`. Check if `solver` is not `None`. +- Configure performance: `solver.SetTimeLimit(time_limit_ms)` and `solver.SetNumThreads(num_threads)`. + +### Step 2 - Solve and Check Status +- Invoke `solver.Solve()`. +- Check status using `result_status = solver.ResultStatus()`, not `solver.Objective().Value()`. Verify status is `OPTIMAL` or `FEASIBLE`. + +### Step 3 - Extract Solution with Tolerance +- For binary variables, use `if var.solution_value() > 0.5:` to determine selection/coverage. +- Collect lists: `selected_items = [i for i in items if x[i].solution_value() > 0.5]`. +- Calculate derived metrics: `total_cost = sum(cost[i] for i in selected_items)`. + +### Step 4 - Validate Solution +- Verify budget: Ensure `total_cost <= budget`. +- Verify coverage: For each target where `y[t].solution_value() > 0.5`, confirm at least one item in `coverage_map[t]` is selected. + +### Step 5 - Output Structured Results +- Return a JSON object containing solver status, objective value, total cost, selected items, covered targets, and a validation flag. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('SCIP') +if not solver: + raise RuntimeError('Solver not available.') +# ... (build variables, constraints, objective as per modeling stage) + +# solve with status / termination checks +solver.SetTimeLimit(30000) +solver.SetNumThreads(4) +result_status = solver.Solve() + +status_map = {0: 'OPTIMAL', 1: 'FEASIBLE', 2: 'INFEASIBLE', ...} +status = status_map.get(result_status, 'UNKNOWN') +if status not in ['OPTIMAL', 'FEASIBLE']: + return {'status': status, 'message': 'No feasible solution found.'} + +# Extract solution +selected = [i for i in items if x[i].solution_value() > 0.5] +covered = [t for t in targets if y[t].solution_value() > 0.5] +total_cost = sum(cost[i] for i in selected) +obj_value = solver.Objective().Value() +# ... (validation and output) +``` + +### Common Pitfalls +- Assuming `solver.Objective().Value()` is valid without checking `ResultStatus()` first, which can lead to errors on infeasible models. +- Using `== 1.0` to interpret binary variables, risking misclassification due to floating-point tolerances. +- Not setting a time limit, potentially causing the solver to run indefinitely on large instances. + +# Workflow 2 (Pyomo with CBC/HiGHS) + +## Modeling stage + +### Strategy Overview +Model the problem using Pyomo's `ConcreteModel` for a declarative formulation. This approach cleanly separates model construction from solver interaction and supports multiple open-source backends like CBC and HiGHS. + +### Step 1 - Define Pyomo Sets and Parameters +- Define sets: `model.items = pyo.Set(initialize=items)`, `model.targets = pyo.Set(initialize=targets)`. +- Define parameters: `model.cost = pyo.Param(model.items, initialize=cost_dict)`, `model.weight = pyo.Param(model.targets, initialize=weight_dict)`, `model.budget = pyo.Param(initialize=budget)`. +- Define coverage parameter: `model.coverage = pyo.Param(model.targets, within=pyo.Any, initialize=coverage_map)`. + +### Step 2 - Create Binary Variables +- Define selection variables: `model.x = pyo.Var(model.items, within=pyo.Binary)`. +- Define coverage variables: `model.y = pyo.Var(model.targets, within=pyo.Binary)`. + +### Step 3 - Build Coverage Activation Constraints +- Define a rule: `def coverage_rule(model, t): return model.y[t] <= sum(model.x[i] for i in model.coverage[t])`. +- Create constraint: `model.coverage_constr = pyo.Constraint(model.targets, rule=coverage_rule)`. + +### Step 4 - Add Budget Constraint +- Define rule: `def budget_rule(model): return sum(model.cost[i] * model.x[i] for i in model.items) <= model.budget`. +- Create constraint: `model.budget_constr = pyo.Constraint(rule=budget_rule)`. + +### Step 5 - Define Weighted Objective +- Define objective: `model.obj = pyo.Objective(expr=sum(model.weight[t] * model.y[t] for t in model.targets), sense=pyo.maximize)`. + +### Formulation Template +```json +{ + "sets": ["items", "targets"], + "parameters": { + "cost": {"item": "float"}, + "weight": {"target": "float"}, + "budget": "float", + "coverage": {"target": ["list_of_items"]} + }, + "decision_variables": { + "x": {"item": "binary"}, + "y": {"target": "binary"} + }, + "objective": { + "sense": "max", + "expression": "sum(weight[t] * y[t] for t in targets)" + }, + "constraints": [ + "y[t] <= sum(x[i] for i in coverage[t]) for each target t", + "sum(cost[i] * x[i] for i in items) <= budget" + ] +} +``` + +### Common Pitfalls +- Using mutable data structures (like lists) inside Pyomo rules without proper handling, which can cause unexpected behavior. +- Defining the coverage parameter with an incorrect domain (e.g., `pyo.Reals`) instead of `pyo.Any` for a list. +- Forgetting to initialize all parameters, leading to `KeyError` during model construction. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the CBC or HiGHS solver via `SolverFactory`. Configure solver options, handle solution loading explicitly, and implement comprehensive status checking. + +### Step 1 - Initialize Solver with Options +- Create solver: `solver = pyo.SolverFactory('cbc')`. +- Set options: `solver.options['seconds'] = time_limit`, `solver.options['ratio'] = optimality_gap`. + +### Step 2 - Solve with Explicit Solution Loading +- Solve with `load_solutions=False`: `results = solver.solve(model, load_solutions=False, tee=False)`. +- Check termination condition: `termination = str(results.solver.termination_condition)`. +- Check solver status: `status = results.solver.status`. + +### Step 3 - Load and Extract Solution +- If status is `ok` and termination is `optimal` or `feasible`, load solution: `model.solutions.load_from(results)`. +- Extract selected items: `selected = [i for i in model.items if pyo.value(model.x[i]) > 0.5]`. +- Calculate total cost and objective value. + +### Step 4 - Validate and Output +- Perform the same logical validation as in Workflow 1. +- Package results into a structured dictionary or JSON object. + +### Step 5 - Implement Solver Fallback +- If the primary solver fails, try an alternative (e.g., switch from `'cbc'` to `'highs'`). +- Maintain the same model structure; only change the solver factory. + +### Code Usage +```python +# build model from formulation +model = pyo.ConcreteModel() +model.items = pyo.Set(initialize=items) +# ... (build model as per modeling stage) + +# solve with status / termination checks +solver = pyo.SolverFactory('cbc') +solver.options['seconds'] = 30 +solver.options['ratio'] = 0.0 + +results = solver.solve(model, load_solutions=False, tee=False) +status = results.solver.status +termination = str(results.solver.termination_condition) + +if status == pyo.SolverStatus.ok and termination in ['optimal', 'feasible']: + model.solutions.load_from(results) + selected = [i for i in model.items if pyo.value(model.x[i]) > 0.5] + covered = [t for t in model.targets if pyo.value(model.y[t]) > 0.5] + total_cost = sum(pyo.value(model.cost[i]) for i in selected) + obj_value = pyo.value(model.obj) + # ... (validation and output) +else: + return {'status': 'FAILED', 'termination': termination, 'solver_status': str(status)} +``` + +### Common Pitfalls +- Attempting to access variable values via `pyo.value()` before loading the solution, resulting in `ValueError`. +- Not checking both `solver.status` and `termination_condition`, potentially misinterpreting suboptimal or failed solves. +- Using `tee=True` in production without capturing or suppressing the verbose solver log output. diff --git a/skills/optskills/skill_library/binary_network_flow_routing.md b/skills/optskills/skill_library/binary_network_flow_routing.md new file mode 100644 index 0000000..6d3978d --- /dev/null +++ b/skills/optskills/skill_library/binary_network_flow_routing.md @@ -0,0 +1,216 @@ +--- +name: Binary Network Flow Routing +description: | + Model and solve network flow problems with binary arc selection variables, enforcing flow conservation and minimizing linear costs, using either a direct solver API or a modeling framework. +--- + +# Workflow 1 (Direct Solver API) + +## Modeling stage + +### Strategy Overview +This workflow uses a direct solver API (e.g., OR-Tools) to construct a binary network flow model. It is procedural, building variables and constraints in loops, and is well-suited for integration into larger scripts or performance-critical applications. + +### Step 1 - Define Network Structure +- Enumerate all nodes and all possible directed arcs (i,j) where i ≠ j. +- Store arc costs in a dictionary keyed by (i,j) for efficient lookup. +- Define node supply/demand in a dictionary, where positive values are supply, negative are demand, and zero are transshipment nodes. For a single-path routing problem, set source supply = 1, sink demand = -1, and all others = 0. + +### Step 2 - Create Binary Decision Variables +- For each directed arc, create a binary variable `x[i,j] ∈ {0,1}` using `solver.BoolVar(name)`. +- Store variables in a dictionary keyed by (i,j) for easy reference in constraints and objective. + +### Step 3 - Formulate Flow Conservation Constraints +- For each node `n`, create a constraint: `sum(outgoing flow) - sum(incoming flow) = supply_demand[n]`. +- Compute sums by iterating over all arcs and filtering based on origin/destination. + +### Step 4 - Define Linear Cost Objective +- Formulate the objective as the sum of `cost[i,j] * x[i,j]` over all arcs. +- Set the sense to minimization. + +### Formulation Template +```json +{ + "sets": [ + "NODES: list of node identifiers", + "ARCS: list of directed arcs (i,j) where i, j ∈ NODES and i ≠ j" + ], + "parameters": [ + "cost[ARCS]: linear cost per arc usage", + "supply_demand[NODES]: net flow required at each node" + ], + "decision_variables": [ + "x[ARCS] ∈ {0, 1}: binary arc selection" + ], + "objective": { + "sense": "min", + "expression": "sum(cost[i,j] * x[i,j] for all (i,j) in ARCS)" + }, + "constraints": [ + "flow_conservation[n] for each n in NODES: sum(x[n,j] for j if (n,j) in ARCS) - sum(x[i,n] for i if (i,n) in ARCS) = supply_demand[n]" + ] +} +``` + +### Common Pitfalls +- Forgetting to exclude self-loops (i,i) from the arc set, which can lead to nonsensical solutions. +- Incorrectly signing the supply/demand parameter in the flow balance equation, causing flow to reverse direction. +- Using a sparse cost dictionary without ensuring all possible arcs are defined, which may unintentionally prohibit valid routes. + +## Solving stage + +### Strategy Overview +Solve the constructed model using a Mixed-Integer Programming (MIP) solver like SCIP or CBC via a direct API. Configure solver settings for performance and reproducibility, then extract and verify the solution. + +### Step 1 - Configure and Execute Solver +- Instantiate the solver (e.g., `solver = ort.Solver.CreateSolver("SCIP")`) and verify it is not null. +- Set a time limit (e.g., `[TIME_LIMIT]` milliseconds), optimality gap tolerance (e.g., 0.0 for exact), number of threads, and a random seed for reproducibility. +- Call the solver's `Solve()` method. + +### Step 2 - Check Solution Status +- Check if the solver status is `OPTIMAL` or `FEASIBLE`. +- If the status is not acceptable, output a structured error message and halt. + +### Step 3 - Extract and Verify Solution +- Retrieve the objective value. +- Iterate over all binary variables, collecting arcs where `solution_value() > 0.5` (accounting for numerical tolerance). +- Optionally, recalculate node balances from the selected arcs to verify flow conservation. +- Reconstruct the path from source to sink by following selected arcs sequentially. + +### Code Usage +```python +# build model from formulation +import ortools.linear_solver.pywraplp as ort + +solver = ort.Solver.CreateSolver("SCIP") +if not solver: + raise Exception("Solver not created") +# ... (variable and constraint creation as per modeling stage) +solver.Minimize(objective_expr) + +# solve with status / termination checks +solver.SetTimeLimit(30000) # milliseconds +solver.SetNumThreads(4) +status = solver.Solve() + +if status in [ort.Solver.OPTIMAL, ort.Solver.FEASIBLE]: + obj_val = solver.Objective().Value() + solution_arcs = [] + for (i, j), var in x_vars.items(): + if var.solution_value() > 0.5: + solution_arcs.append((i, j)) + # Output or further processing +else: + # Handle failure + print(f"RESULT_JSON:{{\"status\": \"failed\", \"solver_status\": {status}}}") +``` + +### Common Pitfalls +- Not setting a time limit, which can cause the solver to run indefinitely on difficult instances. +- Failing to check for `FEASIBLE` status in addition to `OPTIMAL`, potentially discarding valid heuristic solutions. +- Using a strict equality (`== 1.0`) to check binary variable values, which may fail due to solver tolerances; always use a threshold (e.g., `> 0.5`). + +# Workflow 2 (Modeling Framework with Pyomo) + +## Modeling stage + +### Strategy Overview +This workflow uses the Pyomo modeling framework to declaratively define the binary network flow problem. It separates model specification from solver execution, improving readability and maintainability, especially for complex models. + +### Step 1 - Declare Model Sets and Parameters +- Define `pyo.Set` objects for `nodes` and `arcs`. +- Define `pyo.Param` objects for `cost` (indexed by arcs) and `supply_demand` (indexed by nodes). + +### Step 2 - Declare Binary Variables and Objective +- Define `pyo.Var` with `domain=pyo.Binary` for arc selection, indexed by the arc set. +- Define a `pyo.Objective` with the sense `minimize` and expression as the sum of cost times variable over all arcs. + +### Step 3 - Implement Flow Balance Rule +- Create a `pyo.Constraint` indexed by nodes. +- For each node, the rule function calculates outflow and inflow using conditional sums over the arc set and enforces equality with the supply/demand parameter. + +### Formulation Template +```json +{ + "sets": [ + "NODES: pyomo Set", + "ARCS: pyomo Set (dimen=2)" + ], + "parameters": [ + "cost: pyomo Param indexed by ARCS", + "supply_demand: pyomo Param indexed by NODES" + ], + "decision_variables": [ + "x: pyomo Var indexed by ARCS, domain=Binary" + ], + "objective": { + "sense": "min", + "expression": "sum(cost[i,j] * x[i,j] for (i,j) in ARCS)" + }, + "constraints": [ + "flow_balance[n] for each n in NODES: sum(x[n,j] for j in NODES if (n,j) in ARCS) - sum(x[i,n] for i in NODES if (i,n) in ARCS) == supply_demand[n]" + ] +} +``` + +### Common Pitfalls +- Defining the arc set incorrectly (e.g., missing arcs or including self-loops) leading to an infeasible or incorrect model. +- Using a `ConcreteModel` but initializing parameters with rules that depend on mutable data, causing initialization errors. +- Writing constraint rules that inefficiently iterate over the entire node set for each node, impacting build time for large networks. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a backend solver (e.g., Gurobi, CBC). Configure solver options, execute, and rigorously check termination conditions before extracting results. Includes post-solution verification. + +### Step 1 - Configure Solver and Execute +- Use `pyo.SolverFactory` to instantiate the solver (e.g., `"gurobi"` or `"cbc"`). +- Set solver options: time limit (e.g., `[TIME_LIMIT]` seconds), optimality gap, threads, and seed. +- Call `solver.solve(model, tee=False)` to execute. + +### Step 2 - Validate Termination Status +- Check `results.solver.status == SolverStatus.ok`. +- Check `results.solver.termination_condition` is `optimal` or `feasible`. +- If checks fail, output a structured failure payload. + +### Step 3 - Extract, Verify, and Report Solution +- Retrieve the objective value using `pyo.value(model.obj)`. +- Iterate over `model.arcs` to find selected arcs (`pyo.value(model.x[i,j]) > 0.5`). +- Optionally, run a verification function to recompute node balances and ensure they match supply/demand within tolerance. +- Reconstruct the path from source to sink by following selected arcs sequentially. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition +import json + +model = pyo.ConcreteModel() +model.nodes = pyo.Set(initialize=NODE_LIST) +model.arcs = pyo.Set(initialize=ARC_LIST, dimen=2) +# ... (parameter, variable, objective, and constraint definitions as per modeling stage) + +# solve with status / termination checks +solver = pyo.SolverFactory("gurobi") +solver.options["TimeLimit"] = 30 +solver.options["MIPGap"] = 0.0 +results = solver.solve(model, tee=False) + +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}): + obj_val = pyo.value(model.obj) + selected_arcs = [(i, j) for (i, j) in model.arcs if pyo.value(model.x[i, j]) > 0.5] + print(f"RESULT:{obj_val}") +else: + failure_payload = { + "status": "failed", + "termination_condition": str(results.solver.termination_condition) + } + print(f"RESULT_JSON:{json.dumps(failure_payload)}") +``` + +### Common Pitfalls +- Accessing variable values without first checking that the solver terminated successfully, which may raise an error. +- Setting `MIPGap=0.0` without a time limit on very large problems, potentially causing excessively long run times. +- Forgetting to import necessary modules (`SolverStatus`, `TerminationCondition`) for proper status checking. diff --git a/skills/optskills/skill_library/binary_selection_with_knapsack_constraint.md b/skills/optskills/skill_library/binary_selection_with_knapsack_constraint.md new file mode 100644 index 0000000..0f41c58 --- /dev/null +++ b/skills/optskills/skill_library/binary_selection_with_knapsack_constraint.md @@ -0,0 +1,264 @@ +--- +name: Binary Selection with Knapsack Constraint +description: | + Model and solve binary selection problems with a single capacity constraint using either a dedicated knapsack solver or a general-purpose MILP solver, ensuring robust solution extraction and verification. +--- + +# Workflow 1 (Dedicated Knapsack Solver) + +## Modeling stage + +### Strategy Overview +Use a specialized knapsack algorithm for efficiency and simplicity. This workflow is ideal for pure 0-1 knapsack problems with a single resource constraint, leveraging optimized solvers that require minimal model setup. + +### Step 1 - Structure Problem Data +- Organize item data into parallel lists or arrays: `values` for benefits and `weights` for resource consumption. +- Define a scalar `capacity` representing the resource limit. +- Validate data integrity by checking list lengths and ensuring all weights are non-negative. + +### Step 2 - Map to Solver Input Format +- Recognize the solver's required input format: `profits` (values), `weights` as a list of lists (even for one dimension), and `capacities` as a list. +- Ensure the indexing of values and weights is consistent for all items. + +### Formulation Template +```json +{ + "sets": [ + {"name": "items", "indices": "range(n_items)"} + ], + "parameters": [ + {"name": "value", "indexed_by": "items", "values": "list_of_values"}, + {"name": "weight", "indexed_by": "items", "values": "list_of_weights"}, + {"name": "capacity", "value": "scalar_limit"} + ], + "decision_variables": [ + {"name": "x", "type": "binary", "indexed_by": "items"} + ], + "objective": { + "sense": "max", + "expression": "sum_{i in items} value[i] * x[i]" + }, + "constraints": [ + {"name": "capacity_limit", "expression": "sum_{i in items} weight[i] * x[i] <= capacity"} + ] +} +``` + +### Common Pitfalls +- Passing `weights` as a flat list instead of a list of lists to the solver's `init` method. +- Assuming the solver's `solve()` method takes data arguments; it should be called after `init()`. +- Not verifying the solution's feasibility by recalculating total weight against the capacity. + +## Solving stage + +### Strategy Overview +Instantiate a dedicated knapsack solver, load the pre-structured data, solve, and extract the solution using the solver's specific API. Always implement error handling and solution verification. + +### Step 1 - Initialize Solver +- Import the dedicated knapsack module (e.g., `from ortools.algorithms.python import knapsack_solver`). +- Create a solver instance with an appropriate algorithm type, such as `KNAPSACK_MULTIDIMENSION_BRANCH_AND_BOUND_SOLVER` for exact solutions. + +### Step 2 - Load Data and Solve +- Call `solver.init(values, [weights], [capacity])` to load the problem data. Note the nested list structure for weights. +- Execute `solver.solve()` without arguments to compute the solution. +- Wrap the solve call in a try-except block to handle potential API errors gracefully. + +### Step 3 - Extract and Verify Solution +- Retrieve the computed objective value from the solver. +- Identify selected items by iterating and checking `solver.best_solution_contains(i)`. +- Manually calculate the total selected weight and verify it does not exceed the capacity. +- Report key results: objective value, selected items, total weight used, and remaining capacity. + +### Code Usage +```python +from ortools.algorithms.python import knapsack_solver + +# 1. Data Preparation +values = [...] # List of item values +weights = [...] # List of item weights +capacity = ... # Scalar capacity +# Ensure weights is a list of lists for a single dimension +solver_weights = [weights] + +# 2. Solver Initialization +solver = knapsack_solver.KnapsackSolver( + knapsack_solver.SolverType.KNAPSACK_MULTIDIMENSION_BRANCH_AND_BOUND_SOLVER, + "KnapsackSolver" +) + +# 3. Load Data and Solve +solver.init(values, solver_weights, [capacity]) +try: + computed_value = solver.solve() +except Exception as e: + print(f"Solver error: {e}") + computed_value = None + +# 4. Extract and Verify Solution +if computed_value is not None: + selected_items = [i for i in range(len(values)) if solver.best_solution_contains(i)] + total_weight = sum(weights[i] for i in selected_items) + # Verification + if total_weight <= capacity: + status = "optimal_or_best_found" + else: + status = "solution_infeasible" +else: + status = "failed" + selected_items = [] + total_weight = 0 + +# 5. Output Structured Results +result = { + "status": status, + "objective_value": computed_value, + "selected_items": selected_items, + "total_weight_used": total_weight, + "remaining_capacity": capacity - total_weight +} +``` + +### Common Pitfalls +- Using incorrect import paths for the dedicated solver library. +- Not checking if the solver returned `None` as the objective value, indicating a failure. +- Accepting the solver's solution without manually recalculating constraint satisfaction. + +# Workflow 2 (General-Purpose MILP Solver) + +## Modeling stage + +### Strategy Overview +Formulate the binary selection problem as a Mixed-Integer Linear Program (MILP) using a modeling library. This approach is flexible, allowing for future extensions (e.g., additional constraints) and leverages widely available open-source solvers. + +### Step 1 - Define Model Structure +- Create a model object (e.g., Pyomo `ConcreteModel`). +- Define an index set for all items (e.g., using `pyo.Set` or `pyo.RangeSet`). + +### Step 2 - Declare Parameters and Variables +- Declare parameters for `value` and `weight`, indexed by the item set. +- Declare binary decision variables `x[i]` for each item, where `1` indicates selection. + +### Step 3 - Formulate Objective and Constraint +- Set the objective to maximize the sum of `value[i] * x[i]`. +- Add a single linear constraint: the sum of `weight[i] * x[i]` must be less than or equal to the `capacity`. + +### Formulation Template +```json +{ + "sets": [ + {"name": "I", "indices": "set_of_item_identifiers"} + ], + "parameters": [ + {"name": "value", "indexed_by": "I", "values": "dictionary_or_callable"}, + {"name": "weight", "indexed_by": "I", "values": "dictionary_or_callable"}, + {"name": "capacity", "value": "scalar_limit"} + ], + "decision_variables": [ + {"name": "x", "type": "binary", "indexed_by": "I"} + ], + "objective": { + "sense": "max", + "expression": "sum(value[i] * x[i] for i in I)" + }, + "constraints": [ + {"name": "capacity_constraint", "expression": "sum(weight[i] * x[i] for i in I) <= capacity"} + ] +} +``` + +### Common Pitfalls +- Not using consistent indexing between parameters, variables, and data sources. +- Forgetting to set the objective sense to `maximize`. +- Hard-coding data within the model construction, reducing reusability. + +## Solving stage + +### Strategy Overview +Use a general-purpose MILP solver (e.g., CBC, HiGHS, SCIP) via a modeling library's interface. Configure solver options, solve the model, and rigorously check the solver status before extracting and verifying the solution. + +### Step 1 - Configure and Execute Solver +- Instantiate a solver factory (e.g., `SolverFactory("cbc")`). +- Set practical options: time limit (`seconds`), optimality gap tolerance (`ratio` or `mip_rel_gap`), and thread count for parallelism. +- Call the solver with the model. Consider using `load_solutions=False` initially to control solution loading. + +### Step 2 - Check Solver Status +- After solving, check `solver.status` (should be `SolverStatus.ok`). +- Check `solver.termination_condition`. Accept `TerminationCondition.optimal` or `.feasible` as successful outcomes. + +### Step 3 - Extract and Validate Solution +- If status is acceptable, load the solution into the model variables. +- Extract selected items by filtering variables where `value(x[i]) > 0.5` (accounting for numerical tolerance). +- Recalculate the total objective value and total weight to verify against the model's reported values and the capacity constraint. +- Package results in a structured format (e.g., JSON) for downstream use. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# 1. Build Model +def build_knapsack_model(items, value_dict, weight_dict, capacity): + model = pyo.ConcreteModel() + model.I = pyo.Set(initialize=items) + model.value = pyo.Param(model.I, initialize=value_dict) + model.weight = pyo.Param(model.I, initialize=weight_dict) + model.capacity = pyo.Param(initialize=capacity, mutable=True) + model.x = pyo.Var(model.I, domain=pyo.Binary) + model.obj = pyo.Objective( + expr=sum(model.value[i] * model.x[i] for i in model.I), + sense=pyo.maximize + ) + model.cap_con = pyo.Constraint( + expr=sum(model.weight[i] * model.x[i] for i in model.I) <= model.capacity + ) + return model + +# 2. Solve Model +items = [...] # List of item identifiers +value = {...} # Dict: item_id -> value +weight = {...} # Dict: item_id -> weight +capacity = ... + +model = build_knapsack_model(items, value, weight, capacity) +solver = pyo.SolverFactory("cbc") # Can substitute "highs" or "scip" +solver.options["seconds"] = 30 +solver.options["ratio"] = 0.0 + +# Solve with controlled solution loading +results = solver.solve(model, load_solutions=False) + +# 3. Check Status and Extract +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + model.solutions.load_from(results) + selected_items = [i for i in model.I if pyo.value(model.x[i]) > 0.5] + total_value = float(pyo.value(model.obj)) + total_weight = sum(weight[i] for i in selected_items) + # Verification + if total_weight <= capacity: + result_status = "optimal" if term == TerminationCondition.optimal else "feasible" + else: + result_status = "solution_infeasible" +else: + result_status = f"failed_{status}:{term}" + selected_items = [] + total_value = None + total_weight = 0 + +# 4. Output Structured Results +result = { + "status": result_status, + "objective_value": total_value, + "selected_items": selected_items, + "total_weight_used": total_weight, + "remaining_capacity": capacity - total_weight +} +``` + +### Common Pitfalls +- Assuming solver availability without checking imports or installation. +- Extracting variable values without first checking the solver status and termination condition. +- Not verifying the solution's feasibility by recalculating the constraint satisfaction manually. +- Setting an excessively tight optimality gap or time limit for simple problems, wasting resources. diff --git a/skills/optskills/skill_library/binary_set_covering_with_cost_minimization.md b/skills/optskills/skill_library/binary_set_covering_with_cost_minimization.md new file mode 100644 index 0000000..94a4cc6 --- /dev/null +++ b/skills/optskills/skill_library/binary_set_covering_with_cost_minimization.md @@ -0,0 +1,220 @@ +--- +name: Binary Set Covering with Cost Minimization +description: | + Model and solve binary set covering problems where a subset of items must be selected to satisfy coverage requirements while minimizing total cost, using either OR-Tools or Pyomo. +--- + +# Workflow 1 (OR-Tools SCIP Solver) + +## Modeling stage + +### Strategy Overview +Define binary decision variables for each candidate item, enforce set covering constraints as linear inequalities requiring at least a minimum number of selected items from specified subsets, and formulate a linear cost minimization objective. + +### Step 1 - Define Decision Variables +- Create a binary integer variable for each candidate item using `solver.IntVar(0, 1, name)` to represent selection status (1 = selected, 0 = not selected). +- Store variables in a dictionary keyed by item identifiers for easy access during constraint and objective construction. + +### Step 2 - Enforce Set Covering Constraints +- For each coverage requirement, construct a linear inequality that sums the binary variables of items in the relevant subset and enforces a lower bound: `solver.Add(sum(variables_in_subset) >= required_count)`. +- When multiple coverage requirements share identical item subsets, include each as a separate constraint; duplicates do not alter the feasible region but ensure completeness. + +### Step 3 - Build Minimization Objective +- Create an empty `Objective()` object using `solver.Objective()`. +- For each item, set its cost coefficient with `objective.SetCoefficient(variable, cost)`. +- Call `objective.SetMinimization()` to specify the optimization sense. + +### Formulation Template +```json +{ + "sets": ["I: set of candidate items"], + "parameters": ["c_i: cost of selecting item i", "S_j: subset of items for coverage requirement j", "r_j: minimum number of items to select from S_j"], + "decision_variables": ["x_i ∈ {0,1}: 1 if item i is selected"], + "objective": { + "sense": "min", + "expression": "sum_{i∈I} c_i * x_i" + }, + "constraints": ["sum_{i∈S_j} x_i >= r_j, ∀j"] +} +``` + +### Common Pitfalls +- Forgetting to set the objective sense to minimization, which defaults to maximization in OR-Tools. +- Using `solver.Sum()` instead of Python's built-in `sum()` for small expressions; both work but `sum()` is more readable for simple cases. +- Not storing variables in a dictionary, leading to difficulty when referencing them in constraints. + +## Solving stage + +### Strategy Overview +Initialize a SCIP solver, configure time limits and parallelism, solve the model, check solution status, and extract binary decisions with a threshold. + +### Step 1 - Initialize Solver and Configure Options +- Create solver instance with `pywraplp.Solver.CreateSolver("SCIP")` and verify it is not `None` to handle missing solver gracefully. +- Set a time limit in milliseconds using `solver.SetTimeLimit(milliseconds)` and thread count with `solver.SetNumThreads(n)`. + +### Step 2 - Solve and Check Status +- Call `solver.Solve()` and store the result status. +- Check if status is `pywraplp.Solver.OPTIMAL` or `pywraplp.Solver.FEASIBLE` before reading solution values. + +### Step 3 - Extract Results +- Retrieve variable values using `variable.solution_value()` and compare against a threshold (e.g., `> 0.5`) to determine binary decisions. +- Obtain the objective value with `solver.Objective().Value()`. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +def solve_set_covering(items, costs, coverage_requirements): + """ + items: list of item identifiers + costs: dict mapping item -> cost + coverage_requirements: list of (subset_of_items, min_required) + """ + solver = pywraplp.Solver.CreateSolver("SCIP") + if not solver: + return {"status": "failed", "reason": "SCIP solver not available"} + + # Step 1: Decision variables + x = {i: solver.IntVar(0, 1, f"x_{i}") for i in items} + + # Step 2: Set covering constraints + for subset, required in coverage_requirements: + solver.Add(sum(x[i] for i in subset) >= required) + + # Step 3: Objective + objective = solver.Objective() + for i in items: + objective.SetCoefficient(x[i], costs[i]) + objective.SetMinimization() + + # Solving stage + solver.SetTimeLimit(30000) # 30 seconds + solver.SetNumThreads(4) + status = solver.Solve() + + if status not in (pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE): + return {"status": "failed", "reason": f"Solver status: {status}"} + + selected = [i for i in items if x[i].solution_value() > 0.5] + return { + "status": "success", + "objective": solver.Objective().Value(), + "selected_items": selected + } +``` + +### Common Pitfalls +- Not checking solver initialization failure, leading to cryptic errors later. +- Using integer comparison (`== 1`) instead of threshold comparison for floating-point solution values. +- Forgetting to cast objective value to float for JSON serialization. + +# Workflow 2 (Pyomo with CBC/GLPK) + +## Modeling stage + +### Strategy Overview +Use Pyomo's abstract modeling framework with binary variables, a ConstraintList for flexible constraint addition, and a linear cost minimization objective. This approach supports systematic constraint generation and easy debugging. + +### Step 1 - Define Binary Variables +- Create a Pyomo ConcreteModel and define binary variables using `pyo.Var(domain=pyo.Binary)` for each candidate item. +- Index variables by item identifiers for clean mapping to cost and constraint data. + +### Step 2 - Add Set Covering Constraints +- Use `pyo.ConstraintList()` to dynamically add constraints. +- For each coverage requirement, add a constraint expression: `sum(var[i] for i in subset) >= required`. +- When generating constraints systematically (e.g., all non-empty subsets), use `itertools.combinations` to enumerate subsets and feasible minimum requirements. + +### Step 3 - Formulate Objective +- Define a linear objective expression: `sum(cost[i] * var[i] for i in items)`. +- Set the objective sense to minimization with `sense=pyo.minimize`. + +### Formulation Template +```json +{ + "sets": ["I: set of candidate items"], + "parameters": ["c_i: cost of selecting item i", "S_j: subset of items for coverage requirement j", "r_j: minimum number of items to select from S_j"], + "decision_variables": ["x_i ∈ {0,1}: 1 if item i is selected"], + "objective": { + "sense": "min", + "expression": "sum_{i∈I} c_i * x_i" + }, + "constraints": ["sum_{i∈S_j} x_i >= r_j, ∀j"] +} +``` + +### Common Pitfalls +- Adding duplicate constraints to ConstraintList unnecessarily; while harmless, it increases model size. +- Forgetting to import itertools when generating subsets systematically. +- Using mutable data structures (lists) in constraint expressions that change after model creation. + +## Solving stage + +### Strategy Overview +Configure a CBC or GLPK solver with time limits and optimality requirements, solve the model, verify solver status, and extract results in a structured format. + +### Step 1 - Configure Solver +- Initialize solver with `pyo.SolverFactory("cbc")` or `pyo.SolverFactory("glpk")`. +- Set solver options: for CBC use `options["seconds"] = 30` and `options["ratio"] = 0.0`; for GLPK use `options["tmlim"] = 30` and `options["mipgap"] = 0.0`. + +### Step 2 - Solve and Validate Status +- Call `solver.solve(model, tee=False)` and store results. +- Check `results.solver.status == SolverStatus.ok` and `results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}`. + +### Step 3 - Extract and Output Results +- Retrieve variable values with `int(pyo.value(var[i]))` and objective with `float(pyo.value(model.obj))`. +- Output results as a JSON payload with a prefix like `RESULT_JSON:` for structured parsing. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition +import itertools + +def solve_set_covering_pyomo(items, costs, coverage_requirements): + """ + items: list of item identifiers + costs: dict mapping item -> cost + coverage_requirements: list of (subset_of_items, min_required) + """ + model = pyo.ConcreteModel() + model.items = pyo.Set(initialize=items) + + # Step 1: Decision variables + model.x = pyo.Var(model.items, domain=pyo.Binary) + + # Step 2: Set covering constraints + model.constraints = pyo.ConstraintList() + for subset, required in coverage_requirements: + model.constraints.add(sum(model.x[i] for i in subset) >= required) + + # Step 3: Objective + def obj_rule(m): + return sum(costs[i] * m.x[i] for i in m.items) + model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + + # Solving stage + solver = pyo.SolverFactory("cbc") + solver.options["seconds"] = 30 + solver.options["ratio"] = 0.0 + + results = solver.solve(model, tee=False) + + if results.solver.status != SolverStatus.ok: + return {"status": "failed", "reason": f"Solver status: {results.solver.status}"} + + term = results.solver.termination_condition + if term not in (TerminationCondition.optimal, TerminationCondition.feasible): + return {"status": "failed", "reason": f"Termination: {term}"} + + selected = [i for i in items if int(pyo.value(model.x[i])) == 1] + return { + "status": "success", + "objective": float(pyo.value(model.obj)), + "selected_items": selected + } +``` + +### Common Pitfalls +- Not checking both solver status and termination condition, leading to acceptance of invalid solutions. +- Using `pyo.value()` without casting to int for binary variables, which may return floating-point values. +- Forgetting to set `tee=False` to suppress solver output in production code. diff --git a/skills/optskills/skill_library/binary_subset_selection_with_pairwise_interactions.md b/skills/optskills/skill_library/binary_subset_selection_with_pairwise_interactions.md new file mode 100644 index 0000000..ae2a80c --- /dev/null +++ b/skills/optskills/skill_library/binary_subset_selection_with_pairwise_interactions.md @@ -0,0 +1,227 @@ +--- +name: Binary Subset Selection with Pairwise Interactions +description: | + Model and solve subset selection problems with cardinality constraints and pairwise interaction objectives using linearized binary variables. +--- + +# Workflow 1 (Pyomo with MIP Solver) + +## Modeling stage + +### Strategy Overview +This workflow uses Pyomo to formulate a Mixed-Integer Programming (MIP) model. It explicitly linearizes the quadratic pairwise selection term using auxiliary binary variables and standard logical linking constraints, suitable for solvers like Gurobi, CPLEX, or HiGHS. + +### Step 1 - Define Core Selection Variables +- Define a set `N` representing all candidate elements. +- Create binary decision variables `x[i]` for each `i` in `N`, where `x[i] = 1` indicates element `i` is selected. + +### Step 2 - Define Pairwise Activation Variables +- For each ordered pair `(i, j)` where `i != j`, create auxiliary binary variables `z[i,j]`. +- The variable `z[i,j]` will be forced to equal `x[i] * x[j]` via constraints. + +### Step 3 - Enforce Logical Linking +- Add constraints `z[i,j] <= x[i]` for all `i != j`. +- Add constraints `z[i,j] <= x[j]` for all `i != j`. +- Add constraints `z[i,j] >= x[i] + x[j] - 1` for all `i != j`. This set of constraints enforces `z[i,j] = 1` if and only if `x[i] = 1` and `x[j] = 1`. + +### Step 4 - Apply Cardinality Constraint +- Add a single constraint: `sum(x[i] for i in N) == k`, where `k` is the required number of selected elements. + +### Step 5 - Formulate Pairwise Sum Objective +- Define a parameter `d[i,j]` representing the pairwise contribution (e.g., distance, similarity, cost). +- Formulate the objective as `maximize sum(d[i,j] * z[i,j] for i in N for j in N if i != j)`. + +### Formulation Template +```json +{ + "sets": [ + "N: set of candidate elements" + ], + "parameters": [ + "k: required number of selected elements (integer)", + "d[i,j]: pairwise contribution value for ordered pair (i, j) where i != j" + ], + "decision_variables": [ + "x[i]: binary, 1 if element i is selected", + "z[i,j]: binary, 1 if both i and j are selected (i != j)" + ], + "objective": { + "sense": "max", + "expression": "sum_{i in N, j in N, i != j} d[i,j] * z[i,j]" + }, + "constraints": [ + "cardinality: sum_{i in N} x[i] == k", + "link_z_to_x_i: z[i,j] <= x[i] for all i, j in N, i != j", + "link_z_to_x_j: z[i,j] <= x[j] for all i, j in N, i != j", + "enforce_conjunction: z[i,j] >= x[i] + x[j] - 1 for all i, j in N, i != j" + ] +} +``` + +### Common Pitfalls +- Creating `z[i,i]` variables for diagonal pairs, which are unnecessary and can complicate the model. Ensure `i != j`. +- Using asymmetric pairwise data `d[i,j]` when the problem context implies symmetry (e.g., distance). If symmetric, consider using `i < j` to reduce variable count. +- Over-constraining the model by adding redundant constraints beyond the standard linearization. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a MIP solver via the `SolverFactory`. The focus is on robust solver configuration, explicit solution status checking, and verification of results. + +### Step 1 - Configure Solver with Deterministic Settings +- Instantiate the solver (e.g., `SolverFactory("gurobi")`, `SolverFactory("highs")`). +- Set key options: a time limit, zero optimality gap tolerance for exact solutions, a fixed random seed for reproducibility, and thread count. +- Example: `solver.options["TimeLimit"] = 30`, `solver.options["MIPGap"] = 0.0`, `solver.options["Seed"] = 42`, `solver.options["Threads"] = 4`. + +### Step 2 - Solve and Check Status +- Execute `results = solver.solve(model, tee=False)`. +- Check if the solver status is `SolverStatus.ok`. +- Check if the termination condition is `TerminationCondition.optimal` or `TerminationCondition.feasible`. + +### Step 3 - Load and Extract Solution +- If status checks pass, load the solution: `model.solutions.load_from(results)`. +- Extract selected elements: `selected = [i for i in model.N if pyo.value(model.x[i]) > 0.5]`. +- Compute the objective value: `obj_val = pyo.value(model.obj)`. + +### Step 4 - Verify Solution (Optional, for small instances) +- For validation, enumerate all `k`-combinations of elements. +- Calculate the objective for each combination by summing the relevant `d[i,j]` values. +- Confirm the solver's solution matches the best found via enumeration. + +### Code Usage +```python +import pyomo.environ as pyo + +# Assume `model` is built according to the modeling stage +solver = pyo.SolverFactory("highs") # or "gurobi", "cplex" +solver.options["time_limit"] = 30 +solver.options["mip_rel_gap"] = 0.0 +solver.options["threads"] = 4 + +results = solver.solve(model, tee=False) + +from pyomo.opt import SolverStatus, TerminationCondition +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + # Load solution before accessing values + model.solutions.load_from(results) + obj_val = pyo.value(model.obj) + selected = [i for i in model.N if pyo.value(model.x[i]) > 0.5] + print(f"RESULT:{obj_val}") + print(f"Selected: {selected}") +else: + # Handle failure + error_payload = {"solver_status": str(status), "termination_condition": str(term)} + print(f"ERROR:{error_payload}") +``` + +### Common Pitfalls +- Accessing variable values (`pyo.value`) before loading the solution, which raises an error. Always load the solution first. +- Setting an excessively low time limit or zero gap for large instances, causing the solver to fail to find a feasible solution. +- Forgetting to set a random seed, leading to non-reproducible results across runs. + +# Workflow 2 (ORTools CP-SAT with Direct Multiplication) + +## Modeling stage + +### Strategy Overview +This workflow uses Google's OR-Tools CP-SAT solver, which natively supports linear constraints and offers `AddMultiplicationEquality` to handle the product of binary variables without manually adding the three linearization constraints. This leads to a cleaner model. + +### Step 1 - Define Core Selection Variables +- Create the CP-SAT model instance. +- For each element `i` in set `N`, create a binary variable `x[i]` using `model.NewBoolVar()`. + +### Step 2 - Enforce Cardinality Constraint +- Use `model.Add(sum(x[i] for i in N) == k)` to enforce the exact selection count. + +### Step 3 - Handle Pairwise Products for Objective +- For each relevant pair `(i, j)` (e.g., `i < j` for symmetric objectives), create an auxiliary variable `z[i,j]` using `model.NewBoolVar()`. +- Use `model.AddMultiplicationEquality(z[i,j], [x[i], x[j]])` to enforce `z[i,j] == x[i] * x[j]`. This single constraint replaces the three linear constraints used in Workflow 1. + +### Step 4 - Formulate Linear Objective +- Define the objective as `maximize sum(d[i,j] * z[i,j] for all defined pairs)`. +- Use `model.Maximize()` to set the objective expression. + +### Formulation Template +```json +{ + "sets": [ + "N: set of candidate elements" + ], + "parameters": [ + "k: required number of selected elements (integer)", + "d[i,j]: pairwise contribution value for pair (i, j). For symmetric problems, can be defined for i < j." + ], + "decision_variables": [ + "x[i]: boolean CP-SAT variable", + "z[i,j]: boolean CP-SAT variable representing x[i] * x[j]" + ], + "objective": { + "sense": "max", + "expression": "sum_{defined pairs (i,j)} d[i,j] * z[i,j]" + }, + "constraints": [ + "cardinality: sum_{i in N} x[i] == k", + "product_relation: z[i,j] == x[i] * x[j] for all defined pairs (via AddMultiplicationEquality)" + ] +} +``` + +### Common Pitfalls +- Manually adding the three linear constraints (`<=`, `<=`, `>=`) in CP-SAT, which is less efficient and more error-prone than using `AddMultiplicationEquality`. +- Defining `z` variables for all ordered pairs `(i, j)` when the objective is symmetric, unnecessarily doubling the variable count. Use `i < j` for symmetric `d`. +- Incorrectly using `model.NewIntVar` instead of `model.NewBoolVar` for binary variables. + +## Solving stage + +### Strategy Overview +Solve the model using the CP-SAT solver with appropriate parameters. The workflow emphasizes efficient solving and simple, robust result extraction due to CP-SAT's solution proto structure. + +### Step 1 - Configure Solver Parameters +- Create a `CpSolverParameters()` object. +- Set parameters like `max_time_in_seconds`, `num_search_workers` (parallelism), and `random_seed` for reproducibility. +- For small instances, use default parameters or minimal configuration to avoid overhead. + +### Step 2 - Solve and Check Status +- Instantiate `CpSolver()` and call `solver.Solve(model, parameters)`. +- Check the status: `status == cp_model.OPTIMAL` or `status == cp_model.FEASIBLE`. + +### Step 3 - Extract Solution +- If status is acceptable, iterate over `x[i]` variables: `selected = [i for i in N if solver.Value(x[i]) == 1]`. +- The objective value is obtained via `solver.ObjectiveValue()`. + +### Step 4 - Optional Verification +- For small `N`, verify by enumerating combinations as in Workflow 1. + +### Code Usage +```python +from ortools.sat.python import cp_model + +model = cp_model.CpModel() +# Assume variables x and z are created, constraints and objective are added + +solver = cp_model.CpSolver() +# Set parameters appropriately for problem size +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 4 # Adjust based on available cores +solver.parameters.random_seed = 42 + +status = solver.Solve(model) + +if status in [cp_model.OPTIMAL, cp_model.FEASIBLE]: + obj_val = solver.ObjectiveValue() + selected = [i for i in N if solver.Value(x[i]) == 1] + print(f"RESULT:{obj_val}") + print(f"Selected: {selected}") +else: + # Handle failure + error_payload = {"solver_status": status} + print(f"ERROR:{error_payload}") +``` + +### Common Pitfalls +- Setting excessive parallelism (`num_search_workers`) for tiny problems, which can introduce overhead without benefit. +- Not checking for both `OPTIMAL` and `FEASIBLE` statuses, potentially discarding good feasible solutions when a time limit is reached. +- Assuming `solver.ObjectiveValue()` is valid for `FEASIBLE` status; it is, but only if an objective was set. diff --git a/skills/optskills/skill_library/binaryselectionmaximindistance.md b/skills/optskills/skill_library/binaryselectionmaximindistance.md new file mode 100644 index 0000000..63caf9c --- /dev/null +++ b/skills/optskills/skill_library/binaryselectionmaximindistance.md @@ -0,0 +1,268 @@ +--- +name: BinarySelectionMaximinDistance +description: | + Model and solve binary selection problems with pairwise activation constraints to maximize the minimum distance (or other pairwise metric) among selected items, using either a direct CP-SAT or a Pyomo-based MILP approach. + +--- +# Workflow 1 (CP-SAT Direct Formulation) + +## Modeling stage + +### Strategy Overview +This workflow uses Google OR-Tools' CP-SAT solver directly. It is ideal for problems with purely binary and logical constraints, offering a concise API for variable creation, constraint addition, and objective definition without an intermediate algebraic modeling layer. + +### Step 1 - Define Core Selection Variables +- Create a binary decision variable `x[i]` for each candidate item `i` in the set `N`. This variable equals 1 if the item is selected. +- Use `model.NewBoolVar(f"x_{i}")` to instantiate each variable. + +### Step 2 - Define Pair Activation Variables +- For each unordered pair `(i, j)` where `i < j`, create an auxiliary binary variable `y[(i, j)]`. +- This variable indicates whether both items `i` and `j` are selected. Use `model.NewBoolVar(f"y_{i}_{j}")`. + +### Step 3 - Link Pair Activation to Selection +- Add logical constraints to enforce `y[(i, j)] = 1` if and only if `x[i] = 1` and `x[j] = 1`. +- Implement with three linear constraints per pair: + - `y[(i, j)] <= x[i]` + - `y[(i, j)] <= x[j]` + - `y[(i, j)] >= x[i] + x[j] - 1` + +### Step 4 - Enforce Selection Cardinality +- Add a single linear constraint to select exactly `K` items: `sum(x[i] for i in N) == K`. + +### Step 5 - Formulate Maximin Objective +- Introduce a continuous variable `z` to represent the minimum distance among selected pairs. Define it with appropriate bounds, e.g., `model.NewNumVar(0, M, "z")`. +- For each pair `(i, j)`, add a conditional lower bound constraint using a big-M formulation: `z <= distance[i][j] + M * (1 - y[(i, j)])`. +- The constant `M` must be larger than any possible pairwise distance to deactivate the constraint when the pair is not selected (`y=0`). A safe choice is `max(distance) * 1.1` or a fixed large value like `1e6`. +- Set the objective to maximize `z`. + +### Formulation Template +```json +{ + "sets": [ + "N: Set of candidate items.", + "P: Set of unordered pairs (i, j) where i, j in N and i < j." + ], + "parameters": [ + "distance[i][j]: Non-negative distance (or metric) for pair (i, j).", + "K: Exact number of items to select (integer).", + "M: A sufficiently large constant (e.g., max(distance) * 2)." + ], + "decision_variables": [ + "x[i] ∈ {0, 1}, ∀ i ∈ N. Selection indicator.", + "y[(i, j)] ∈ {0, 1}, ∀ (i, j) ∈ P. Pair activation indicator.", + "z ∈ [0, M]. Continuous variable for the minimum distance." + ], + "objective": { + "sense": "max", + "expression": "z" + }, + "constraints": [ + "sum(x[i] for i in N) == K", + "y[(i, j)] <= x[i], ∀ (i, j) ∈ P", + "y[(i, j)] <= x[j], ∀ (i, j) ∈ P", + "y[(i, j)] >= x[i] + x[j] - 1, ∀ (i, j) ∈ P", + "z <= distance[i][j] + M * (1 - y[(i, j)]), ∀ (i, j) ∈ P" + ] +} +``` + +### Common Pitfalls +- Setting `M` too small, which can cut off valid solutions. Calculate it as `max(distance) * 1.1` or a fixed large value like `1e6`. +- Forgetting to define the pair set `P` for unordered pairs, leading to duplicate variables and constraints for `(i, j)` and `(j, i)`. +- Not handling the trivial case where `K < 2`. The model is still valid, but `z` may be unbounded. Consider adding a constraint `sum(y) >= 1` if `K >= 2` is guaranteed. + +## Solving stage + +### Strategy Overview +Solve the model using the CP-SAT solver, which is designed for linear constraints over Boolean and integer variables. Configure solver parameters for performance and reliability, then extract and validate the solution. + +### Step 1 - Instantiate Solver and Configure +- Create a solver instance: `solver = cp_model.CpSolver()`. +- Set key parameters to control the search: + - `solver.parameters.max_time_in_seconds = [TIME_LIMIT]` + - `solver.parameters.num_search_workers = [NUM_CORES]` + - `solver.parameters.random_seed = [SEED]` for reproducibility. + - For an exact solution, ensure `solver.parameters.relative_gap_limit = 0.0`. + +### Step 2 - Execute Solve and Check Status +- Call `status = solver.Solve(model)`. +- Check the result status against `cp_model.OPTIMAL`, `cp_model.FEASIBLE`, `cp_model.INFEASIBLE`, or `cp_model.UNKNOWN`. +- **Prerequisite Check:** If the status is not `OPTIMAL` or `FEASIBLE`, do not proceed to solution extraction. Handle the failure (e.g., log status, return empty solution). + +### Step 3 - Extract and Post-process Solution +- If the solve was successful, retrieve the objective value: `obj_val = solver.ObjectiveValue()`. +- Extract selected items by checking `solver.Value(x_var) == 1` for each `x` variable. +- Optionally, extract active pairs by checking `solver.Value(y_var) == 1`. +- Package the solution (selected indices, objective value, solver status) into a structured output (e.g., a dictionary). + +### Code Usage +```python +# build model from formulation +import ortools.sat.python.cp_model as cp + +model = cp.CpModel() +# ... (build variables and constraints as per Modeling Stage) + +# solve with status / termination checks +solver = cp.CpSolver() +# Apply configuration +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 4 +solver.parameters.random_seed = 42 + +status = solver.Solve(model) + +if status in (cp.OPTIMAL, cp.FEASIBLE): + selected_items = [i for i, var in x_vars.items() if solver.Value(var) == 1] + min_distance = solver.ObjectiveValue() + solution = { + "status": "OPTIMAL" if status == cp.OPTIMAL else "FEASIBLE", + "objective": min_distance, + "selected": selected_items + } +else: + # Do not output pseudo numeric answers when execution fails. + solution = { + "status": "INFEASIBLE" if status == cp.INFEASIBLE else "UNKNOWN", + "objective": None, + "selected": [] + } +print(solution) +``` + +### Common Pitfalls +- Assuming `solver.Value(var)` is valid without checking the solve status first, which can cause runtime errors. +- Not setting a time limit for large instances, potentially causing the solver to run indefinitely. +- Misinterpreting `FEASIBLE` as `OPTIMAL`; for reporting, distinguish between proven optimal and heuristic solutions. + +# Workflow 2 (Pyomo MILP with External Solver) + +## Modeling stage + +### Strategy Overview +This workflow uses Pyomo, an algebraic modeling language, to formulate the problem as a Mixed-Integer Linear Program (MILP). It separates the model definition from the solver interface, allowing flexibility to use various solvers (e.g., Gurobi, SCIP, CBC) and enabling more complex model extensions. + +### Step 1 - Define Abstract Sets and Parameters +- Declare an abstract Pyomo `Set` for candidate items `model.N`. +- Declare a `Set` for unordered pairs `model.P`, defined as a subset of `model.N × model.N` with `i < j`. +- Declare `Param` for distances `model.dist` indexed over `model.P` and the selection count `model.K`. + +### Step 2 - Define Decision Variables +- Create binary selection variables: `model.x = Var(model.N, within=Binary)`. +- Create binary pair activation variables: `model.y = Var(model.P, within=Binary)`. +- Create a continuous variable for the minimum distance: `model.z = Var(within=NonNegativeReals, bounds=(0, M))`. + +### Step 3 - Enforce Logical and Cardinality Constraints +- Add the cardinality constraint: `sum(model.x[i] for i in model.N) == model.K`. +- For each pair in `model.P`, add the three linking constraints using Pyomo's `Constraint` construct. +- Use a `ConstraintList` or a rule-based `Constraint` for efficient generation. + +### Step 4 - Implement Maximin via Big-M Constraints +- For each pair `(i, j)` in `model.P`, add constraint: `model.z <= model.dist[i, j] + M * (1 - model.y[i, j])`. +- This ensures `model.z` is bounded by the distance of every active pair. + +### Step 5 - Define the Objective +- Set the objective to maximize `model.z`: `model.obj = Objective(expr=model.z, sense=maximize)`. + +### Formulation Template +```json +{ + "sets": [ + "N: Pyomo Set of candidate items.", + "P: Pyomo Set of unordered pairs (i, j) where i, j in N and i < j." + ], + "parameters": [ + "dist[i, j]: Pyomo Param for distance, indexed over P.", + "K: Pyomo Param (or scalar) for the number of items to select.", + "M: Scalar large constant for big-M constraints." + ], + "decision_variables": [ + "x[i]: Pyomo Var, within=Binary, ∀ i ∈ N.", + "y[i, j]: Pyomo Var, within=Binary, ∀ (i, j) ∈ P.", + "z: Pyomo Var, within=NonNegativeReals." + ], + "objective": { + "sense": "max", + "expression": "z" + }, + "constraints": [ + "cardinality: sum(x[i] for i in N) == K", + "link_lower_i: y[i, j] <= x[i], ∀ (i, j) ∈ P", + "link_lower_j: y[i, j] <= x[j], ∀ (i, j) ∈ P", + "link_upper: y[i, j] >= x[i] + x[j] - 1, ∀ (i, j) ∈ P", + "min_distance: z <= dist[i, j] + M * (1 - y[i, j]), ∀ (i, j) ∈ P" + ] +} +``` + +### Common Pitfalls +- Defining the pair set `P` incorrectly, leading to key errors when accessing `dist[i, j]`. Ensure it aligns with the distance dictionary's keys. +- Using a mutable default argument (like a list) in Pyomo constraint rules; define rules with explicit function signatures. +- Forgetting to deactivate the big-M constraint for inactive pairs by setting `M` large enough; otherwise, `z` may be incorrectly constrained. + +## Solving stage + +### Strategy Overview +Use Pyomo's `SolverFactory` to interface with an external MILP solver (e.g., Gurobi, SCIP, CBC). Configure solver-specific options for performance, then solve and rigorously check the termination condition before extracting results. + +### Step 1 - Instantiate Solver and Set Options +- Create a solver object: `solver = SolverFactory('SOLVER_NAME')` (e.g., `'gurobi'`, `'scip'`, `'cbc'`). +- Set solver options to control the optimization: + - Time limit: `solver.options['TimeLimit'] = [TIME_LIMIT]` + - Optimality gap tolerance: `solver.options['MIPGap'] = 0.0` for exact solution. + - Thread count: `solver.options['Threads'] = [NUM_CORES]` + - Random seed for reproducibility, if supported. + +### Step 2 - Solve and Check Termination Status +- Execute the solve: `results = solver.solve(model, tee=False)`. +- Check the high-level solver status: `status = results.solver.status`. +- Check the detailed termination condition: `term = results.solver.termination_condition`. +- **Prerequisite Check:** A successful solve requires `status == SolverStatus.ok` and `term` in `{TerminationCondition.optimal, TerminationCondition.feasible}`. Do not trust non-zero return codes or infeasible/unknown statuses. + +### Step 3 - Extract and Validate Solution +- If the solve was successful, access the objective value: `obj_val = model.obj()`. +- Extract selected items by evaluating `value(model.x[i]) > 0.5`. +- For verification, compute the actual minimum distance among selected items using the original distance data to ensure it matches `obj_val`. +- Return the solution in a structured format. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo + +model = pyo.ConcreteModel() +# ... (build sets, params, variables, constraints, objective as per Modeling Stage) + +# solve with status / termination checks +solver = pyo.SolverFactory('scip') # Example with SCIP +solver.options['limits/time'] = 30 +solver.options['limits/gap'] = 0.0 + +results = solver.solve(model, tee=False) + +from pyomo.opt import SolverStatus, TerminationCondition +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in (TerminationCondition.optimal, TerminationCondition.feasible): + selected_items = [i for i in model.N if pyo.value(model.x[i]) > 0.5] + min_distance = pyo.value(model.z) + solution = { + "status": "OPTIMAL" if term == TerminationCondition.optimal else "FEASIBLE", + "objective": min_distance, + "selected": selected_items + } +else: + # Do not output pseudo numeric answers when execution fails. + solution = { + "status": str(term), + "objective": None, + "selected": [] + } +print(solution) +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition`, which can lead to extracting invalid solutions from interrupted or infeasible runs. +- Assuming variable values are loaded automatically; after solving, Pyomo automatically loads the solution into the model object. +- Using `tee=True` in production code, which prints extensive solver logs; reserve it for debugging. diff --git a/skills/optskills/skill_library/bipartite_flow_allocation_with_capacity.md b/skills/optskills/skill_library/bipartite_flow_allocation_with_capacity.md new file mode 100644 index 0000000..8ea0144 --- /dev/null +++ b/skills/optskills/skill_library/bipartite_flow_allocation_with_capacity.md @@ -0,0 +1,228 @@ +--- +name: Bipartite Flow Allocation with Capacity +description: | + Model and solve bipartite resource allocation problems with supply-demand balance, individual capacity limits, and linear costs using linear programming. + +--- +# Workflow 1 (Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +This workflow uses Pyomo's abstract modeling syntax to define a structured linear program. It is ideal for problems where data is cleanly separated from model logic, enabling easy modification and reuse. + +### Step 1 - Define Sets and Parameters +- Define two distinct sets: `sources` (e.g., employees, factories) and `destinations` (e.g., projects, warehouses). +- Load or define parameter dictionaries for `supply`, `demand`, `capacity`, and `cost`, ensuring they are indexed appropriately by source, destination, or both. + +### Step 2 - Instantiate Model and Variables +- Instantiate a `ConcreteModel`. +- Define a continuous, non-negative decision variable `x[i, j]` for the flow from source `i` to destination `j`. + +### Step 3 - Formulate Objective and Constraints +- Formulate a linear objective to minimize total cost: `sum(cost[i,j] * x[i,j] for i in sources for j in destinations)`. +- Add supply constraints: total outflow from each source must equal its supply. +- Add demand constraints: total inflow to each destination must equal its demand. +- Add capacity constraints: each individual flow `x[i,j]` must not exceed its upper bound. + +### Formulation Template +```json +{ + "sets": ["sources", "destinations"], + "parameters": [ + "supply[s] for s in sources", + "demand[d] for d in destinations", + "capacity[s, d] for s in sources, d in destinations", + "cost[s, d] for s in sources, d in destinations" + ], + "decision_variables": ["x[s, d] >= 0"], + "objective": { + "sense": "min", + "expression": "sum(cost[s, d] * x[s, d])" + }, + "constraints": [ + "supply_balance[s]: sum(x[s, d] for d in destinations) == supply[s]", + "demand_balance[d]: sum(x[s, d] for s in sources) == demand[d]", + "capacity_limit[s, d]: x[s, d] <= capacity[s, d]" + ] +} +``` + +### Common Pitfalls +- Forgetting to define the domain of variables as `NonNegativeReals`, leading to negative flows. +- Mismatching indices between parameter dictionaries and constraint rules, causing `KeyError`. +- Using `==` for supply/demand constraints when the problem might require `<=` or `>=` for surplus/shortage. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using an efficient open-source LP solver (HiGHS or CBC) with proper configuration and robust solution status checking to ensure a valid result is extracted. + +### Step 1 - Configure and Execute Solver +- Instantiate a solver factory (e.g., `SolverFactory('highs')` or `SolverFactory('cbc')`). +- Set practical solver options like `time_limit` and `threads` for performance control. +- Call `solver.solve(model, tee=False)` to execute. + +### Step 2 - Verify Solution Status +- Check the solver status (`results.solver.status`) is `SolverStatus.ok`. +- Check the termination condition (`results.solver.termination_condition`) is `optimal` or `feasible`. +- If checks fail, print the status and termination condition for diagnostics before proceeding. + +### Step 3 - Extract and Validate Results +- Extract the objective value using `pyo.value(model.obj)`. +- Iterate through the `model.x` variable to retrieve the optimal flows. +- Optionally, implement a post-solve verification by recalculating constraint sums to ensure they match supply and demand parameters. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# ... (Model built as per Modeling Stage) + +# Solve +solver = pyo.SolverFactory('highs') # or 'cbc' +solver.options['time_limit'] = 30 +results = solver.solve(model, tee=False) + +# Check results +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + objective_value = float(pyo.value(model.obj)) + # Extract solution + solution = {(i, j): pyo.value(model.x[i, j]) for i in model.sources for j in model.destinations} + print(f"Optimal cost: {objective_value}") + # Add verification logic here +else: + print(f"Solver failed. Status: {status}, Termination: {term}") +``` + +### Common Pitfalls +- Not checking both solver status *and* termination condition, potentially extracting results from an infeasible or error state. +- Assuming the solver variable `model.x` is automatically populated; always use `pyo.value()` to access the solution. +- Setting `tee=True` in production scripts, which can clutter logs with excessive solver output. + +# Workflow 2 (Google OR-Tools) + +## Modeling stage + +### Strategy Overview +This workflow uses Google OR-Tools' imperative API to build the model step-by-step. It is well-suited for rapid prototyping and integration into applications where a more procedural coding style is preferred. + +### Step 1 - Initialize Solver and Data Structures +- Create a linear solver instance (e.g., `pywraplp.Solver.CreateSolver('GLOP')`). +- Check the solver was successfully instantiated (`if solver is None:`). +- Store parameters in lists or dictionaries for easy access. + +### Step 2 - Create Variables with Bounds +- Create decision variables `x[i][j]` using `solver.NumVar(lower_bound, upper_bound, name)`. +- Directly encode the `capacity[i][j]` as the variable's upper bound during creation, setting the lower bound to 0. + +### Step 3 - Build Constraints and Objective +- For each source `i`, create a constraint `sum(x[i][j] for j) == supply[i]` using `solver.Add()`. +- For each destination `j`, create a constraint `sum(x[i][j] for i) == demand[j]`. +- Set the objective to minimize `sum(cost[i][j] * x[i][j])` using `solver.Minimize()`. + +### Formulation Template +```json +{ + "sets": ["sources", "destinations"], + "parameters": [ + "supply[i] for i in sources", + "demand[j] for j in destinations", + "capacity[i][j] for i in sources, j in destinations", + "cost[i][j] for i in sources, j in destinations" + ], + "decision_variables": ["x[i][j] where 0 <= x[i][j] <= capacity[i][j]"], + "objective": { + "sense": "min", + "expression": "sum(cost[i][j] * x[i][j])" + }, + "constraints": [ + "for each i: sum_j x[i][j] == supply[i]", + "for each j: sum_i x[i][j] == demand[j]" + ] +} +``` + +### Common Pitfalls +- Using `solver.IntVar` instead of `solver.NumVar` for continuous flows, unnecessarily making the problem an integer program. +- Adding constraints in nested loops without clearing or reusing constraint objects, which is inefficient but not an error in OR-Tools. +- Not naming variables meaningfully, making debugging difficult for larger models. + +## Solving stage + +### Strategy Overview +Solve the model using OR-Tools' solver object, check the result status, and extract the solution. The imperative style makes solution extraction straightforward via the variable objects. + +### Step 1 - Invoke Solver and Check Status +- Call `solver.Solve()`. +- Check the result status against `pywraplp.Solver.OPTIMAL` or `FEASIBLE`. Handle other statuses (INFEASIBLE, UNBOUNDED) with appropriate error messages. + +### Step 2 - Extract Solution and Verify +- If optimal/feasible, retrieve the objective value via `solver.Objective().Value()`. +- Iterate through all variable objects to get their solution values with `var.solution_value()`. +- Implement an optional verification loop to sum flows and compare against original supply/demand parameters. + +### Step 3 - Report Results and Handle Failures +- Print a summary of the objective value and key assignments. +- In case of failure, use the solver status to provide a clear diagnostic message (e.g., "Model is infeasible; check supply/demand totals"). + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# ... (Data structures defined: supply, demand, capacity, cost) + +# Initialize solver +solver = pywraplp.Solver.CreateSolver('GLOP') +if solver is None: + raise Exception('Solver backend not available.') + +# Create variables +x = {} +for i in sources: + for j in destinations: + x[i, j] = solver.NumVar(0, capacity[i][j], f'x_{i}_{j}') + +# Add supply constraints +for i in sources: + ct = solver.Constraint(supply[i], supply[i]) + for j in destinations: + ct.SetCoefficient(x[i, j], 1) + +# Add demand constraints +for j in destinations: + ct = solver.Constraint(demand[j], demand[j]) + for i in sources: + ct.SetCoefficient(x[i, j], 1) + +# Set objective +objective = solver.Objective() +for i in sources: + for j in destinations: + objective.SetCoefficient(x[i, j], cost[i][j]) +objective.SetMinimization() + +# Solve +status = solver.Solve() + +# Check and extract +if status in (pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE): + print(f'Optimal cost: {solver.Objective().Value()}') + # Extract flows + for i in sources: + for j in destinations: + flow_val = x[i, j].solution_value() + if flow_val > 1e-6: # Print non-zero assignments + print(f'{i} -> {j}: {flow_val}') +else: + print(f'No optimal solution found. Status: {status}') +``` + +### Common Pitfalls +- Confusing `solver.Solve()` return status codes with model feasibility; `OPTIMAL` and `FEASIBLE` are both success codes. +- Forgetting to check `if solver is None` after creation, leading to runtime errors if the requested backend is unavailable. +- Not using a tolerance (e.g., `1e-6`) when checking for non-zero flows, as solvers may return very small numerical values. diff --git a/skills/optskills/skill_library/bipartiteassignmentflow.md b/skills/optskills/skill_library/bipartiteassignmentflow.md new file mode 100644 index 0000000..db5627e --- /dev/null +++ b/skills/optskills/skill_library/bipartiteassignmentflow.md @@ -0,0 +1,217 @@ +--- +name: BipartiteAssignmentFlow +description: | + Model and solve linear bipartite assignment problems with flow quantities, demand satisfaction, and unconstrained supply to maximize total profit or minimize total cost. +--- + +# Workflow 1 (Linear Programming with OR-Tools) + +## Modeling stage + +### Strategy Overview +This workflow models the problem as a linear program using Google's OR-Tools `pywraplp` API. It is ideal for prototyping and solving pure LP formulations with a clean, imperative modeling style. The solver backend (GLOP) is efficient for continuous problems. + +### Step 1 - Define Data Structures +- Organize problem data into lists or dictionaries for clear indexing. Use `profit[i][j]` for the per-unit profit from source `i` to sink `j` and `demand[j]` for the required flow to each sink. +- Use `solver.infinity()` to represent unlimited upper bounds, reflecting the unconstrained supply capacity. + +### Step 2 - Create Decision Variables +- Instantiate a continuous, non-negative decision variable `x[i][j]` for each source-sink pair using `solver.NumVar(lb, ub, name)`. +- Set the lower bound (`lb`) to `0` and the upper bound (`ub`) to `solver.infinity()` to enforce non-negativity and allow any flow quantity. + +### Step 3 - Formulate the Objective +- Build a linear expression by summing `profit[i][j] * x[i][j]` over all variable indices. +- Set the objective sense to maximization using `solver.Maximize(expression)` or `solver.Minimize(expression)` for cost minimization. + +### Step 4 - Add Demand Satisfaction Constraints +- For each sink `j`, create an equality constraint with `solver.Constraint(rhs, rhs)`, where `rhs` is `demand[j]`. +- Iterate over all sources `i` and set the coefficient of `x[i][j]` to `1.0` using `constraint.SetCoefficient(var, coeff)` to enforce `sum_i x[i][j] == demand[j]`. + +### Formulation Template +```json +{ + "sets": [ + "sources: list of source identifiers", + "sinks: list of sink identifiers" + ], + "parameters": [ + "profit[sources][sinks]: per-unit profit matrix", + "demand[sinks]: required flow for each sink" + ], + "decision_variables": [ + "x[sources][sinks]: continuous, >= 0" + ], + "objective": { + "sense": "max", + "expression": "sum_{i in sources, j in sinks} profit[i][j] * x[i][j]" + }, + "constraints": [ + "demand_satisfaction: for each j in sinks, sum_{i in sources} x[i][j] == demand[j]" + ] +} +``` + +### Common Pitfalls +- Forgetting to set the upper bound to infinity, which incorrectly imposes a default bound of 1.0 and limits flow. +- Building constraints with incorrect coefficient signs or values, leading to infeasible or suboptimal solutions. +- Not using a tolerance (e.g., `1e-6`) when checking variable values, which may misclassify near-zero flows as active. + +## Solving stage + +### Strategy Overview +The solving stage uses the OR-Tools wrapper to invoke the GLOP solver, configures runtime limits, rigorously checks the solution status, and extracts a clean, validated result. + +### Step 1 - Configure and Execute the Solver +- Create the solver instance with `pywraplp.Solver.CreateSolver('GLOP')`. +- Set a reasonable time limit using `solver.SetTimeLimit(milliseconds)` to prevent excessive runtime. +- Call `solver.Solve()` to initiate the optimization. + +### Step 2 - Verify Solution Status +- Check the solver's return status using `status = solver.Solve()`. +- Proceed only if `status` is `pywraplp.Solver.OPTIMAL` or `pywraplp.Solver.FEASIBLE`. Handle other statuses (e.g., `INFEASIBLE`, `UNBOUNDED`) with appropriate error messages. + +### Step 3 - Extract and Validate Results +- Retrieve the objective value via `solver.Objective().Value()`. +- Iterate through all decision variables, using `.solution_value()` to get their values. Filter and store only those with values exceeding a small tolerance (e.g., `> 1e-6`). +- Optionally, perform a verification step by recomputing total flow per sink from the extracted solution and comparing it to the original demand. + +### Step 4 - Package and Output Results +- Structure the results into a dictionary or JSON payload containing the status, objective value, and a dictionary of non-zero flows. +- Include a simple constraint satisfaction report for debugging purposes. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('GLOP') +# ... (variable and constraint creation as per modeling stage) +solver.SetTimeLimit(30000) # 30-second limit + +# solve with status / termination checks +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + obj_val = solver.Objective().Value() + solution = {} + for i in sources: + for j in sinks: + val = x[i][j].solution_value() + if val > 1e-6: + solution[(i, j)] = val + # Package results... +else: + # Handle infeasible/unbounded/time limit status + print(f"Solver did not find an optimal solution. Status: {status}") +``` + +### Common Pitfalls +- Assuming the solver always returns `OPTIMAL`; failing to check for `FEASIBLE` or other statuses can lead to runtime errors when extracting values. +- Not applying a tolerance when filtering the solution, resulting in verbose output with many near-zero values. +- Omitting the time limit configuration, which may cause the process to hang on large or numerically challenging instances. + +# Workflow 2 (Structured Modeling with Pyomo) + +## Modeling stage + +### Strategy Overview +This workflow uses Pyomo's abstract or concrete modeling to define the problem in a declarative, solver-agnostic manner. It separates model specification from solver execution, promoting reusability and integration with advanced features like piecewise linear functions or custom callbacks. + +### Step 1 - Define Abstract Sets and Parameters +- Declare Pyomo `Set` objects for `model.sources` and `model.sinks`. +- Declare `Param` objects for `model.profit` (indexed by both sets) and `model.demand` (indexed by sinks). Use `initialize` with a dictionary or rule to populate data. + +### Step 2 - Declare Decision Variables +- Define a Pyomo `Var` object `model.x`, indexed over `model.sources` and `model.sinks`. +- Set the domain to `pyo.NonNegativeReals` to enforce non-negativity. No upper bound is specified, reflecting unconstrained supply. + +### Step 3 - Construct the Objective Function +- Define a `pyo.Objective` rule that sums `model.profit[i, j] * model.x[i, j]` over all indices. +- Set the sense to `pyo.maximize` for profit maximization. + +### Step 4 - Implement Demand Constraints via Rules +- Define a `pyo.Constraint` object indexed by `model.sinks`. +- For each sink `j`, the constraint rule should return the expression `sum(model.x[i, j] for i in model.sources) == model.demand[j]`. + +### Formulation Template +```json +{ + "sets": [ + "sources: Pyomo Set", + "sinks: Pyomo Set" + ], + "parameters": [ + "profit[sources, sinks]: Pyomo Param", + "demand[sinks]: Pyomo Param" + ], + "decision_variables": [ + "x[sources, sinks]: Pyomo Var, domain=NonNegativeReals" + ], + "objective": { + "sense": "maximize", + "expression": "sum(profit[i,j] * x[i,j] for i in sources for j in sinks)" + }, + "constraints": [ + "demand_satisfaction: for each j in sinks, sum(x[i,j] for i in sources) == demand[j]" + ] +} +``` + +### Common Pitfalls +- Confusing 1-based and 0-based indexing when initializing parameters from Python data structures, leading to `KeyError`. +- Defining constraint rules that modify global state or have side effects, which can cause unpredictable behavior during model construction. +- Forgetting to deactivate the default `sense` on the objective, which defaults to minimization. + +## Solving stage + +### Strategy Overview +The solving stage uses Pyomo's `SolverFactory` to interface with a backend solver (e.g., HiGHS, CBC). It emphasizes robust status checking, solution extraction through the model object, and post-solution validation. + +### Step 1 - Select and Configure the Solver +- Instantiate a solver object using `SolverFactory('solver_name')`, e.g., `'highs'` or `'cbc'`. +- Set solver options such as `time_limit` and `threads` for performance control, using `solver.options['key'] = value`. + +### Step 2 - Solve and Check Termination Conditions +- Execute `results = solver.solve(model, tee=False)`. +- Inspect `results.solver.status` (should be `SolverStatus.ok`) and `results.solver.termination_condition` (should be `optimal` or `feasible`). Handle other conditions like `infeasible` or `unbounded` appropriately. + +### Step 3 - Extract and Process the Solution +- Retrieve the objective value via `pyo.value(model.obj)` (if the objective is named `obj`). +- Iterate over `model.x` to access variable values using `pyo.value(model.x[i, j])` or `model.x[i, j].value`. +- Apply a tolerance to filter out near-zero flows and store the non-zero allocations in a clean dictionary. + +### Step 4 - Validate and Report +- Programmatically verify that the extracted solution satisfies the demand constraints within a small numerical tolerance. +- Package the final output, including solver status, objective value, and the filtered solution dictionary. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo +model = pyo.ConcreteModel() +model.sources = pyo.Set(initialize=sources_list) +model.sinks = pyo.Set(initialize=sinks_list) +# ... (parameter, variable, objective, and constraint definitions as per modeling stage) + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = 30 +results = solver.solve(model) + +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in (pyo.TerminationCondition.optimal, + pyo.TerminationCondition.feasible)): + obj_val = pyo.value(model.obj) + solution = {} + for i in model.sources: + for j in model.sinks: + val = pyo.value(model.x[i, j]) + if val > 1e-6: + solution[(i, j)] = val + # Package results... +else: + # Handle unsuccessful termination + print(f"Solver failed: {results.solver.termination_condition}") +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition`; a status of `ok` with a termination of `maxTimeLimit` indicates a suboptimal solution. +- Directly printing the entire model variable dictionary, which can be overwhelming for large problems. +- Assuming the solver updates the model in-place for all solver interfaces; some require explicit loading of the solution. diff --git a/skills/optskills/skill_library/capacitated_arc_routing_problem_carp_solver.md b/skills/optskills/skill_library/capacitated_arc_routing_problem_carp_solver.md new file mode 100644 index 0000000..9f50d1e --- /dev/null +++ b/skills/optskills/skill_library/capacitated_arc_routing_problem_carp_solver.md @@ -0,0 +1,481 @@ +--- +name: Capacitated Arc Routing Problem (CARP) Solver +description: | + Models and solves the Capacitated Arc Routing Problem using either a MILP-based task sequencing formulation or a state-based MILP formulation, with solver-specific extraction and validation. +--- + +# Workflow 1 (Task-Sequencing MILP with OR-Tools CP-SAT) + +## Modeling stage + +### Strategy Overview +Decompose arc routing into task sequencing by treating each required edge as a task with two possible traversal directions. Use binary assignment variables for vehicle-task-direction combinations, predecessor/successor variables for route continuity, and flow conservation constraints to enforce route structure. + +### Step 1 - Define Task and Direction Sets +- For each required edge (u,v), define two tasks: direction 0 (enter at u, exit at v) and direction 1 (enter at v, exit at u). +- Precompute `start_node[task, direction]` and `end_node[task, direction]` for all task-direction pairs. + +### Step 2 - Create Binary Assignment Variables +- Define `x[vehicle, task, direction]` as a binary variable indicating whether a vehicle services a given task in a specific direction. +- Define `first[vehicle, task, direction]` and `last[vehicle, task, direction]` as binary variables marking the first and last serviced tasks for each vehicle. +- Define `y[vehicle, task1, dir1, task2, dir2]` as a binary variable indicating that vehicle services task1 in dir1 immediately before task2 in dir2. + +### Step 3 - Enforce Flow Conservation and Route Structure +- For each (vehicle, task, direction): `first + sum(incoming y) == x` and `last + sum(outgoing y) == x`. +- **Prerequisite Check:** Enforce that each vehicle has **exactly one** first task and **exactly one** last task **if and only if** it services at least one task. Use auxiliary binary variable `vehicle_used[vehicle]` and constraints: + - `sum(task, direction) x[vehicle, task, direction] >= 1 => vehicle_used[vehicle] == 1` + - `sum(task, direction) x[vehicle, task, direction] == 0 => vehicle_used[vehicle] == 0` + - `sum(task, direction) first[vehicle, task, direction] == vehicle_used[vehicle]` + - `sum(task, direction) last[vehicle, task, direction] == vehicle_used[vehicle]` +- This prevents cycles without designated start/end points and ensures a valid route structure. + +### Step 4 - Add Capacity and Coverage Constraints +- Capacity: For each vehicle, `sum(demand[task] * x[vehicle, task, direction]) <= capacity`. +- Coverage: For each task, `sum(x[vehicle, task, direction] over vehicles and directions) == 1`. + +### Step 5 - Prevent Invalid Transitions +- Add constraint `y[vehicle, task, dir1, task, dir2] == 0` to prevent self-loops where a task immediately follows itself. + +### Step 6 - Formulate Objective +- Minimize total travel distance = sum over vehicles of (distance from depot to first task's start node) + (service traversal distance for each assigned task) + (deadhead distance between consecutive tasks) + (distance from last task's end node back to depot). + +### Formulation Template +```json +{ + "sets": ["VEHICLES", "TASKS", "DIRECTIONS"], + "parameters": ["demand[TASKS]", "capacity[VEHICLES]", "start_node[TASKS, DIRECTIONS]", "end_node[TASKS, DIRECTIONS]", "dist_matrix[NODES, NODES]"], + "decision_variables": [ + "x[VEHICLES, TASKS, DIRECTIONS] binary", + "first[VEHICLES, TASKS, DIRECTIONS] binary", + "last[VEHICLES, TASKS, DIRECTIONS] binary", + "y[VEHICLES, TASKS, DIRECTIONS, TASKS, DIRECTIONS] binary", + "vehicle_used[VEHICLES] binary" + ], + "objective": { + "sense": "min", + "expression": "sum(v in VEHICLES, t in TASKS, d in DIRECTIONS) (dist_matrix[DEPOT, start_node[t,d]] * first[v,t,d] + dist_matrix[end_node[t,d], DEPOT] * last[v,t,d] + dist_matrix[end_node[t,d], start_node[t2,d2]] * y[v,t,d,t2,d2] + dist_matrix[start_node[t,d], end_node[t,d]] * x[v,t,d])" + }, + "constraints": [ + "forall v in VEHICLES, t in TASKS, d in DIRECTIONS: first[v,t,d] + sum(t2,d2) y[v,t2,d2,t,d] == x[v,t,d]", + "forall v in VEHICLES, t in TASKS, d in DIRECTIONS: last[v,t,d] + sum(t2,d2) y[v,t,d,t2,d2] == x[v,t,d]", + "forall v in VEHICLES: sum(t,d) demand[t] * x[v,t,d] <= capacity[v]", + "forall t in TASKS: sum(v,d) x[v,t,d] == 1", + "forall v in VEHICLES, t in TASKS, d1 in DIRECTIONS, d2 in DIRECTIONS: y[v,t,d1,t,d2] == 0", + "forall v in VEHICLES: sum(t,d) x[v,t,d] >= 1 -> vehicle_used[v] == 1", + "forall v in VEHICLES: sum(t,d) x[v,t,d] == 0 -> vehicle_used[v] == 0", + "forall v in VEHICLES: sum(t,d) first[v,t,d] == vehicle_used[v]", + "forall v in VEHICLES: sum(t,d) last[v,t,d] == vehicle_used[v]" + ] +} +``` + +### Common Pitfalls +- Forgetting to include all four distance components (depot-to-first, service, deadhead, last-to-depot) in the objective. +- Using incomplete flow conservation without enforcing exactly one first/last per used vehicle, leading to disconnected route fragments or cycles. +- Using `itertools.product` for enumeration instead of solver variables for realistic instance sizes. +- Not linking `vehicle_used` correctly to assignment variables, causing invalid route extraction. + +## Solving stage + +### Strategy Overview +Implement the model using OR-Tools CP-SAT solver with binary integer variables. Configure solver parameters for time limits and optimality gaps, then extract and validate the solution by reconstructing routes from the decision variables. + +### Step 1 - Initialize Solver and Variables +- Create a `CpModel()` instance. +- Add all binary decision variables (`x`, `first`, `last`, `y`, `vehicle_used`) using `model.NewBoolVar()`. + +### Step 2 - Add Constraints +- Add flow conservation and route structure constraints using `model.Add()`. +- Add capacity constraints using `model.Add(sum(...) <= capacity)`. +- Add coverage constraints using `model.Add(sum(...) == 1)`. +- Add self-loop prevention constraints. +- **Prerequisite Check:** Add vehicle usage linking constraints using `model.Add(sum(x) >= 1).OnlyEnforceIf(vehicle_used)` and `model.Add(sum(x) == 0).OnlyEnforceIf(vehicle_used.Not())`. Enforce `sum(first) == vehicle_used` and `sum(last) == vehicle_used`. + +### Step 3 - Set Objective +- Build the linear expression for total travel distance. +- Use `model.Minimize(expression)` to set the objective. + +### Step 4 - Configure and Solve +- Create a `CpSolver()` instance. +- Set parameters: `solver.parameters.max_time_in_seconds = [TIME_LIMIT]`, `solver.parameters.relative_gap_limit = [GAP]`. +- Call `status = solver.Solve(model)`. + +### Step 5 - Extract and Validate Solution +- **Early Warning:** Check `status == cp_model.OPTIMAL` or `status == cp_model.FEASIBLE`. If not, return status immediately. **Do not proceed with extraction on infeasible or unknown status.** +- For each vehicle, verify `vehicle_used` is consistent with assigned tasks. If `vehicle_used` is 1, ensure exactly one `first` and one `last` variable is 1. +- For each vehicle with `vehicle_used == 1`, iterate over tasks to find `first` variable with value 1, then follow `y` variables to reconstruct the sequence. +- **Fallback Guidance:** If a vehicle has assigned tasks but no clear start/end sequence, flag the solution as potentially infeasible and re-check constraint satisfaction. +- Compute actual route distance from the node sequence and compare with objective value to validate. + +### Code Usage +```python +from ortools.sat.python import cp_model + +def build_carp_model(vehicles, tasks, directions, demand, capacity, start_node, end_node, dist_matrix, depot): + model = cp_model.CpModel() + + # Decision variables + x = {} + first = {} + last = {} + y = {} + vehicle_used = {} + for v in vehicles: + vehicle_used[v] = model.NewBoolVar(f'used_{v}') + for t in tasks: + for d in directions: + x[v,t,d] = model.NewBoolVar(f'x_{v}_{t}_{d}') + first[v,t,d] = model.NewBoolVar(f'first_{v}_{t}_{d}') + last[v,t,d] = model.NewBoolVar(f'last_{v}_{t}_{d}') + for t2 in tasks: + for d2 in directions: + y[v,t,d,t2,d2] = model.NewBoolVar(f'y_{v}_{t}_{d}_{t2}_{d2}') + + # Flow conservation + for v in vehicles: + for t in tasks: + for d in directions: + incoming = sum(y[v,t2,d2,t,d] for t2 in tasks for d2 in directions) + outgoing = sum(y[v,t,d,t2,d2] for t2 in tasks for d2 in directions) + model.Add(first[v,t,d] + incoming == x[v,t,d]) + model.Add(last[v,t,d] + outgoing == x[v,t,d]) + + # Vehicle usage linking + for v in vehicles: + total_assigned = sum(x[v,t,d] for t in tasks for d in directions) + model.Add(total_assigned >= 1).OnlyEnforceIf(vehicle_used[v]) + model.Add(total_assigned == 0).OnlyEnforceIf(vehicle_used[v].Not()) + model.Add(sum(first[v,t,d] for t in tasks for d in directions) == vehicle_used[v]) + model.Add(sum(last[v,t,d] for t in tasks for d in directions) == vehicle_used[v]) + + # Capacity constraints + for v in vehicles: + model.Add(sum(demand[t] * x[v,t,d] for t in tasks for d in directions) <= capacity[v]) + + # Coverage constraints + for t in tasks: + model.Add(sum(x[v,t,d] for v in vehicles for d in directions) == 1) + + # Self-loop prevention + for v in vehicles: + for t in tasks: + for d1 in directions: + for d2 in directions: + model.Add(y[v,t,d1,t,d2] == 0) + + # Objective + obj_terms = [] + for v in vehicles: + for t in tasks: + for d in directions: + obj_terms.append(dist_matrix[depot][start_node[t,d]] * first[v,t,d]) + obj_terms.append(dist_matrix[end_node[t,d]][depot] * last[v,t,d]) + obj_terms.append(dist_matrix[start_node[t,d]][end_node[t,d]] * x[v,t,d]) + for t2 in tasks: + for d2 in directions: + obj_terms.append(dist_matrix[end_node[t,d]][start_node[t2,d2]] * y[v,t,d,t2,d2]) + model.Minimize(sum(obj_terms)) + + return model, x, first, last, y, vehicle_used + +def solve_carp(model, x, first, last, y, vehicle_used, vehicles, tasks, directions, time_limit=60): + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = time_limit + solver.parameters.relative_gap_limit = 0.0 + + status = solver.Solve(model) + + # Early failure detection: do not extract on non-feasible status + if status not in (cp_model.OPTIMAL, cp_model.FEASIBLE): + return {"status": "INFEASIBLE" if status == cp_model.INFEASIBLE else "UNKNOWN", "objective": None} + + result = {"status": "OPTIMAL" if status == cp_model.OPTIMAL else "FEASIBLE", + "objective": solver.ObjectiveValue()} + + # Extract routes + routes = {} + for v in vehicles: + if solver.Value(vehicle_used[v]) == 1: + route = [] + # Find first task + for t in tasks: + for d in directions: + if solver.Value(first[v,t,d]) == 1: + route.append((t,d)) + break + if route: + break + + # Follow sequence + while route: + current_t, current_d = route[-1] + next_found = False + for t2 in tasks: + for d2 in directions: + if solver.Value(y[v,current_t,current_d,t2,d2]) == 1: + route.append((t2,d2)) + next_found = True + break + if next_found: + break + if not next_found: + break + + routes[v] = route + else: + routes[v] = [] + + result["routes"] = routes + return result + +# Usage +# model, x, first, last, y, vehicle_used = build_carp_model(vehicles, tasks, directions, demand, capacity, start_node, end_node, dist_matrix, depot) +# result = solve_carp(model, x, first, last, y, vehicle_used, vehicles, tasks, directions) +# print(f"RESULT_JSON:{json.dumps(result)}") +``` + +### Common Pitfalls +- Not checking solver status before extracting variable values, leading to runtime errors. +- Assuming all vehicles must be used; the model naturally allows vehicles with no assigned tasks. +- Forgetting to set a time limit, causing the solver to run indefinitely on larger instances. +- Extracting routes without verifying `vehicle_used` and start/end consistency, risking incorrect reconstruction. + +# Workflow 2 (State-Based MILP with HiGHS) + +## Modeling stage + +### Strategy Overview +Model the CARP as a state-based MILP where each required edge has two service states (orientations). Use binary assignment and sequencing variables with Miller-Tucker-Zemlin (MTZ) subtour elimination constraints to ensure valid routes. + +### Step 1 - Define Service States +- For each required edge (u,v) with demand d, define state 0 (enter at u, exit at v) and state 1 (enter at v, exit at u). +- Precompute entry and exit nodes for each state. + +### Step 2 - Create Binary Assignment Variables +- Define `x[vehicle, edge, state]` = 1 if vehicle services edge in that state. +- Define `start[vehicle, edge, state]` and `end[vehicle, edge, state]` as binary variables marking first and last serviced edges. + +### Step 3 - Create Sequencing Variables +- Define `z[vehicle, edge1, state1, edge2, state2]` = 1 if vehicle services edge1 in state1 immediately before edge2 in state2. + +### Step 4 - Enforce Flow Conservation and Route Structure +- For each (vehicle, edge, state): `start + sum(incoming z) == x` and `end + sum(outgoing z) == x`. +- **Prerequisite Check:** Enforce that each vehicle has exactly one start and one end edge **if and only if** it services at least one edge. Use auxiliary binary variable `vehicle_active[vehicle]` and constraints: + - `sum(edge, state) x[vehicle, edge, state] >= 1 => vehicle_active[vehicle] == 1` + - `sum(edge, state) x[vehicle, edge, state] == 0 => vehicle_active[vehicle] == 0` + - `sum(edge, state) start[vehicle, edge, state] == vehicle_active[vehicle]` + - `sum(edge, state) end[vehicle, edge, state] == vehicle_active[vehicle]` + +### Step 5 - Add Capacity and Coverage Constraints +- Capacity: For each vehicle, `sum(demand[edge] * x[vehicle, edge, state]) <= capacity`. +- Coverage: For each edge, `sum(x[vehicle, edge, state] over vehicles and states) == 1`. + +### Step 6 - Add MTZ Subtour Elimination +- Define continuous position variables `u[vehicle, edge, state]` bounded between 0 and (number of edges + 1). +- For each `z[vehicle, e1, s1, e2, s2]` = 1, enforce `u[vehicle, e2, s2] >= u[vehicle, e1, s1] + 1` using a big-M formulation: `u[e2,s2] >= u[e1,s1] + 1 - M*(1 - z[e1,s1,e2,s2])` with `M = len(edges) + 1`. + +### Step 7 - Formulate Objective +- Minimize total travel distance = sum over vehicles of (distance from depot to first edge's entry node) + (service traversal cost for each edge) + (deadhead cost between consecutive edges) + (distance from last edge's exit node back to depot). + +### Formulation Template +```json +{ + "sets": ["VEHICLES", "EDGES", "STATES"], + "parameters": ["demand[EDGES]", "capacity[VEHICLES]", "entry_node[EDGES, STATES]", "exit_node[EDGES, STATES]", "dist_matrix[NODES, NODES]"], + "decision_variables": [ + "x[VEHICLES, EDGES, STATES] binary", + "start[VEHICLES, EDGES, STATES] binary", + "end[VEHICLES, EDGES, STATES] binary", + "z[VEHICLES, EDGES, STATES, EDGES, STATES] binary", + "u[VEHICLES, EDGES, STATES] continuous", + "vehicle_active[VEHICLES] binary" + ], + "objective": { + "sense": "min", + "expression": "sum(v in VEHICLES, e in EDGES, s in STATES) (dist_matrix[DEPOT, entry_node[e,s]] * start[v,e,s] + dist_matrix[exit_node[e,s], DEPOT] * end[v,e,s] + dist_matrix[entry_node[e,s], exit_node[e,s]] * x[v,e,s]) + sum(v in VEHICLES, e1 in EDGES, s1 in STATES, e2 in EDGES, s2 in STATES) dist_matrix[exit_node[e1,s1], entry_node[e2,s2]] * z[v,e1,s1,e2,s2]" + }, + "constraints": [ + "forall v in VEHICLES, e in EDGES, s in STATES: start[v,e,s] + sum(e2,s2) z[v,e2,s2,e,s] == x[v,e,s]", + "forall v in VEHICLES, e in EDGES, s in STATES: end[v,e,s] + sum(e2,s2) z[v,e,s,e2,s2] == x[v,e,s]", + "forall v in VEHICLES: sum(e,s) demand[e] * x[v,e,s] <= capacity[v]", + "forall e in EDGES: sum(v,s) x[v,e,s] == 1", + "forall v in VEHICLES, e1 in EDGES, s1 in STATES, e2 in EDGES, s2 in STATES: u[v,e2,s2] >= u[v,e1,s1] + 1 - (len(EDGES)+1) * (1 - z[v,e1,s1,e2,s2])", + "forall v in VEHICLES, e in EDGES, s in STATES: 0 <= u[v,e,s] <= len(EDGES) + 1", + "forall v in VEHICLES: sum(e,s) x[v,e,s] >= 1 -> vehicle_active[v] == 1", + "forall v in VEHICLES: sum(e,s) x[v,e,s] == 0 -> vehicle_active[v] == 0", + "forall v in VEHICLES: sum(e,s) start[v,e,s] == vehicle_active[v]", + "forall v in VEHICLES: sum(e,s) end[v,e,s] == vehicle_active[v]" + ] +} +``` + +### Common Pitfalls +- Not bounding MTZ position variables, leading to unboundedness or numerical issues. +- Forgetting to include the service traversal distance in the objective (distance from entry to exit node for each assigned edge). +- Using a big-M value that is too small, causing MTZ constraints to be ineffective for subtour elimination. +- Not enforcing the link between vehicle activity and start/end variables, allowing cycles without designated start/end. + +## Solving stage + +### Strategy Overview +Implement the model using HiGHS solver via Pyomo or directly with Python-MIP. Configure solver parameters for time limits and optimality gaps, then extract and validate the solution by reconstructing routes from the decision variables. + +### Step 1 - Initialize Model and Variables +- Create a `Model()` instance from Python-MIP or Pyomo. +- Add all binary decision variables (`x`, `start`, `end`, `z`, `vehicle_active`) and continuous variables (`u`). + +### Step 2 - Add Constraints +- Add flow conservation and route structure constraints. +- Add capacity and coverage constraints. +- Add MTZ subtour elimination constraints using big-M formulation with `M = len(edges) + 1`. +- **Prerequisite Check:** Add vehicle activity linking constraints using indicator constraints or big-M. + +### Step 3 - Set Objective +- Build the linear expression for total travel distance. +- Set the objective to minimize. + +### Step 4 - Configure and Solve +- Set solver parameters: `model.optimize(max_seconds=[TIME_LIMIT], mip_rel_gap=[GAP])`. +- Check solver status after optimization. + +### Step 5 - Extract and Validate Solution +- **Early Warning:** Verify `status == SolverStatus.ok` and `termination_condition` is `optimal` or `feasible`. If not, return status immediately. **Do not output pseudo numeric answers when execution fails.** +- For each vehicle, check `vehicle_active` consistency. If active, ensure exactly one `start` and one `end` variable is 1. +- For each active vehicle, find the start edge-state pair, then follow `z` variables to reconstruct the sequence. +- **Fallback Guidance:** If a vehicle has assigned edges but no clear start/end sequence, flag the solution as potentially infeasible and re-check constraint satisfaction. +- Manually verify capacity constraints and coverage. +- Recompute total distance from the solution to validate the objective value. + +### Code Usage +```python +from mip import Model, BINARY, CONTINUOUS, minimize, xsum, OptimizationStatus + +def build_carp_model_mip(vehicles, edges, states, demand, capacity, entry_node, exit_node, dist_matrix, depot): + model = Model("CARP") + + # Decision variables + x = {} + start = {} + end = {} + z = {} + u = {} + vehicle_active = {} + for v in vehicles: + vehicle_active[v] = model.add_var(var_type=BINARY, name=f'active_{v}') + for e in edges: + for s in states: + x[v,e,s] = model.add_var(var_type=BINARY, name=f'x_{v}_{e}_{s}') + start[v,e,s] = model.add_var(var_type=BINARY, name=f'start_{v}_{e}_{s}') + end[v,e,s] = model.add_var(var_type=BINARY, name=f'end_{v}_{e}_{s}') + u[v,e,s] = model.add_var(lb=0, ub=len(edges)+1, name=f'u_{v}_{e}_{s}') + for e2 in edges: + for s2 in states: + z[v,e,s,e2,s2] = model.add_var(var_type=BINARY, name=f'z_{v}_{e}_{s}_{e2}_{s2}') + + # Flow conservation + for v in vehicles: + for e in edges: + for s in states: + incoming = xsum(z[v,e2,s2,e,s] for e2 in edges for s2 in states) + outgoing = xsum(z[v,e,s,e2,s2] for e2 in edges for s2 in states) + model += start[v,e,s] + incoming == x[v,e,s] + model += end[v,e,s] + outgoing == x[v,e,s] + + # Vehicle activity linking + for v in vehicles: + total_assigned = xsum(x[v,e,s] for e in edges for s in states) + model += total_assigned >= 1 - (len(edges)*2) * (1 - vehicle_active[v]) + model += total_assigned <= (len(edges)*2) * vehicle_active[v] + model += xsum(start[v,e,s] for e in edges for s in states) == vehicle_active[v] + model += xsum(end[v,e,s] for e in edges for s in states) == vehicle_active[v] + + # Capacity constraints + for v in vehicles: + model += xsum(demand[e] * x[v,e,s] for e in edges for s in states) <= capacity[v] + + # Coverage constraints + for e in edges: + model += xsum(x[v,e,s] for v in vehicles for s in states) == 1 + + # MTZ subtour elimination + M = len(edges) + 1 + for v in vehicles: + for e1 in edges: + for s1 in states: + for e2 in edges: + for s2 in states: + if e1 != e2 or s1 != s2: + model += u[v,e2,s2] >= u[v,e1,s1] + 1 - M * (1 - z[v,e1,s1,e2,s2]) + + # Objective + obj = xsum( + dist_matrix[depot][entry_node[e,s]] * start[v,e,s] + + dist_matrix[exit_node[e,s]][depot] * end[v,e,s] + + dist_matrix[entry_node[e,s]][exit_node[e,s]] * x[v,e,s] + for v in vehicles for e in edges for s in states + ) + obj += xsum( + dist_matrix[exit_node[e1,s1]][entry_node[e2,s2]] * z[v,e1,s1,e2,s2] + for v in vehicles for e1 in edges for s1 in states for e2 in edges for s2 in states + ) + model.objective = minimize(obj) + + return model, x, start, end, z, vehicle_active + +def solve_carp_mip(model, x, start, end, z, vehicle_active, vehicles, edges, states, time_limit=60): + model.optimize(max_seconds=time_limit, mip_rel_gap=0.0) + + # Early failure detection: do not extract on non-feasible status + if model.status not in (OptimizationStatus.OPTIMAL, OptimizationStatus.FEASIBLE): + return {"status": "INFEASIBLE" if model.status == OptimizationStatus.INFEASIBLE else "UNKNOWN", "objective": None} + + result = {"status": "OPTIMAL" if model.status == OptimizationStatus.OPTIMAL else "FEASIBLE", + "objective": model.objective_value} + + # Extract routes + routes = {} + for v in vehicles: + if vehicle_active[v].x >= 0.5: + route = [] + # Find start edge-state + for e in edges: + for s in states: + if start[v,e,s].x >= 0.5: + route.append((e,s)) + break + if route: + break + + # Follow sequence + while route: + current_e, current_s = route[-1] + next_found = False + for e2 in edges: + for s2 in states: + if z[v,current_e,current_s,e2,s2].x >= 0.5: + route.append((e2,s2)) + next_found = True + break + if next_found: + break + if not next_found: + break + + routes[v] = route + else: + routes[v] = [] + + result["routes"] = routes + return result + +# Usage +# model, x, start, end, z, vehicle_active = build_carp_model_mip(vehicles, edges, states, demand, capacity, entry_node, exit_node, dist_matrix, depot) +# result = solve_carp_mip(model, x, start, end, z, vehicle_active, vehicles, edges, states) +# print(f"RESULT_JSON:{json.dumps(result)}") +``` + +### Common Pitfalls +- Using a big-M value that is too small, causing MTZ constraints to be ineffective for subtour elimination. +- Not checking for numerical stability when using large distance values in the objective. +- Forgetting to handle the case where a vehicle has no assigned tasks (no start variable is 1). +- Extracting routes without verifying `vehicle_active` and start/end consistency, risking incorrect reconstruction. diff --git a/skills/optskills/skill_library/capacitated_facility_location_milp.md b/skills/optskills/skill_library/capacitated_facility_location_milp.md new file mode 100644 index 0000000..1327258 --- /dev/null +++ b/skills/optskills/skill_library/capacitated_facility_location_milp.md @@ -0,0 +1,223 @@ +--- +name: Capacitated Facility Location MILP +description: | + Model and solve capacitated facility location problems with fixed opening costs and linear transportation costs using mixed-integer linear programming (MILP), with robust solver integration and solution validation. + +--- + +# Workflow 1 (Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +Use Pyomo's abstract modeling to define a MILP formulation, separating model logic from data. Designed for clarity, maintainability, and compatibility with open-source solvers like HiGHS and CBC. + +### Step 1 - Define Model Sets and Parameters +- Declare sets for facilities (`F`) and customers (`C`) using `pyo.Set()`. +- Define parameters `fixed_cost[f]`, `capacity[f]`, `demand[c]`, and `shipping_cost[f,c]` using `pyo.Param()`. +- **Verify data consistency**: Ensure total demand does not exceed total available capacity (`sum(capacity[f] for f in F)`) to avoid trivially infeasible models. +- **Ensure consistent indexing**: If cost data is provided in (customer, facility) format, transpose it to (facility, customer) for modeling convenience. + +### Step 2 - Declare Decision Variables +- Create binary variables `y[f]` for facility opening (`domain=pyo.Binary`). +- Create binary variables `x[f, c]` for customer assignment (`domain=pyo.Binary`). This enforces single-sourcing. + +### Step 3 - Formulate Objective Function +- Construct a linear objective to minimize total cost: `sum(fixed_cost[f] * y[f] for f in F) + sum(shipping_cost[f,c] * x[f,c] for f in F for c in C)`. +- Set `sense=pyo.minimize`. + +### Step 4 - Implement Core Constraints +- **Demand satisfaction**: For each customer `c`, `sum(x[f, c] for f in F) == 1`. +- **Capacity linking**: For each facility `f`, `sum(demand[c] * x[f, c] for c in C) <= capacity[f] * y[f]`. + +### Formulation Template +```json +{ + "sets": ["facilities", "customers"], + "parameters": ["fixed_cost[facilities]", "capacity[facilities]", "demand[customers]", "shipping_cost[facilities, customers]"], + "decision_variables": ["y[facilities] ∈ {0,1}", "x[facilities, customers] ∈ {0,1}"], + "objective": { + "sense": "min", + "expression": "sum(fixed_cost[f] * y[f] for f in facilities) + sum(shipping_cost[f,c] * x[f,c] for f in facilities for c in customers)" + }, + "constraints": [ + "demand_satisfaction[c]: sum(x[f,c] for f in facilities) == 1 for each c in customers", + "capacity_link[f]: sum(demand[c] * x[f,c] for c in customers) <= capacity[f] * y[f] for each f in facilities" + ] +} +``` + +### Common Pitfalls +- Incomplete parameter initialization causing `KeyError`. Ensure dictionaries are fully populated for all set indices. +- Incorrect `sum()` usage; pass a single iterable (e.g., `sum(x[f,c] for f in F for c in C)`). +- Mis-specifying the capacity linking constraint; ensure it is `demand[c] * x[f,c]`. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a configured solver, with explicit checks for solution status and careful loading of results to ensure robustness. + +### Step 1 - Configure and Instantiate Solver +- Instantiate the solver via `SolverFactory('solver_name')` (e.g., `'highs'` or `'cbc'`). +- Set performance options: `time_limit=[TIME_LIMIT]`, `mip_rel_gap=0.0` for exact solution, and `threads`. +- **Enable solver output for debugging**: Set `tee=True` to view branch-and-bound progress. + +### Step 2 - Solve with Status Control +- Execute the solve command with `load_solutions=False` to prevent automatic loading. +- Capture the results object for status checking. + +### Step 3 - Check Termination Status and Load Solution +- Verify the solver status (`SolverStatus.ok`) and termination condition (`TerminationCondition.optimal` or `.feasible`). +- Only if the status is acceptable, load the solution into the model using `model.solutions.load_from(results)`. + +### Step 4 - Extract and Validate Solution +- Extract the objective value using `pyo.value(model.obj)`. +- **Handle floating-point precision**: If all input costs are integers, round the solver's objective value to the nearest integer or recompute the exact cost using the integer solution values. +- Identify opened facilities by checking `y[f].value > 0.5` (with tolerance). +- **Implement solution verification**: + 1. Verify each customer is assigned to exactly one facility: `sum(x[f,c].value for f in F) == 1` for all `c`. + 2. Verify capacity constraints: For each facility `f`, compute `assigned = sum(demand[c] * x[f,c].value for c in C)` and check `assigned <= capacity[f] * y[f].value + 1e-6`. +- **Calculate cost breakdown manually**: Recompute fixed costs and shipping costs from variable values to validate the reported objective. +- **Extract and report the solution**: List opened facilities and the assignment of each customer, along with capacity utilization percentages. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverFactory, SolverStatus, TerminationCondition + +# 1. Build Model (ConcreteModel, Sets, Vars, Objective, Constraints) +# ... model definition as per Modeling Stage ... + +# 2. Instantiate Solver & Set Options +solver = SolverFactory('highs') +solver.options['time_limit'] = [TIME_LIMIT] +solver.options['mip_rel_gap'] = 0.0 + +# 3. Solve with status control +results = solver.solve(model, tee=True, load_solutions=False) + +# 4. Check Status & Termination +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + model.solutions.load_from(results) # Load only after status check + objective_value = float(pyo.value(model.obj)) + # 5. Extract and validate solution + opened_facilities = [f for f in model.F if pyo.value(model.y[f]) > 0.5] + # ... further processing and validation ... +else: + # Handle non-optimal/infeasible case + print(f"Solver failed: Status={status}, Termination={term}") +``` + +### Common Pitfalls +- Loading solutions automatically (`load_solutions=True`) without checking status first. +- Misinterpreting termination condition; `feasible` is acceptable for a usable solution. +- Not using a tolerance when checking binary variable values. +- Forgetting to multiply by `demand[c]` in the capacity verification step. + +# Workflow 2 (OR-Tools with SCIP/CBC) + +## Modeling stage + +### Strategy Overview +Use Google's OR-Tools `pywraplp` API for a direct, imperative model build. Suited for rapid prototyping and lightweight modeling. + +### Step 1 - Initialize Solver and Define Variable Scope +- Create a solver instance using `pywraplp.Solver.CreateSolver("SCIP")` or `"CBC"`. +- Define index sets for facilities (`F`) and customers (`C`) as lists. + +### Step 2 - Create Decision Variables +- Create binary variables for facility opening: `y[i] = solver.IntVar(0, 1, name)`. +- Create binary assignment variables: `x[i, j] = solver.IntVar(0, 1, name)`. + +### Step 3 - Build Objective Function Term-by-Term +- Instantiate the objective with `solver.Objective()`. +- Add fixed cost terms: `objective.SetCoefficient(y[i], fixed_cost[i])`. +- Add transportation cost terms: `objective.SetCoefficient(x[i, j], shipping_cost[i][j])`. +- Set the optimization sense to minimization with `objective.SetMinimization()`. + +### Step 4 - Add Constraints Directly +- For each customer `j`, add a demand satisfaction constraint: `solver.Add(sum(x[i, j] for i in F) == 1)`. +- For each facility `i`, add the capacity linking constraint: `solver.Add(sum(demand[j] * x[i, j] for j in C) <= capacity[i] * y[i])`. + +### Formulation Template +```json +{ + "sets": ["facilities", "customers"], + "parameters": ["fixed_cost[facilities]", "capacity[facilities]", "demand[customers]", "shipping_cost[facilities, customers]"], + "decision_variables": ["y[facilities] ∈ {0,1}", "x[facilities, customers] ∈ {0,1}"], + "objective": { + "sense": "min", + "expression": "sum(fixed_cost[i] * y[i] for i in facilities) + sum(shipping_cost[i][j] * x[i,j] for i in facilities for j in customers)" + }, + "constraints": [ + "demand_satisfaction[j]: sum(x[i,j] for i in facilities) == 1 for each j in customers", + "capacity_link[i]: sum(demand[j] * x[i,j] for j in customers) <= capacity[i] * y[i] for each i in facilities" + ] +} +``` + +### Common Pitfalls +- Incorrectly building the capacity linking constraint; ensure it is `sum(demand[j] * x[i,j]) <= capacity[i] * y[i]`. +- Using `solver.infinity()` for binary variable bounds; use `IntVar(0, 1)`. +- Omitting variable-coefficient pairs in the objective, leading to an incorrect total cost. + +## Solving stage + +### Strategy Overview +Execute the solver, handle its status codes, and extract solution values directly from the OR-Tools variable objects. + +### Step 1 - Configure Solver Settings +- Set a time limit using `solver.SetTimeLimit([TIME_LIMIT_MS])`. +- Control parallelism with `solver.SetNumThreads([NUMBER])`. + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve()`. +- Check if the status is `solver.OPTIMAL` or `solver.FEASIBLE`. Handle other statuses appropriately. + +### Step 3 - Extract Solution Values +- If the status is acceptable, obtain the objective value via `solver.Objective().Value()`. +- **Handle floating-point precision**: If all input costs are integers, round the objective value to the nearest integer or recompute the exact cost using the integer solution values. +- Determine opened facilities by checking `y[i].solution_value() > 0.5`. +- Retrieve assignment decisions using `x[i, j].solution_value()`. + +### Step 4 - Post-Solve Validation +- **Implement solution verification**: + 1. Verify each customer is assigned to exactly one facility. + 2. For each facility `i`, compute `assigned = sum(demand[j] * x[i,j].solution_value() for j in C)` and check `assigned <= capacity[i] * y[i].solution_value() + 1e-6`. +- **Calculate cost breakdown manually**: Recompute fixed and shipping costs from extracted variable values to validate the solver's reported objective. +- **Extract and report the solution**: List opened facilities and the assignment of each customer, along with capacity utilization percentages. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# 1. Initialize Solver +solver = pywraplp.Solver.CreateSolver("SCIP") +solver.SetTimeLimit([TIME_LIMIT_MS]) +solver.SetNumThreads(4) + +# 2. Build Model (Variables, Objective, Constraints) +# ... model definition as per Modeling Stage ... + +# 3. Solve and check status +status = solver.Solve() + +# 4. Extract results if feasible/optimal +if status in (solver.OPTIMAL, solver.FEASIBLE): + total_cost = solver.Objective().Value() + opened_facilities = [i for i in F if y[i].solution_value() > 0.5] + assignments = {(i, j): x[i, j].solution_value() for i in F for j in C if x[i, j].solution_value() > 0.5} + # 5. Optional validation + # ... +else: + print(f"Solver did not find a feasible solution. Status: {status}") +``` + +### Common Pitfalls +- Confusing `solver.OPTIMAL` with `solver.FEASIBLE`; the latter provides a valid solution without optimality guarantee. +- Not using a tolerance when checking binary variable values. +- Assuming the solver's internal time limit units are seconds; OR-Tools uses milliseconds. +- Forgetting to multiply by `demand[j]` in the capacity verification step. diff --git a/skills/optskills/skill_library/capacitated_transportation_problem_solver.md b/skills/optskills/skill_library/capacitated_transportation_problem_solver.md new file mode 100644 index 0000000..a42e5ab --- /dev/null +++ b/skills/optskills/skill_library/capacitated_transportation_problem_solver.md @@ -0,0 +1,284 @@ +--- +name: Capacitated Transportation Problem Solver +description: | + Model and solve balanced capacitated transportation problems with linear costs, supply/demand equality, and per-route capacity limits using continuous flow variables. +--- + +# Workflow 1 (OR-Tools Linear Solver) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a linear program using the OR-Tools wrapper for the GLOP solver. Variables are created with explicit upper bounds to embed capacity constraints, and equality constraints enforce exact supply and demand fulfillment. + +### Step 1 - Define Data Structures +- Organize problem data into Python lists or dictionaries for supply, demand, cost, and capacity. +- Use zero-based indexing for sources (facilities) and destinations (centers). +- Verify that total supply equals total demand for a balanced problem. + +### Step 2 - Create Variables with Bounds +- Instantiate a `pywraplp.Solver` with the `"GLOP"` backend. +- Create a non-negative continuous variable `x[i][j]` for each source-destination pair. +- Set the variable's upper bound directly to `capacity[i][j]` during creation to enforce per-route limits. + +### Step 3 - Add Supply and Demand Constraints +- For each source `i`, create a linear equality constraint: `sum_j x[i][j] = supply[i]`. +- For each destination `j`, create a linear equality constraint: `sum_i x[i][j] = demand[j]`. +- Use `solver.Constraint(value, value)` to set both lower and upper bounds to the same value. + +### Step 4 - Define Linear Objective +- Create the objective function: `minimize sum_i sum_j cost[i][j] * x[i][j]`. +- Set coefficients for each variable using `objective.SetCoefficient(x[i][j], cost[i][j])`. + +### Formulation Template +```json +{ + "sets": [ + "sources", + "destinations" + ], + "parameters": [ + "supply[sources]", + "demand[destinations]", + "cost[sources][destinations]", + "capacity[sources][destinations]" + ], + "decision_variables": [ + "flow[sources][destinations] >= 0" + ], + "objective": { + "sense": "min", + "expression": "sum_{i in sources, j in destinations} cost[i][j] * flow[i][j]" + }, + "constraints": [ + "supply_constraint[i in sources]: sum_{j in destinations} flow[i][j] = supply[i]", + "demand_constraint[j in destinations]: sum_{i in sources} flow[i][j] = demand[j]", + "capacity_constraint[i in sources, j in destinations]: flow[i][j] <= capacity[i][j]" + ] +} +``` + +### Common Pitfalls +- Forgetting to check for problem balance (total supply vs. total demand) before building the model, which leads to infeasibility. +- Setting variable upper bounds to `infinity` instead of the specific `capacity[i][j]`, which omits the per-route limit. +- Not verifying the solver status (`OPTIMAL` or `FEASIBLE`) before extracting solution values. + +## Solving stage + +### Strategy Overview +Solve the linear program using the GLOP solver, check termination status, and extract the solution. Implement post-solve verification to ensure constraints are satisfied within a numerical tolerance. + +### Step 1 - Execute Solve and Check Status +- Call `solver.Solve()` and capture the status. +- Check if the status is `solver.OPTIMAL` or `solver.FEASIBLE`. If not, log the solver status code and exit. + +### Step 2 - Extract and Verify Solution +- Retrieve the objective value using `solver.Objective().Value()`. +- Iterate through all variables to get flow values `x[i][j].solution_value()`. +- Optionally, recompute aggregate flows for each source and destination to verify supply/demand constraints are met within a tolerance (e.g., 1e-6). + +### Step 3 - Report Results +- Print the objective value in a parseable format (e.g., `RESULT:{objective_value}`). +- For debugging, log a summary of non-zero flows and their utilization relative to capacity. + +### Code Usage +```python +# build model from formulation +import pywraplp + +# 1. Initialize solver +solver = pywraplp.Solver.CreateSolver("GLOP") +if not solver: + raise RuntimeError("Solver not available.") + +# 2. Define data (placeholders) +num_sources = len(supply) +num_dests = len(demand) +# supply = [...]; demand = [...]; cost = [[...]]; capacity = [[...]] + +# 3. Create variables with capacity bounds +x = {} +for i in range(num_sources): + for j in range(num_dests): + x[i, j] = solver.NumVar(0, capacity[i][j], f'flow_{i}_{j}') + +# 4. Add supply constraints +for i in range(num_sources): + ct = solver.Constraint(supply[i], supply[i]) + for j in range(num_dests): + ct.SetCoefficient(x[i, j], 1) + +# 5. Add demand constraints +for j in range(num_dests): + ct = solver.Constraint(demand[j], demand[j]) + for i in range(num_sources): + ct.SetCoefficient(x[i, j], 1) + +# 6. Set objective +objective = solver.Objective() +for i in range(num_sources): + for j in range(num_dests): + objective.SetCoefficient(x[i, j], cost[i][j]) +objective.SetMinimization() + +# solve with status / termination checks +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + print(f"RESULT:{objective.Value()}") + # Optional verification + for i in range(num_sources): + total = sum(x[i, j].solution_value() for j in range(num_dests)) + if abs(total - supply[i]) > 1e-6: + print(f"Warning: supply mismatch for source {i}") +else: + print(f"Solver failed with status: {status}") +``` + +### Common Pitfalls +- Assuming the solver always returns an optimal solution without checking the status. +- Not handling numerical precision issues when verifying constraint satisfaction. +- Using `solver.INFINITY` as an upper bound, which can hide modeling errors if capacity data is missing. + +# Workflow 2 (Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +Model the problem using Pyomo's abstract or concrete modeling environment. Define sets, parameters, variables, and constraints declaratively. This approach separates the model formulation from the solver interface, enabling easy solver swapping. + +### Step 1 - Define Abstract Sets and Parameters +- Create Pyomo `Set` objects for sources and destinations. +- Define `Param` objects for supply, demand, cost, and capacity, indexed over the appropriate sets. +- Use dictionaries to initialize parameter data. + +### Step 2 - Declare Decision Variables +- Create a `Var` object `model.flow` indexed over the Cartesian product of source and destination sets. +- Specify the domain as `pyo.NonNegativeReals` for continuous, non-negative flows. + +### Step 3 - Construct Constraints via Rules +- Define a rule function for the supply constraint that returns `sum(model.flow[i, j] for j in destinations) == supply[i]`. +- Define a rule function for the demand constraint that returns `sum(model.flow[i, j] for i in sources) == demand[j]`. +- Define a rule function for the capacity constraint that returns `model.flow[i, j] <= capacity[i, j]`. + +### Step 4 - Formulate the Objective +- Define the objective function as `sum(cost[i, j] * model.flow[i, j] for i in sources for j in destinations)`. +- Set the sense to `minimize`. + +### Formulation Template +```json +{ + "sets": [ + "sources", + "destinations" + ], + "parameters": [ + "supply[sources]", + "demand[destinations]", + "cost[sources, destinations]", + "capacity[sources, destinations]" + ], + "decision_variables": [ + "flow[sources, destinations] in NonNegativeReals" + ], + "objective": { + "sense": "min", + "expression": "sum_{i in sources, j in destinations} cost[i,j] * flow[i,j]" + }, + "constraints": [ + "supply_rule(i): sum_{j in destinations} flow[i,j] == supply[i]", + "demand_rule(j): sum_{i in sources} flow[i,j] == demand[j]", + "capacity_rule(i,j): flow[i,j] <= capacity[i,j]" + ] +} +``` + +### Common Pitfalls +- Incorrectly indexing parameters in constraint rules, leading to `KeyError`. +- Forgetting to initialize all parameters before creating the model instance, resulting in uninitialized data errors. +- Using mutable default arguments (like lists) in Pyomo rule functions. + +## Solving stage + +### Strategy Overview +Instantiate a solver (HiGHS or CBC) via Pyomo's `SolverFactory`, configure it with performance options, solve the model, and rigorously check the termination condition before extracting results. + +### Step 1 - Configure and Execute Solver +- Create a solver object: `solver = SolverFactory('highs')` (or `'cbc'`). +- Set solver options such as time limit (`time_limit`), optimality gap (`mip_rel_gap` for MIPs), and thread count (`threads`). +- Call `results = solver.solve(model, tee=False)` to solve silently, or `tee=True` for log output. + +### Step 2 - Validate Termination Status +- Check `results.solver.status` is `SolverStatus.ok`. +- Check `results.solver.termination_condition` is `TerminationCondition.optimal` or `...feasible`. +- If status is not acceptable, log the termination condition and investigate infeasibility. + +### Step 3 - Extract and Verify Solution +- Retrieve the objective value: `pyo.value(model.obj)`. +- Access variable values: `pyo.value(model.flow[i, j])`. +- Perform post-solve verification by recomputing constraint left-hand sides and comparing to parameters with a tolerance. + +### Step 4 - Output Structured Results +- Print the objective value in a consistent format (e.g., `RESULT:{objective_value}`). +- Optionally, generate a report of the solution, listing non-zero flows and constraint slack. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# 1. Create a ConcreteModel +model = pyo.ConcreteModel() + +# 2. Define sets (placeholders) +model.sources = pyo.Set(initialize=range(num_sources)) +model.dests = pyo.Set(initialize=range(num_dests)) + +# 3. Define parameters (initialize with your data) +model.supply = pyo.Param(model.sources, initialize=supply_dict) +model.demand = pyo.Param(model.dests, initialize=demand_dict) +model.cost = pyo.Param(model.sources, model.dests, initialize=cost_dict) +model.capacity = pyo.Param(model.sources, model.dests, initialize=capacity_dict) + +# 4. Define variables +model.flow = pyo.Var(model.sources, model.dests, domain=pyo.NonNegativeReals) + +# 5. Define objective +def obj_rule(m): + return sum(m.cost[i, j] * m.flow[i, j] for i in m.sources for j in m.dests) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + +# 6. Define constraints +def supply_rule(m, i): + return sum(m.flow[i, j] for j in m.dests) == m.supply[i] +model.supply_con = pyo.Constraint(model.sources, rule=supply_rule) + +def demand_rule(m, j): + return sum(m.flow[i, j] for i in m.sources) == m.demand[j] +model.demand_con = pyo.Constraint(model.dests, rule=demand_rule) + +def capacity_rule(m, i, j): + return m.flow[i, j] <= m.capacity[i, j] +model.capacity_con = pyo.Constraint(model.sources, model.dests, rule=capacity_rule) + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') # or 'cbc' +solver.options['time_limit'] = 30 +results = solver.solve(model, tee=False) + +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in (TerminationCondition.optimal, TerminationCondition.feasible)): + print(f"RESULT:{pyo.value(model.obj)}") + # Optional verification loop + for i in model.sources: + total = sum(pyo.value(model.flow[i, j]) for j in model.dests) + if abs(total - pyo.value(model.supply[i])) > 1e-6: + print(f"Verification warning for source {i}") +else: + print(f"Solver failed: {results.solver.termination_condition}") +``` + +### Common Pitfalls +- Confusing `SolverStatus` with `TerminationCondition`; both must be checked. +- Not setting the `sense` on the objective, defaulting to minimization. +- Attempting to access variable values from an unsolved or infeasible model, causing `ValueError`. diff --git a/skills/optskills/skill_library/capacitatedassignmentsolver.md b/skills/optskills/skill_library/capacitatedassignmentsolver.md new file mode 100644 index 0000000..d00b10d --- /dev/null +++ b/skills/optskills/skill_library/capacitatedassignmentsolver.md @@ -0,0 +1,220 @@ +--- +name: CapacitatedAssignmentSolver +description: | + Solves binary assignment problems with exactly-one assignment and knapsack-style capacity constraints by formulating a Mixed-Integer Linear Program (MILP) and solving it with a configured MILP solver. + +--- +# Workflow 1 (Direct Solver API) + +## Modeling stage + +### Strategy Overview +Model the problem directly using a solver's native Python API (e.g., OR-Tools, PuLP). This approach is procedural, keeps the model close to the solver, and is efficient for straightforward implementations. + +### Step 1 - Define Data Structures +- Organize problem data into clear dictionaries or lists for costs, capacities, and requirements. Use descriptive keys like `cost[(i, j)]`, `capacity[j]`, `requirement[(i, j)]`. +- Define sets for assignable items `I` and resources `J` as iterable lists. + +### Step 2 - Create Solver and Variables +- Instantiate a MILP solver (e.g., `pywraplp.Solver.CreateSolver('SCIP')`). Check for solver availability. +- Create binary decision variables `x[(i, j)]` for all `i` in `I`, `j` in `J`. Name them clearly, like `assign`. + +### Step 3 - Formulate Exactly-One Constraints +- For each item `i`, add a constraint: `sum(x[(i, j)] for j in J) == 1`. This ensures each item is assigned to exactly one resource. + +### Step 4 - Formulate Knapsack Capacity Constraints +- For each resource `j`, add a constraint: `sum(requirement[(i, j)] * x[(i, j)] for i in I) <= capacity[j]`. This enforces resource limits. + +### Step 5 - Define Linear Objective +- Set the objective to minimize total cost: `solver.Minimize(sum(cost[(i, j)] * x[(i, j)] for i in I for j in J))`. + +### Formulation Template +```json +{ + "sets": ["I (items)", "J (resources)"], + "parameters": ["cost[i][j]", "capacity[j]", "requirement[i][j]"], + "decision_variables": ["x[i][j] ∈ {0,1}"], + "objective": { + "sense": "min", + "expression": "sum(cost[i][j] * x[i][j] for i in I for j in J)" + }, + "constraints": [ + "sum(x[i][j] for j in J) == 1, for all i in I", + "sum(requirement[i][j] * x[i][j] for i in I) <= capacity[j], for all j in J" + ] +} +``` + +### Common Pitfalls +- Forgetting to check if the solver instance was created successfully (`if solver is None:`). +- Using incorrect indices when populating parameters, leading to `KeyError`. +- Not setting a time limit for large instances, causing long, unbound runs. + +## Solving stage + +### Strategy Overview +Solve the model using the configured solver, extract the solution, and verify its correctness and optimality. Handle solver statuses and produce parseable output. + +### Step 1 - Configure Solver Parameters +- Set practical solver parameters: `solver.SetTimeLimit(30000)` for a time limit and `solver.SetNumThreads(4)` to utilize multiple cores. + +### Step 2 - Invoke Solver and Check Status +- Call `solver.Solve()`. +- Check the result status: `status in (solver.OPTIMAL, solver.FEASIBLE)`. Handle `solver.INFEASIBLE` or `solver.UNBOUNDED` appropriately. + +### Step 3 - Extract and Verify Solution +- If optimal or feasible, iterate over variables `x[(i, j)]` and collect assignments where `x[(i, j)].solution_value() > 0.5`. +- Calculate derived metrics like actual resource usage per `j` to verify capacity constraints are satisfied. + +### Step 4 - Output Structured Results +- Print the objective value in a consistent, parseable format (e.g., `print(f"RESULT:{solver.Objective().Value()}")`). +- Optionally, output a detailed JSON with assignments and resource usage for debugging. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('SCIP') +if solver is None: + raise Exception('Solver not available.') +x = {} +for i in I: + for j in J: + x[(i, j)] = solver.BoolVar(f'x_{i}_{j}') + +# Add constraints +for i in I: + solver.Add(sum(x[(i, j)] for j in J) == 1) +for j in J: + solver.Add(sum(requirement[(i, j)] * x[(i, j)] for i in I) <= capacity[j]) + +# Set objective +objective_terms = [] +for i in I: + for j in J: + objective_terms.append(cost[(i, j)] * x[(i, j)]) +solver.Minimize(sum(objective_terms)) + +# solve with status / termination checks +solver.SetTimeLimit(30000) +status = solver.Solve() + +if status in (solver.OPTIMAL, solver.FEASIBLE): + print(f'RESULT:{solver.Objective().Value()}') + # Extract assignments... +else: + print('No feasible solution found.') +``` + +### Common Pitfalls +- Assuming the solver always returns an optimal solution without checking the status. +- Extracting variable values without a tolerance check (use `> 0.5` for binary variables). +- Not verifying calculated resource usage against the original capacity constraints. + +# Workflow 2 (Pyomo Modeling Framework) + +## Modeling stage + +### Strategy Overview +Model the problem using Pyomo, an algebraic modeling language in Python. This approach is declarative, separates model from data, and facilitates integration with various solvers. + +### Step 1 - Define Abstract Sets and Parameters +- Define Pyomo `Set` objects for items `model.I` and resources `model.J`. +- Define `Param` objects for `model.cost`, `model.capacity`, and `model.requirement`, initialized from dictionaries. + +### Step 2 - Declare Decision Variables +- Declare binary variables `model.x` indexed over `model.I` and `model.J` using `pyo.Var(domain=pyo.Binary)`. + +### Step 3 - Formulate Exactly-One Constraints +- Add a constraint for each item `i`: `sum(model.x[i, j] for j in model.J) == 1`. Use a Pyomo `Constraint` rule. + +### Step 4 - Formulate Knapsack Capacity Constraints +- Add a constraint for each resource `j`: `sum(model.requirement[i, j] * model.x[i, j] for i in model.I) <= model.capacity[j]`. + +### Step 5 - Define the Objective +- Set the objective to minimize total cost using `pyo.Objective(expr=sum(model.cost[i, j] * model.x[i, j] for i in model.I for j in model.J), sense=pyo.minimize)`. + +### Formulation Template +```json +{ + "sets": ["I (items)", "J (resources)"], + "parameters": ["cost[i][j]", "capacity[j]", "requirement[i][j]"], + "decision_variables": ["x[i][j] ∈ {0,1}"], + "objective": { + "sense": "min", + "expression": "sum(cost[i][j] * x[i][j] for i in I for j in J)" + }, + "constraints": [ + "sum(x[i][j] for j in J) == 1, for all i in I", + "sum(requirement[i][j] * x[i][j] for i in I) <= capacity[j], for all j in J" + ] +} +``` + +### Common Pitfalls +- Incorrectly initializing `Param` objects with missing indices, causing runtime errors. +- Using Python's `sum` inside a Pyomo expression rule instead of the built-in `pyo.summation`. +- Forgetting to deactivate the dual updates if using `solver.options['threads']` with certain solvers. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a MILP solver like CBC or HiGHS, configured via `SolverFactory`. Robustly handle solver results and extract the solution. + +### Step 1 - Configure and Invoke Solver +- Create a solver object: `solver = pyo.SolverFactory('cbc')`. +- Set solver options: `solver.options['seconds'] = 30`, `solver.options['ratio'] = 0.0` (for optimality gap), `solver.options['threads'] = 4`. + +### Step 2 - Solve and Check Termination +- Call `results = solver.solve(model, tee=False)`. +- Check `results.solver.status` is `SolverStatus.ok` and `results.solver.termination_condition` is `TerminationCondition.optimal` or `...feasible`. + +### Step 3 - Extract Solution and Verify +- Use `pyo.value(model.x[i, j])` to get variable values. Collect assignments where the value is `> 0.5`. +- Recalculate resource usage and total cost to verify against the model's constraints and objective. + +### Step 4 - Output Results +- Print the objective value from `pyo.value(model.obj)` in a parseable format (e.g., `RESULT:`). +- Optionally, output assignments and verification metrics as structured JSON. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=items) +model.J = pyo.Set(initialize=resources) +model.cost = pyo.Param(model.I, model.J, initialize=cost_dict) +model.capacity = pyo.Param(model.J, initialize=capacity_dict) +model.requirement = pyo.Param(model.I, model.J, initialize=requirement_dict) +model.x = pyo.Var(model.I, model.J, domain=pyo.Binary) + +def obj_rule(m): + return sum(m.cost[i, j] * m.x[i, j] for i in m.I for j in m.J) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + +def assign_rule(m, i): + return sum(m.x[i, j] for j in m.J) == 1 +model.assign_constr = pyo.Constraint(model.I, rule=assign_rule) + +def capacity_rule(m, j): + return sum(m.requirement[i, j] * m.x[i, j] for i in m.I) <= m.capacity[j] +model.capacity_constr = pyo.Constraint(model.J, rule=capacity_rule) + +# solve with status / termination checks +solver = pyo.SolverFactory('cbc') +solver.options['seconds'] = 30 +solver.options['ratio'] = 0.0 +results = solver.solve(model, tee=False) + +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in (pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible)): + print(f'RESULT:{pyo.value(model.obj)}') + # Extract assignments... +else: + print('Solver did not find a feasible solution.') +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition` after solving. +- Attempting to access `pyo.value` on an undefined variable if the solver failed. +- Misconfiguring solver options for the specific solver backend (e.g., `'ratio'` vs. `'mipgap'`). diff --git a/skills/optskills/skill_library/cardinality_constrained_pairwise_interaction_maximization.md b/skills/optskills/skill_library/cardinality_constrained_pairwise_interaction_maximization.md new file mode 100644 index 0000000..8fc3b23 --- /dev/null +++ b/skills/optskills/skill_library/cardinality_constrained_pairwise_interaction_maximization.md @@ -0,0 +1,293 @@ +--- +name: Cardinality-Constrained Pairwise Interaction Maximization +description: | + Model and solve selection problems where exactly K items are chosen to maximize the sum of directed pairwise interaction scores between selected items, using linearized binary variables and logical constraints. +--- + +# Workflow 1 (Pyomo with Gurobi/Highs) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) using binary selection and interaction variables. Linearize the logical relationship between selection and interaction via constraints, enabling solution by standard MILP solvers. + +### Step 1 - Define Sets and Parameters +- Define a set `NODES` representing all candidate items (e.g., nodes, elements). +- Define a parameter `SCORE[i, j]` for each ordered pair `(i, j)` where `i != j`, representing the directed interaction score from item `i` to item `j`. +- Define a scalar parameter `K` for the exact number of items to select. + +### Step 2 - Define Decision Variables +- Create binary selection variables `x[i]` for each item `i` in `NODES`. `x[i] = 1` if item `i` is selected. +- Create binary interaction variables `y[i, j]` for each ordered pair `(i, j)` where `i != j`. `y[i, j] = 1` if the directed interaction from `i` to `j` is active (i.e., counted in the objective). + +### Step 3 - Formulate Constraints +- Add a cardinality constraint: the sum of all `x[i]` must equal `K`. +- Add logical activation constraints: `y[i, j] <= x[i]` and `y[i, j] <= x[j]`. An interaction can only be active if both its source and target items are selected. +- Add a forcing constraint: `y[i, j] >= x[i] + x[j] - 1`. If both items are selected, the interaction variable must be active (equal to 1). + +### Step 4 - Define Objective +- Maximize the sum of scores over all active directed interactions: `sum( SCORE[i, j] * y[i, j] for all i, j where i != j )`. + +### Formulation Template +```json +{ + "sets": [ + {"name": "NODES", "description": "Set of all candidate items."}, + {"name": "ORDERED_PAIRS", "description": "Set of all ordered pairs (i, j) where i != j, i, j in NODES."} + ], + "parameters": [ + {"name": "SCORE", "index": "ORDERED_PAIRS", "description": "Directed interaction score from i to j."}, + {"name": "K", "description": "Exact number of items to select."} + ], + "decision_variables": [ + {"name": "x", "index": "NODES", "type": "binary", "description": "1 if item i is selected."}, + {"name": "y", "index": "ORDERED_PAIRS", "type": "binary", "description": "1 if directed interaction from i to j is active."} + ], + "objective": { + "sense": "max", + "expression": "sum( SCORE[i,j] * y[i,j] for (i,j) in ORDERED_PAIRS )" + }, + "constraints": [ + {"name": "select_exactly_k", "expression": "sum( x[i] for i in NODES ) == K" }, + {"name": "y_leq_source", "expression": "y[i,j] <= x[i]", "index": "ORDERED_PAIRS"}, + {"name": "y_leq_target", "expression": "y[i,j] <= x[j]", "index": "ORDERED_PAIRS"}, + {"name": "y_geq_both_selected", "expression": "y[i,j] >= x[i] + x[j] - 1", "index": "ORDERED_PAIRS"} + ] +} +``` + +### Common Pitfalls +- Assuming symmetric interaction scores without verifying the problem statement. Always clarify if `SCORE[i,j]` is directed or if the objective sums over unordered pairs. +- Creating overly complex formulations with unnecessary auxiliary variables for small problem instances where enumeration is feasible. +- Forgetting the forcing constraint (`y[i,j] >= x[i] + x[j] - 1`), which is necessary to correctly link the interaction variable to the selection variables. + +## Solving stage + +### Strategy Overview +Implement the MILP formulation in Pyomo and solve it using a high-performance solver like Gurobi or HiGHS. Configure solver parameters for performance and reliability, and implement robust status checking and result extraction. + +### Step 1 - Model Construction +- Instantiate a `pyo.ConcreteModel()`. +- Define `pyo.Set` objects for `NODES` and `ORDERED_PAIRS`. +- Define `pyo.Param` objects for `SCORE` and `K`. +- Define `pyo.Var` objects for `x` and `y` with `domain=pyo.Binary`. + +### Step 2 - Constraint and Objective Addition +- Add the cardinality constraint using `pyo.Constraint` and `sum()`. +- Add the three families of logical constraints using `pyo.Constraint` with rule functions or indexed constructions. +- Add the objective using `pyo.Objective` with `sense=pyo.maximize`. + +### Step 3 - Solver Configuration and Execution +- Create a solver instance (e.g., `pyo.SolverFactory('gurobi')`). +- Set key solver parameters: `TimeLimit`, `MIPGap`, `Threads`, and `Seed` for reproducibility. +- Call `solver.solve(model, tee=False)` to execute the solve. + +### Step 4 - Status Verification and Result Extraction +- Check the solver status (`pyo.SolverStatus`) and model termination condition (`pyo.TerminationCondition`). +- If the solution is optimal or feasible, extract the objective value using `pyo.value(model.obj)`. +- Iterate over the `x` and `y` variables to list selected items and active interactions. +- If the solve fails or is infeasible, return a structured error message. + +### Code Usage +```python +import pyomo.environ as pyo + +# 1. Build Model +model = pyo.ConcreteModel() +model.NODES = pyo.Set(initialize=NODES_LIST) +model.ORDERED_PAIRS = pyo.Set(initialize=ORDERED_PAIRS_LIST, dimen=2) + +def score_init(model, i, j): + return SCORE_DICT[(i, j)] +model.SCORE = pyo.Param(model.ORDERED_PAIRS, initialize=score_init) +model.K = pyo.Param(initialize=K_VALUE) + +model.x = pyo.Var(model.NODES, domain=pyo.Binary) +model.y = pyo.Var(model.ORDERED_PAIRS, domain=pyo.Binary) + +# 2. Add Constraints +def cardinality_rule(model): + return sum(model.x[i] for i in model.NODES) == model.K +model.select_exactly_k = pyo.Constraint(rule=cardinality_rule) + +def y_le_source_rule(model, i, j): + return model.y[i, j] <= model.x[i] +model.y_le_source = pyo.Constraint(model.ORDERED_PAIRS, rule=y_le_source_rule) + +def y_le_target_rule(model, i, j): + return model.y[i, j] <= model.x[j] +model.y_le_target = pyo.Constraint(model.ORDERED_PAIRS, rule=y_le_target_rule) + +def y_ge_both_rule(model, i, j): + return model.y[i, j] >= model.x[i] + model.x[j] - 1 +model.y_ge_both = pyo.Constraint(model.ORDERED_PAIRS, rule=y_ge_both_rule) + +# 3. Add Objective +def obj_rule(model): + return sum(model.SCORE[i, j] * model.y[i, j] for (i, j) in model.ORDERED_PAIRS) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.maximize) + +# 4. Solve +solver = pyo.SolverFactory('gurobi') # or 'highs' +solver.options['TimeLimit'] = 30 +solver.options['MIPGap'] = 0.0 +solver.options['Threads'] = 4 +solver.options['Seed'] = 42 + +results = solver.solve(model, tee=False) + +# 5. Check Status and Extract Results +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition == pyo.TerminationCondition.optimal): + # Solution is optimal + objective_value = pyo.value(model.obj) + selected_items = [i for i in model.NODES if pyo.value(model.x[i]) > 0.5] + # ... extract other results +else: + # Handle non-optimal status (e.g., feasible, infeasible, error) + # Return structured output with status info +``` + +### Common Pitfalls +- Not checking solver status and termination condition before extracting results, leading to errors on infeasible or failed solves. +- Using excessive tool calls or solving the same core problem multiple times with minor variations instead of clarifying the problem interpretation first. +- Mixing analysis code (e.g., manual enumeration) with solver code in the same execution flow, creating maintenance complexity. + +# Workflow 2 (OR-Tools CP-SAT) + +## Modeling stage + +### Strategy Overview +Formulate the problem using the OR-Tools CP-SAT solver, which natively handles Boolean logic and linear constraints. This approach leverages the solver's efficient propagation and search algorithms for combinatorial problems. + +### Step 1 - Define Model and Data Structures +- Create a `cp_model.CpModel()` object. +- Store the list of items and the directed interaction scores in a dictionary or 2D list. +- Define the cardinality parameter `K`. + +### Step 2 - Define Boolean Selection Variables +- Create Boolean (0-1) selection variables `x[i]` for each item `i` using `model.NewBoolVar(f'x_{i}')`. + +### Step 3 - Define Boolean Interaction Variables +- Create Boolean interaction variables `y[i, j]` for each ordered pair `(i, j)` where `i != j` using `model.NewBoolVar(f'y_{i}_{j}')`. + +### Step 4 - Formulate Constraints via Linear Expressions +- Add a cardinality constraint: `sum(x[i] for all i) == K`. Use `model.Add(sum(x) == K)`. +- Add logical activation constraints using `model.Add(y[i,j] <= x[i])` and `model.Add(y[i,j] <= x[j])`. +- Add the forcing constraint using `model.Add(y[i,j] >= x[i] + x[j] - 1)`. + +### Step 5 - Define Objective +- Create a linear expression for the objective: `sum(score[i,j] * y[i,j] for all i,j)`. +- Maximize this expression using `model.Maximize(objective_expr)`. + +### Formulation Template +```json +{ + "sets": [ + {"name": "NODES", "description": "List of all candidate items."}, + {"name": "ORDERED_PAIRS", "description": "List of all ordered pairs (i, j) where i != j."} + ], + "parameters": [ + {"name": "SCORE", "index": "ORDERED_PAIRS", "description": "Directed interaction score from i to j."}, + {"name": "K", "description": "Exact number of items to select."} + ], + "decision_variables": [ + {"name": "x", "index": "NODES", "type": "BoolVar", "description": "1 if item i is selected."}, + {"name": "y", "index": "ORDERED_PAIRS", "type": "BoolVar", "description": "1 if directed interaction from i to j is active."} + ], + "objective": { + "sense": "max", + "expression": "LinearExpr.Sum( [SCORE[i,j] * y[i,j] for (i,j) in ORDERED_PAIRS] )" + }, + "constraints": [ + {"name": "select_exactly_k", "expression": "LinearExpr.Sum(x) == K" }, + {"name": "y_leq_source", "expression": "y[i,j] <= x[i]", "index": "ORDERED_PAIRS"}, + {"name": "y_leq_target", "expression": "y[i,j] <= x[j]", "index": "ORDERED_PAIRS"}, + {"name": "y_geq_both_selected", "expression": "y[i,j] >= x[i] + x[j] - 1", "index": "ORDERED_PAIRS"} + ] +} +``` + +### Common Pitfalls +- Misinterpreting the interaction score data structure, leading to incorrect objective coefficients. Ensure the score dictionary keys match the ordered pair variable indices. +- Overlooking the need for the forcing constraint, which is as critical in CP-SAT as in MILP for this linearization. +- Creating an unnecessarily large number of interaction variables for problems where the score matrix is very sparse; consider generating variables only for pairs with non-zero scores. + +## Solving stage + +### Strategy Overview +Build the CP-SAT model using the `ortools.sat.python.cp_model` API. Configure the solver with appropriate time limits and optional logging. Solve the model and implement comprehensive checks on the solver response before parsing the solution. + +### Step 1 - Model and Variable Instantiation +- Instantiate `cp_model.CpModel()`. +- Create dictionaries or lists to store `x[i]` and `y[(i,j)]` Boolean variables. + +### Step 2 - Constraint Addition +- Use `model.Add( sum(x.values()) == K )` for the cardinality constraint. +- Use loops over ordered pairs to add the three families of logical constraints via `model.Add()`. + +### Step 3 - Objective Definition and Solving +- Build the objective as a `LinearExpr` by summing `score[i,j] * y[i,j]`. +- Call `model.Maximize(objective)`. +- Create a `cp_model.CpSolver()` instance. +- Set solver parameters like `solver.parameters.max_time_in_seconds` and `solver.parameters.num_search_workers`. +- Execute the solve with `solver.Solve(model)`. + +### Step 4 - Solution Status Checking and Extraction +- Check the solver status: `cp_model.OPTIMAL`, `cp_model.FEASIBLE`, or `cp_model.INFEASIBLE`. +- For optimal or feasible status, retrieve the objective value via `solver.ObjectiveValue()`. +- Evaluate each variable using `solver.Value(var)` to determine selected items and active interactions. +- For infeasible or unknown status, return a clear status message without attempting to extract variable values. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# 1. Instantiate Model and Data +model = cp_model.CpModel() +items = ITEM_LIST +ordered_pairs = ORDERED_PAIRS_LIST +score = SCORE_DICT +K = K_VALUE + +# 2. Create Variables +x = {i: model.NewBoolVar(f'x_{i}') for i in items} +y = {(i, j): model.NewBoolVar(f'y_{i}_{j}') for (i, j) in ordered_pairs} + +# 3. Add Cardinality Constraint +model.Add(sum(x[i] for i in items) == K) + +# 4. Add Logical Constraints +for (i, j) in ordered_pairs: + model.Add(y[i, j] <= x[i]) + model.Add(y[i, j] <= x[j]) + model.Add(y[i, j] >= x[i] + x[j] - 1) + +# 5. Define and Set Objective +objective_expr = sum(score[i, j] * y[i, j] for (i, j) in ordered_pairs) +model.Maximize(objective_expr) + +# 6. Configure and Run Solver +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 4 +# Optional: Enable logging +# solver.parameters.log_search_progress = True + +status = solver.Solve(model) + +# 7. Process Results +if status in [cp_model.OPTIMAL, cp_model.FEASIBLE]: + objective_value = solver.ObjectiveValue() + selected_items = [i for i in items if solver.Value(x[i]) == 1] + active_interactions = [(i, j) for (i, j) in ordered_pairs if solver.Value(y[i, j]) == 1] + # ... proceed with results +else: + # status is INFEASIBLE or UNKNOWN + # Return structured output indicating status +``` + +### Common Pitfalls +- Ignoring solver error messages or status codes. Always check the status and handle `INFEASIBLE` and `UNKNOWN` cases explicitly. +- Implementing complex, solver-specific linearization when the problem size is trivial; for very small `N` and `K`, consider simple enumeration as a verification step. +- Making multiple solver calls with minor formulation changes without first resolving ambiguity in the problem's interpretation of interaction scores. diff --git a/skills/optskills/skill_library/cardinality_constrained_pairwise_selection.md b/skills/optskills/skill_library/cardinality_constrained_pairwise_selection.md new file mode 100644 index 0000000..adb1924 --- /dev/null +++ b/skills/optskills/skill_library/cardinality_constrained_pairwise_selection.md @@ -0,0 +1,265 @@ +--- +name: Cardinality-Constrained Pairwise Selection +description: | + Model and solve combinatorial problems where exactly K elements are selected from a set, and the objective maximizes the sum of pairwise benefits between selected elements, using linearized binary variables for pairwise activation. +--- + +# Workflow 1 (CP-SAT for Exact Binary Optimization) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a pure binary integer program using OR-Tools CP-SAT. The core modeling technique is the linearization of the product `x[i] * x[j]` via auxiliary binary variables and logical constraints, which CP-SAT handles natively and efficiently. + +### Step 1 - Define Core Decision Variables +- Create a binary variable `x[i]` for each element `i` in the set `N` to represent its selection status. +- Create a binary variable `y[(i, j)]` for each ordered pair `(i, j)` in the set `P` (where `i != j`) to represent the activation of the pairwise interaction. + +### Step 2 - Enforce Selection Cardinality +- Add a single linear equality constraint: `sum(x[i] for i in N) == K`. This ensures exactly `K` elements are selected. + +### Step 3 - Link Selection and Pairwise Activation +- For each ordered pair `(i, j)`, add three linear constraints to enforce `y[(i, j)] == x[i] * x[j]`: + - `y[(i, j)] <= x[i]` (activation requires the first element selected). + - `y[(i, j)] <= x[j]` (activation requires the second element selected). + - `y[(i, j)] >= x[i] + x[j] - 1` (activation is mandatory if both elements are selected). + +### Step 4 - Formulate the Objective +- Define the objective as `maximize sum(benefit[(i, j)] * y[(i, j)] for (i, j) in P)`, where `benefit` is a given parameter for each directed pair. + +### Formulation Template +```json +{ + "sets": [ + "N: Set of elements (e.g., nodes, items).", + "P: Set of ordered pairs (i, j) where i, j ∈ N and i ≠ j." + ], + "parameters": [ + "K: Integer, the exact number of elements to select.", + "benefit[(i, j)]: Numeric weight (can be positive or negative) for the directed pair (i, j)." + ], + "decision_variables": [ + "x[i] ∈ {0, 1}, ∀ i ∈ N. 1 if element i is selected.", + "y[(i, j)] ∈ {0, 1}, ∀ (i, j) ∈ P. 1 if the pairwise interaction is active." + ], + "objective": { + "sense": "max", + "expression": "sum_{ (i,j) ∈ P } benefit[(i, j)] * y[(i, j)]" + }, + "constraints": [ + "cardinality: sum_{ i ∈ N } x[i] = K", + "activation_lower_bound: y[(i, j)] ≤ x[i], ∀ (i, j) ∈ P", + "activation_lower_bound_2: y[(i, j)] ≤ x[j], ∀ (i, j) ∈ P", + "activation_upper_bound: y[(i, j)] ≥ x[i] + x[j] - 1, ∀ (i, j) ∈ P" + ] +} +``` + +### Common Pitfalls +- Forgetting to define `y` for *ordered* pairs when benefits are asymmetric, which leads to an incorrect objective. +- Applying the linearization constraints to unordered pairs without adjusting the objective, potentially double-counting or missing benefits. +- Not verifying that the `benefit` parameter is defined for all pairs in `P`; missing keys will cause an error during model construction. + +## Solving stage + +### Strategy Overview +Use the OR-Tools CP-SAT solver, configured for deterministic and efficient search on binary combinatorial problems. The workflow includes explicit solver parameter tuning, solution verification, and a fallback validation method for small instances. + +### Step 1 - Instantiate Model and Variables +- Create a `CpModel()` object. +- Use dictionaries to store `x` and `y` variables, creating them with `model.NewBoolVar(f"x_{i}")` and `model.NewBoolVar(f"y_{i}_{j}")` for traceability. + +### Step 2 - Add Constraints and Objective +- Translate the formulation constraints directly using the model's `Add()` method and linear expression capabilities. +- Set the objective with `model.Maximize()`. + +### Step 3 - Configure and Execute the Solver +- Instantiate a `CpSolver()`. +- Set key parameters: `solver.parameters.max_time_in_seconds = TIME_LIMIT`, `solver.parameters.num_search_workers = NUM_WORKERS`, `solver.parameters.random_seed = SEED`. For optimality proof, set `solver.parameters.relative_gap_limit = 0.0`. +- Call `solver.Solve(model)` and capture the status. + +### Step 4 - Extract and Verify Solution +- Check if the status is `OPTIMAL` or `FEASIBLE`. +- Extract the selected set: `selected = [i for i in N if solver.Value(x[i]) == 1]`. +- Verify cardinality constraint: `len(selected) == K`. +- For small `N`, optionally validate optimality via brute-force enumeration over all `K`-combinations to confirm the solver's solution value. + +### Code Usage +```python +# build model from formulation +from ortools.sat.python import cp_model +model = cp_model.CpModel() + +# Variable creation +x = {i: model.NewBoolVar(f"x_{i}") for i in N} +y = {(i, j): model.NewBoolVar(f"y_{i}_{j}") for (i, j) in P} + +# Cardinality constraint +model.Add(sum(x[i] for i in N) == K) + +# Pairwise consistency constraints +for (i, j) in P: + model.Add(y[(i, j)] <= x[i]) + model.Add(y[(i, j)] <= x[j]) + model.Add(y[(i, j)] >= x[i] + x[j] - 1) + +# Objective +model.Maximize(sum(benefit[(i, j)] * y[(i, j)] for (i, j) in P)) + +# solve with status / termination checks +solver = cp_model.CpSolver() +# Set parameters (e.g., solver.parameters.max_time_in_seconds = 30) +status = solver.Solve(model) + +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + selected = [i for i in N if solver.Value(x[i]) == 1] + objective_value = solver.ObjectiveValue() + # Further processing... +else: + # Handle no solution found + print("Solver did not find a feasible solution.") +``` + +### Common Pitfalls +- Not setting a `max_time_in_seconds` for large instances, risking excessively long runs. +- Misinterpreting the `status`; `FEASIBLE` does not guarantee optimality unless a time or gap limit was set. +- Attempting to use `solver.Value()` on a variable before checking the solver status, which may cause errors. + +# Workflow 2 (Pyomo with MILP Solver) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) using Pyomo's abstract modeling constructs. This approach provides solver-agnostic flexibility and explicit set-based definitions, suitable for integration with commercial (e.g., Gurobi) or open-source (e.g., HiGHS) solvers. + +### Step 1 - Declare Model and Sets +- Instantiate a `ConcreteModel()` or `AbstractModel()`. +- Define Pyomo `Set` objects for the elements `model.N` and the ordered pairs `model.P`. + +### Step 2 - Define Parameters and Variables +- Declare `Param` for `benefit` indexed over `model.P`. +- Declare binary `Var` for selection (`model.x`, indexed over `model.N`) and pairwise activation (`model.y`, indexed over `model.P`). + +### Step 3 - Construct Constraints via Rules +- Define a `Constraint` rule for the cardinality constraint: `sum(model.x[i] for i in model.N) == K`. +- For each pair in `model.P`, define three constraints using rules or a `ConstraintList` to implement the linearization: `model.y[i,j] <= model.x[i]`, `model.y[i,j] <= model.x[j]`, `model.y[i,j] >= model.x[i] + model.x[j] - 1`. + +### Step 4 - Define the Objective +- Use an `Objective` rule: `maximize sum(model.benefit[i,j] * model.y[i,j] for (i,j) in model.P)`. + +### Formulation Template +```json +{ + "sets": [ + "N: Pyomo Set of elements.", + "P: Pyomo Set of ordered pairs (i, j), a subset of N × N with i ≠ j." + ], + "parameters": [ + "K: Integer, cardinality requirement.", + "benefit[i,j]: Pyomo Param defined over P, representing the directed pairwise weight." + ], + "decision_variables": [ + "x[i]: Pyomo Var (domain=Binary), ∀ i ∈ N.", + "y[i,j]: Pyomo Var (domain=Binary), ∀ (i, j) ∈ P." + ], + "objective": { + "sense": "max", + "expression": "sum( benefit[i,j] * y[i,j] for (i,j) in P )" + }, + "constraints": [ + "cardinality_rule: sum( x[i] for i in N ) == K", + "link1_rule: y[i,j] <= x[i], ∀ (i,j) ∈ P", + "link2_rule: y[i,j] <= x[j], ∀ (i,j) ∈ P", + "link3_rule: y[i,j] >= x[i] + x[j] - 1, ∀ (i,j) ∈ P" + ] +} +``` + +### Common Pitfalls +- Defining the set `P` as unordered pairs (`i < j`) but using it with an asymmetric `benefit` parameter, leading to key errors or incorrect objective calculation. +- Using an `AbstractModel` without properly initializing all parameters via a data file or dictionary before instantiation, causing runtime errors. +- Creating overly complex constraint rules that perform unnecessary computations, slowing down model construction for large `N`. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a MILP solver interface (e.g., Gurobi, HiGHS). The workflow emphasizes robust solver configuration, careful handling of solution loading, and post-solution verification to ensure correctness and manage numerical precision. + +### Step 1 - Instantiate Solver and Set Options +- Use `SolverFactory('solver_name')` (e.g., `'gurobi'`, `'highs'`). +- Pass solver options via `options_dict` or keyword arguments (e.g., `TimeLimit=TIME_LIMIT`, `MIPGap=0.0` for optimality, `Threads=NUM_THREADS`, `Seed=SEED`). For HiGHS, note that `threads` may be set via a different option. + +### Step 2 - Solve and Check Termination Status +- Call `results = solver.solve(model, tee=False)`. +- Always check the solver status (`results.solver.status`) and termination condition (`results.solver.termination_condition`). Accept `optimal` or `feasible` as successful. + +### Step 3 - Load and Extract Solution +- If the solve was successful, load the solution into the model. For robustness, especially with HiGHS, use `model.solutions.load_from(results)`. +- Extract selected elements: `selected = [i for i in model.N if value(model.x[i]) > 0.5]`. +- Compute the realized objective by summing `benefit[i,j]` for pairs where both corresponding `x` variables are 1, to verify against the solver's reported objective value. + +### Step 4 - Validate Model Constraints +- Programmatically verify the cardinality constraint: `len(selected) == K`. +- Verify pairwise consistency: for all `(i,j)` in `P`, check that `value(model.y[i,j])` equals `value(model.x[i]) * value(model.x[j])` within a small tolerance. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo +model = pyo.ConcreteModel() + +# Sets +model.N = pyo.Set(initialize=N_set) +model.P = pyo.Set(initialize=P_set, dimen=2) + +# Parameters +model.K = pyo.Param(initialize=K, mutable=True) +model.benefit = pyo.Param(model.P, initialize=benefit_dict) + +# Variables +model.x = pyo.Var(model.N, domain=pyo.Binary) +model.y = pyo.Var(model.P, domain=pyo.Binary) + +# Objective +def obj_rule(m): + return sum(m.benefit[i, j] * m.y[i, j] for (i, j) in m.P) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.maximize) + +# Constraints +def cardinality_rule(m): + return sum(m.x[i] for i in m.N) == m.K +model.cardinality = pyo.Constraint(rule=cardinality_rule) + +def link_rule1(m, i, j): + return m.y[i, j] <= m.x[i] +model.link1 = pyo.Constraint(model.P, rule=link_rule1) + +def link_rule2(m, i, j): + return m.y[i, j] <= m.x[j] +model.link2 = pyo.Constraint(model.P, rule=link_rule2) + +def link_rule3(m, i, j): + return m.y[i, j] >= m.x[i] + m.x[j] - 1 +model.link3 = pyo.Constraint(model.P, rule=link_rule3) + +# solve with status / termination checks +solver = pyo.SolverFactory('solver_name') # e.g., 'gurobi' +solver_options = {'TimeLimit': TIME_LIMIT, 'MIPGap': 0.0} +results = solver.solve(model, options=solver_options) + +# Check status +if results.solver.termination_condition == pyo.TerminationCondition.optimal: + pyo.SolverFactory('solver_name').load_solutions(results, model) + selected = [i for i in model.N if pyo.value(model.x[i]) > 0.5] + # Further processing... +elif results.solver.termination_condition == pyo.TerminationCondition.feasible: + print("Feasible solution found, but not proven optimal.") + # Load solution and proceed with caution +else: + print("Solver failed to find a feasible solution.") +``` + +### Common Pitfalls +- Assuming `pyo.value()` can be called immediately after `solve()` without checking status or loading the solution, leading to `ValueError` or stale variable values. +- Ignoring numerical precision: comparing floating-point objective values directly for equality; use a tolerance or recalculate using integer arithmetic on the selected set. +- For HiGHS, setting the `threads` option when the solver is already multi-threaded via environment variables, which can cause conflicts or ignored parameters. diff --git a/skills/optskills/skill_library/constrained_assignment_with_min_max_objective.md b/skills/optskills/skill_library/constrained_assignment_with_min_max_objective.md new file mode 100644 index 0000000..c6327a2 --- /dev/null +++ b/skills/optskills/skill_library/constrained_assignment_with_min_max_objective.md @@ -0,0 +1,246 @@ +--- +name: Constrained Assignment with Min-Max Objective +description: | + Model and solve discrete assignment problems with domain restrictions, pairwise incompatibilities, and a min-max objective using either CP-SAT or MIP solvers. + +--- + +# Workflow 1 (CP-SAT with Integer Assignment Variables) + +## Modeling stage + +### Strategy Overview +This workflow models the problem using integer decision variables for each entity, directly representing the assigned discrete value. Domain restrictions are enforced via allowed value lists, and incompatibilities are modeled as simple inequalities. The CP-SAT solver is ideal for this discrete, combinatorial formulation. + +### Step 1 - Define Core Sets and Parameters +- Define a set of entities (e.g., `items`) to be assigned. +- Define a set of possible discrete values (e.g., `types`). +- For each entity, define a parameter `allowed_types[i]` listing its permitted values. +- Define a list of incompatible entity pairs `incompatible_pairs`. + +### Step 2 - Create Integer Decision Variables +- Create an integer variable `x[i]` for each entity `i`, representing its assigned type. +- Set the variable's domain to the overall range of possible types (e.g., `[min_type, max_type]`). + +### Step 3 - Enforce Domain Restrictions +- For each entity `i`, add a constraint `AddAllowedAssignments([x[i]], [allowed_types[i]])`. This efficiently restricts the variable's value to its specific allowed list. + +### Step 4 - Enforce Pairwise Incompatibilities +- For each incompatible pair `(i, j)`, add a constraint `x[i] != x[j]` to prevent them from receiving the same assignment. + +### Step 5 - Model the Min-Max Objective +- Create an auxiliary integer variable `max_assigned` to represent the maximum value assigned to any entity. +- For each entity `i`, add a constraint `max_assigned >= x[i]`. +- Set the objective to `Minimize(max_assigned)`. + +### Formulation Template +```json +{ + "sets": [ + "I: set of entities", + "T: set of all possible discrete values", + "P: set of incompatible pairs (i,j) where i,j in I" + ], + "parameters": [ + "allowed[i]: list of values in T permitted for entity i" + ], + "decision_variables": [ + "x[i] in T: assigned value for entity i", + "y: auxiliary variable for maximum assigned value" + ], + "objective": { + "sense": "min", + "expression": "y" + }, + "constraints": [ + "domain[i]: x[i] in allowed[i] for all i in I", + "incompatibility[(i,j)]: x[i] != x[j] for all (i,j) in P", + "max_def[i]: y >= x[i] for all i in I" + ] +} +``` + +### Common Pitfalls +- Creating inequality constraints for domain restrictions instead of using `AddAllowedAssignments`, which is less efficient. +- Forgetting to bound the auxiliary `max_assigned` variable, which can slow down the solver. +- Generating incompatibility constraints for all entity pairs instead of only the specified incompatible pairs. + +## Solving stage + +### Strategy Overview +Use the OR-Tools CP-SAT solver. Configure it for a balance of speed and proof of optimality. After solving, rigorously verify the solution against all constraints and, if needed, prove optimality via a feasibility check. + +### Step 1 - Configure and Run the Solver +- Instantiate a `CpSolver()` object. +- Set a time limit (`solver.parameters.max_time_in_seconds`). +- Enable parallel search (`solver.parameters.num_search_workers`). +- Set a random seed for reproducibility (`solver.parameters.random_seed`). +- Call `solver.Solve(model)` to obtain the status. + +### Step 2 - Check Solver Status and Extract Solution +- Check if `status` is `OPTIMAL` or `FEASIBLE`. +- If successful, retrieve the value of each `x[i]` and the objective `y` using `solver.Value(variable)`. +- If `status` is `INFEASIBLE`, handle the error appropriately without trying to load variable values. + +### Step 3 - Verify Solution Feasibility +- Programmatically verify that each entity's assigned value is in its `allowed[i]` list. +- Verify that for every incompatible pair, the assigned values are different. +- Verify that the reported objective `y` equals the maximum of all assigned `x[i]` values. + +### Step 4 - Prove Optimality (Optional) +- If the status is `OPTIMAL`, optimality is proven by the solver. +- To manually verify, create a feasibility model: add all constraints plus `y <= k-1`, where `k` is the found objective value. Solve; infeasibility confirms `k` is optimal. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# Build model from formulation +model = cp_model.CpModel() +# ... create variables and constraints as per modeling steps ... + +# Solve with status / termination checks +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 + +status = solver.Solve(model) + +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + assignments = {i: solver.Value(x[i]) for i in items} + obj_value = solver.Value(max_assigned) + # ... verification and output ... +else: + print("No feasible solution found.") +``` + +### Common Pitfalls +- Assuming `FEASIBLE` status guarantees optimality; it does not. +- Trying to access `solver.Value(variable)` when the status is not `OPTIMAL` or `FEASIBLE`, causing an error. +- Not verifying the solution, which can catch modeling errors even if the solver returns a status. + +# Workflow 2 (MIP with Binary Assignment Variables) + +## Modeling stage + +### Strategy Overview +This workflow uses a binary variable formulation, common in Mixed-Integer Programming. A binary variable `x[i,t]` indicates if entity `i` is assigned type `t`. This formulation naturally handles domain restrictions and incompatibilities via linear constraints and is suitable for MIP solvers. + +### Step 1 - Define Core Sets and Parameters +- Define a set of entities `I` and a set of types `T`. +- Define a parameter `allowed[i]` as a subset of `T` for each entity. +- Define a set of incompatible pairs `P`. + +### Step 2 - Create Binary Decision Variables +- Create a binary variable `x[i,t]` for each entity `i` and type `t`. + +### Step 3 - Enforce Single Assignment per Entity +- For each entity `i`, add a constraint `sum_{t in T} x[i,t] == 1`. + +### Step 4 - Enforce Domain Restrictions +- For each entity `i` and type `t` **not** in `allowed[i]`, fix the variable: `x[i,t] == 0`. This efficiently reduces the problem size. + +### Step 5 - Enforce Pairwise Incompatibilities +- For each incompatible pair `(i,j)` and for each type `t`, add a constraint `x[i,t] + x[j,t] <= 1`. This prevents both entities from being assigned the same type. + +### Step 6 - Model the Min-Max Objective +- Create a continuous variable `y` to represent the maximum type used. +- For each entity `i`, add a constraint `y >= sum_{t in T} (t * x[i,t])`. This linear expression captures the assigned type for entity `i`. +- Set the objective to `Minimize(y)`. + +### Formulation Template +```json +{ + "sets": [ + "I: set of entities", + "T: set of all possible discrete values", + "P: set of incompatible pairs (i,j) where i,j in I" + ], + "parameters": [ + "allowed[i]: subset of T permitted for entity i" + ], + "decision_variables": [ + "x[i,t] binary: 1 if entity i is assigned type t", + "y continuous: auxiliary variable for maximum assigned value" + ], + "objective": { + "sense": "min", + "expression": "y" + }, + "constraints": [ + "assignment[i]: sum_{t in T} x[i,t] == 1 for all i in I", + "domain[i,t]: x[i,t] == 0 for all i in I, t not in allowed[i]", + "incompatibility[(i,j),t]: x[i,t] + x[j,t] <= 1 for all (i,j) in P, t in T", + "max_def[i]: y >= sum_{t in T} (t * x[i,t]) for all i in I" + ] +} +``` + +### Common Pitfalls +- Creating the incompatibility constraint for all `t in T` without first fixing `x[i,t]` to zero for disallowed types, which creates unnecessary constraints. +- Using an integer variable for `y` when a continuous variable is sufficient and often performs better. +- Forgetting to add the linear expression `sum_{t in T} (t * x[i,t])` to link the binary variables to the value of the assigned type. + +## Solving stage + +### Strategy Overview +Use a MIP solver (e.g., CBC, Gurobi, SCIP). Configure it for exact solution finding. After solving, check both the high-level solver status and the detailed termination condition. Extract and verify the assignment from the binary variables. + +### Step 1 - Configure and Run the Solver +- Instantiate the appropriate solver (e.g., `pyo.SolverFactory('cbc')`). +- Set a time limit (`options={'seconds': TIMEOUT}`). +- Set `MIPGap=0.0` to search for an optimal solution. +- Configure threads for parallelism if supported. +- Set a random seed for reproducibility if supported. +- Call `solver.solve(model, options=...)`. + +### Step 2 - Check Solver Status and Termination +- Check the solver's high-level status (e.g., `pyo.check_optimal_termination(results)`). +- Also check the solver's termination condition from the results object to distinguish between optimal and feasible solutions. +- Do not attempt to load variable values if the termination condition indicates infeasibility. + +### Step 3 - Extract and Interpret the Solution +- If the termination is acceptable, retrieve variable values. +- For each entity `i`, find the type `t` where `x[i,t].value > 0.5`. This is the assigned type. +- Retrieve the objective value `y.value`. + +### Step 4 - Verify Solution and Prove Optimality +- Verify each assignment is within the entity's `allowed[i]` set. +- Verify no incompatible pair shares the same type. +- To prove optimality, solve a feasibility model with an added constraint `y <= k-1` (where `k` is the found objective). Infeasibility confirms optimality. + +### Code Usage +```python +import pyomo.environ as pyo + +# build model from formulation +model = pyo.ConcreteModel() +# ... create sets, variables, and constraints as per modeling steps ... + +# solve with status / termination checks +solver = pyo.SolverFactory('cbc') +options = {'seconds': 30, 'threads': 8, 'randomSeed': 42} +results = solver.solve(model, options=options) + +# Check termination condition +if results.solver.termination_condition == pyo.TerminationCondition.optimal: + status = "Optimal" +elif results.solver.termination_condition == pyo.TerminationCondition.feasible: + status = "Feasible" +else: + status = "Failed" + +if status in ("Optimal", "Feasible"): + # Load solution and process + # ... extract assignments from x[i,t].value ... + obj_value = pyo.value(model.y) + # ... verification and output ... +else: + print("No feasible solution found.") +``` + +### Common Pitfalls +- Relying only on `solver.status` being `ok` without checking `termination_condition`, which may mask suboptimal or infeasible results. +- Attempting to access `.value` on variables before ensuring a solution was found and loaded. +- Manually implementing a binary search for the optimal min-max value instead of letting the solver minimize `y` directly. diff --git a/skills/optskills/skill_library/constrained_cardinality_assignment.md b/skills/optskills/skill_library/constrained_cardinality_assignment.md new file mode 100644 index 0000000..aa7aa05 --- /dev/null +++ b/skills/optskills/skill_library/constrained_cardinality_assignment.md @@ -0,0 +1,284 @@ +--- +name: Constrained Cardinality Assignment +description: | + Model and solve binary assignment problems with cardinality constraints, assignment limits, conditional exclusions, and linear cost minimization using MILP solvers. + +--- +# Workflow 1 (Pyomo with Commercial Solver) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) using Pyomo's high-level modeling constructs. This approach leverages the expressive power of algebraic modeling languages to cleanly separate model logic from solver interaction, suitable for integration with commercial solvers like Gurobi or CPLEX. + +### Step 1 - Define Sets and Parameters +- Define two index sets, `SET_A` and `SET_B`, representing the source and target elements for assignment. +- Define a parameter `cost[a,b]` as a dictionary or 2D array representing the cost of assigning element `a` to element `b`. +- Define a scalar parameter `K` for the exact number of total assignments required. + +### Step 2 - Create Binary Decision Variables +- Create a binary variable `x[a,b]` for each pair `(a,b)` in the Cartesian product of `SET_A` and `SET_B`. +- The variable equals 1 if element `a` is assigned to element `b`, and 0 otherwise. + +### Step 3 - Formulate Assignment Limit Constraints +- For each element `a` in `SET_A`, add a constraint: `sum(x[a,b] for b in SET_B) <= 1`. This ensures each source element is assigned to at most one target. +- For each element `b` in `SET_B`, add a constraint: `sum(x[a,b] for a in SET_A) <= 1`. This ensures each target element receives at most one assignment. + +### Step 4 - Formulate Global Cardinality Constraint +- Add a single constraint: `sum(x[a,b] for a in SET_A for b in SET_B) == K`. This enforces the exact total number of assignments. + +### Step 5 - Formulate Conditional Exclusion Constraints +- For each logical rule "if assignment (a1,b1) is selected, then assignment (a2,b2) cannot be selected", add a linear constraint: `x[a1,b1] + x[a2,b2] <= 1`. +- Compile all such pairwise incompatibilities into a list of tuples for systematic constraint generation. + +### Step 6 - Define the Objective Function +- Define the objective to minimize total assignment cost: `minimize sum(cost[a,b] * x[a,b] for a in SET_A for b in SET_B)`. + +### Formulation Template +```json +{ + "sets": ["SET_A", "SET_B"], + "parameters": ["cost[SET_A, SET_B]", "K"], + "decision_variables": ["x[SET_A, SET_B] ∈ {0,1}"], + "objective": { + "sense": "min", + "expression": "sum(cost[a,b] * x[a,b] for a in SET_A for b in SET_B)" + }, + "constraints": [ + "sum(x[a,b] for b in SET_B) <= 1, ∀ a ∈ SET_A", + "sum(x[a,b] for a in SET_A) <= 1, ∀ b ∈ SET_B", + "sum(x[a,b] for a in SET_A for b in SET_B) == K", + "x[a1,b1] + x[a2,b2] <= 1, ∀ (a1,b1,a2,b2) ∈ EXCLUSION_PAIRS" + ] +} +``` + +### Common Pitfalls +- Using floating-point numbers directly in the cost parameter for an integer-only solver can cause precision issues; scale and convert to integers if necessary. +- Formulating conditional exclusions as separate `if-then` logic instead of the linear inequality `x[a1,b1] + x[a2,b2] <= 1`. +- Omitting the cardinality constraint (`K`) or mis-specifying it as an inequality, which changes the problem's feasible region. +- Not explicitly defining index sets, leading to less readable and maintainable model code. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a commercial MILP solver (e.g., Gurobi) via the `SolverFactory` interface. Configure the solver for deterministic performance, enforce optimality, and implement robust solution extraction and error handling. + +### Step 1 - Instantiate Solver and Set Parameters +- Create a solver object using `SolverFactory('gurobi')`. +- Set solver parameters for reproducibility and performance: `TimeLimit`, `MIPGap=0.0` (for exact optimality), `Threads`, and `Seed`. + +### Step 2 - Solve and Check Status +- Execute `solver.solve(model)` and capture the results object. +- Check the high-level solver status (`results.solver.status == SolverStatus.ok`). +- Check the termination condition (`results.solver.termination_condition`). Accept `optimal` or `feasible` as successful. + +### Step 3 - Extract and Validate Solution +- If successful, iterate over all `x[a,b]` variables. Collect assignments where `pyo.value(x[a,b]) > 0.5`. +- Recalculate the objective value from the extracted assignments and the `cost` parameter as a sanity check. +- Verify that the number of assignments equals `K` and that all limit and exclusion constraints are satisfied. + +### Step 4 - Format and Output Results +- Output a standardized result line, e.g., `RESULT:{objective_value}`. +- Output a detailed list of assignments as tuples `(a, b, cost)`. +- For programmatic use, package results into a structured dictionary or JSON object. + +### Step 5 - Handle Failures +- If the status is not `ok` or termination is `infeasible`, `unbounded`, or `invalid`, output a diagnostic payload containing the solver status and termination condition. +- Do not attempt to extract variable values from failed solves. +- Wrap solver execution in a try-except block to catch runtime errors (e.g., solver license issues, timeouts) and output a clear failure message. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Build model (model definition code from modeling stage) +model = pyo.ConcreteModel() +# ... populate model with sets, variables, constraints, objective + +# Solve +solver = pyo.SolverFactory('gurobi') +solver.options['TimeLimit'] = TIME_LIMIT +solver.options['MIPGap'] = 0.0 +solver.options['Threads'] = N_THREADS +solver.options['Seed'] = SEED +try: + results = solver.solve(model) +except Exception as e: + print(f"FAILURE: solver execution error - {e}") + exit() + +# Check status and extract results +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in [TerminationCondition.optimal, TerminationCondition.feasible]): + # Extract assignments + assignments = [] + for idx in model.x: + if pyo.value(model.x[idx]) > 0.5: + a, b = idx + assignments.append((a, b, model.cost[idx])) + obj_val = sum(c for (_, _, c) in assignments) + print(f"RESULT:{obj_val}") + print(f"Assignments: {assignments}") +else: + # Handle failure + print(f"FAILURE: status={results.solver.status}, termination={results.solver.termination_condition}") +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition`, leading to misinterpretation of suboptimal or incomplete solves. +- Using a loose tolerance (e.g., `> 0.1`) for extracting binary variable values; use `> 0.5` for robustness. +- Setting an excessive number of parallel threads (`Threads`) for small problems, which can degrade performance. +- Failing to provide informative output for infeasible or error cases, complicating debugging. +- Not wrapping solver execution in a try-except block, causing silent failures on runtime errors. + +# Workflow 2 (Python-MIP with CBC) + +## Modeling stage + +### Strategy Overview +Formulate the problem directly using the Python-MIP library, which provides a lower-level, solver-oriented API for constructing MILP models. This workflow is optimized for the open-source CBC solver, offering fine-grained control and efficient model building with a concise syntax. + +### Step 1 - Initialize Model and Add Variables +- Create a `mip.Model` object with `sense=MINIMIZE`. +- In a single loop over the Cartesian product of sets `SET_A` and `SET_B`, add binary variables `x[a,b]` using `model.add_var(var_type=BINARY)`. Store them in a dictionary keyed by `(a,b)`. + +### Step 2 - Build Assignment Limit Constraints +- For each `a` in `SET_A`, create a constraint: `mip.xsum(x[a,b] for b in SET_B) <= 1`. +- For each `b` in `SET_B`, create a constraint: `mip.xsum(x[a,b] for a in SET_A) <= 1`. +- Add each constraint directly to the model using `model.add_constr(...)`. + +### Step 3 - Build Global Cardinality Constraint +- Create a single constraint: `mip.xsum(x[a,b] for a in SET_A for b in SET_B) == K` and add it to the model. + +### Step 4 - Build Conditional Exclusion Constraints +- Iterate through a pre-defined list of incompatible assignment pairs `(a1,b1,a2,b2)`. +- For each pair, add a constraint: `x[a1,b1] + x[a2,b2] <= 1`. + +### Step 5 - Set the Objective Function +- Build the objective expression as `mip.xsum(cost[a,b] * x[a,b] for a in SET_A for b in SET_B)`. +- Assign this expression to `model.objective`. + +### Formulation Template +```json +{ + "sets": ["SET_A", "SET_B"], + "parameters": ["cost[SET_A, SET_B]", "K"], + "decision_variables": ["x[SET_A, SET_B] ∈ {0,1}"], + "objective": { + "sense": "min", + "expression": "sum(cost[a,b] * x[a,b] for a in SET_A for b in SET_B)" + }, + "constraints": [ + "sum(x[a,b] for b in SET_B) <= 1, ∀ a ∈ SET_A", + "sum(x[a,b] for a in SET_A) <= 1, ∀ b ∈ SET_B", + "sum(x[a,b] for a in SET_A for b in SET_B) == K", + "x[a1,b1] + x[a2,b2] <= 1, ∀ (a1,b1,a2,b2) ∈ EXCLUSION_PAIRS" + ] +} +``` + +### Common Pitfalls +- Forgetting to set the model sense (`MINIMIZE`) during initialization. +- Using Python's built-in `sum()` instead of `mip.xsum()` for linear expressions, which is less efficient and can cause performance issues in Python-MIP. +- Storing variable references in a list instead of a dictionary keyed by indices, making constraint construction more error-prone. +- Adding constraints inside deeply nested loops without pre-computing index pairs, hurting model build time. + +## Solving stage + +### Strategy Overview +Solve the model using the integrated CBC solver via Python-MIP's `optimize()` method. Leverage Python-MIP's direct solution query methods and implement parsing logic that handles the solver's status codes directly. + +### Step 1 - Configure and Run Solver +- Set solver parameters via `model.verbose`, `model.max_seconds`, `model.threads`, and `model.max_mip_gap`. +- Call `model.optimize()` to initiate the solve. + +### Step 2 - Interpret Solve Status +- Check `model.status`. Accept `Status.OPTIMAL` or `Status.FEASIBLE` as successful. +- For `Status.OPTIMAL`, the best solution found is proven optimal. For `Status.FEASIBLE`, it is a valid but not necessarily optimal solution. + +### Step 3 - Extract Solution +- If the status is acceptable, iterate through the dictionary of `x` variables. +- Use `var.x` to get the solution value (approximately 0.0 or 1.0). Collect assignments where `var.x > 0.5`. +- Compute the total cost from the extracted assignments and the `cost` dictionary. + +### Step 4 - Output Structured Results +- Output the objective value and list of assignments in a consistent, parseable format. +- Include the solve status and gap information in the output for traceability. + +### Step 5 - Manage Infeasibility and Errors +- If `model.status` is `Status.INFEASIBLE` or `Status.ERROR`, output a clear error message with the status. +- Avoid accessing `var.x` when no solution is available. +- Wrap the `model.optimize()` call in a try-except block to catch runtime errors (e.g., solver not found, memory issues) and output a clear failure message. + +### Code Usage +```python +from mip import Model, MINIMIZE, BINARY, Status + +# Build model +model = Model(sense=MINIMIZE) +x = {} +for a in SET_A: + for b in SET_B: + x[(a, b)] = model.add_var(var_type=BINARY) + +# Add constraints (constraint addition code from modeling stage) +# ... +# Set objective +model.objective = mip.xsum(cost[a, b] * x[(a, b)] for a in SET_A for b in SET_B) + +# Solve +model.verbose = 1 # Set to 0 to suppress solver log +model.max_seconds = TIME_LIMIT +model.threads = N_THREADS +model.max_mip_gap = 0.0 +try: + model.optimize() +except Exception as e: + print(f"FAILURE: solver execution error - {e}") + exit() + +# Check status and extract results +if model.status in [Status.OPTIMAL, Status.FEASIBLE]: + assignments = [] + for (a, b), var in x.items(): + if var.x > 0.5: + assignments.append((a, b, cost[(a, b)])) + obj_val = sum(c for (_, _, c) in assignments) + print(f"RESULT:{obj_val}") + print(f"Assignments: {assignments}") + print(f"Status: {model.status}, Gap: {model.gap}") +else: + print(f"FAILURE: status={model.status}") +``` + +### Common Pitfalls +- Misinterpreting `Status.FEASIBLE` as optimal; check `model.status` explicitly. +- Not setting `model.max_mip_gap = 0.0` when an exact optimal solution is required. +- Accessing `var.x` before checking `model.status`, which may raise an exception if no solution exists. +- Setting `model.verbose = 1` in production code, which can clutter logs; control verbosity based on context. +- Not wrapping solver execution in a try-except block, causing silent failures on runtime errors. + +## Prerequisite Checks (Apply Before Any Workflow) + +### Step 1 - Perform a Single Comprehensive Environment Check +- In one code block, verify the availability of all candidate solvers and libraries (e.g., `pyomo`, `gurobi`, `mip`, `cbc`). +- Use `import` statements within try-except blocks to detect missing packages. +- For solvers, attempt to instantiate a solver object (e.g., `SolverFactory('gurobi')`) and catch import or licensing errors. +- Based on the results, select the appropriate workflow (Pyomo with Gurobi/CPLEX, or Python-MIP with CBC) and proceed directly to model building without re-checking. + +### Step 2 - Validate Input Data Before Modeling +- Ensure `SET_A` and `SET_B` are non-empty lists or sets. +- Ensure `cost` dictionary covers all pairs `(a,b)` in the Cartesian product of `SET_A` and `SET_B`. +- Ensure `K` is an integer between 0 and `min(len(SET_A), len(SET_B))`. +- If any validation fails, output a clear error message and abort. + +### Step 3 - Fallback Guidance +- If no solver is available, output a clear diagnostic message listing which solvers were attempted and why they failed. +- Do not attempt to build or solve a model without a confirmed solver. + +## Performance and Validation Notes +- For small to medium-sized assignment problems (e.g., up to hundreds of agents/tasks), this MILP formulation solves quickly (often in < 0.1 seconds) and guarantees the global optimum. +- After solving, always verify the extracted assignment count equals `K` and that all assignment limit and exclusion constraints are satisfied as a sanity check. +- When the number of source elements equals the number of target elements, the `<= 1` assignment limit constraints combined with the cardinality constraint `== K` naturally enforce a perfect one-to-one matching. For unbalanced problems, adjust `K` accordingly or add dummy elements with zero cost. diff --git a/skills/optskills/skill_library/contiguous_interval_packing_with_non_overlap.md b/skills/optskills/skill_library/contiguous_interval_packing_with_non_overlap.md new file mode 100644 index 0000000..4611888 --- /dev/null +++ b/skills/optskills/skill_library/contiguous_interval_packing_with_non_overlap.md @@ -0,0 +1,239 @@ +--- +name: Contiguous Interval Packing with Non-Overlap +description: | + Model and solve 1D contiguous interval packing problems with fixed lengths and pairwise non-overlap constraints to minimize the maximum used position. +--- + +# Workflow 1 (CP-SAT with Disjunctive Logic) + +## Modeling stage + +### Strategy Overview +Formulate the problem using a CP-SAT solver's native integer variables and logical constraints. The core challenge is encoding pairwise non-overlap as a disjunction, which is efficiently handled via auxiliary Boolean variables and implications. + +### Step 1 - Define Core Variables and Domains +- Define an integer decision variable `start[i]` for each interval `i`, with a lower bound of `0` and an upper bound `UB` (e.g., sum of all interval lengths). +- Define an integer decision variable `end[i]` for each interval `i`, linked to `start[i]` via a length constraint. +- Define an auxiliary integer variable `max_pos` to capture the objective. + +### Step 2 - Enforce Interval Lengths +- For each interval `i`, add the constraint `end[i] == start[i] + length[i]`, where `length[i]` is a fixed parameter. + +### Step 3 - Enforce Pairwise Non-Overlap +- For each unordered pair `(i, j)` that must not overlap, create a Boolean variable `precedes_ij`. +- Add the implication: `precedes_ij == True` → `end[i] <= start[j]`. +- Add the implication: `precedes_ij == False` → `end[j] <= start[i]`. + +### Step 4 - Define the Objective +- Add constraints `max_pos >= end[i]` for all intervals `i`. +- Set the objective to minimize `max_pos`. + +### Formulation Template +```json +{ + "sets": [ + {"name": "I", "description": "Set of intervals/tasks"}, + {"name": "P", "description": "Set of non-overlap pairs (i,j) where i < j"} + ], + "parameters": [ + {"name": "length_i", "for": "i in I", "type": "int", "description": "Fixed length of interval i"} + ], + "decision_variables": [ + {"name": "start_i", "for": "i in I", "type": "int", "lb": 0, "ub": "UB"}, + {"name": "end_i", "for": "i in I", "type": "int", "lb": 0, "ub": "UB"}, + {"name": "max_pos", "type": "int", "lb": 0, "ub": "UB"}, + {"name": "precedes_ij", "for": "(i,j) in P", "type": "bool", "description": "True if interval i ends before j starts"} + ], + "objective": { + "sense": "min", + "expression": "max_pos" + }, + "constraints": [ + {"name": "interval_length", "for": "i in I", "formula": "end_i == start_i + length_i"}, + {"name": "max_pos_bound", "for": "i in I", "formula": "max_pos >= end_i"}, + {"name": "non_overlap", "for": "(i,j) in P", "formula": "(precedes_ij -> (end_i <= start_j)) AND ((not precedes_ij) -> (end_j <= start_i))"} + ] +} +``` + +### Common Pitfalls +- Setting the upper bound `UB` too small, which can make the model infeasible. Use a safe over-estimate like the sum of all lengths. +- Creating duplicate non-overlap constraints for both `(i,j)` and `(j,i)`, which wastes resources. Define the pair set `P` to contain each unordered pair only once. +- Using the same `UB` as the Big-M in logical implications, which can lead to overly loose constraints. For CP-SAT, the `OnlyEnforceIf` pattern does not require a Big-M constant. + +## Solving stage + +### Strategy Overview +Use a CP-SAT solver (e.g., OR-Tools CP-SAT) to find an optimal assignment. Leverage its native support for integer variables, Boolean logic, and parallel search. Always verify solver status and solution feasibility. + +### Step 1 - Solver Configuration +- Instantiate the CP-SAT solver. +- Set a time limit (e.g., `[TIME_LIMIT]` seconds) if needed. +- Configure parallel search (`num_search_workers`) for performance on larger instances. +- Set a `random_seed` for reproducibility. + +### Step 2 - Solve and Check Status +- Invoke the solver's `Solve` method with the model and objective. +- Check the returned status (`OPTIMAL`, `FEASIBLE`, `INFEASIBLE`, etc.). +- Proceed only if status indicates a feasible solution was found. + +### Step 3 - Extract and Validate Solution +- If feasible, retrieve the value of each `start_i` and `end_i` variable. +- Compute `max_pos` from the retrieved `end_i` values to verify it matches the solver's objective value. +- Perform a sanity check: verify all length and non-overlap constraints are satisfied by the extracted values. + +### Step 4 - Optimality Verification (Optional) +- To prove optimality, add a constraint `max_pos <= best_found_value - 1` and attempt to solve. Infeasibility confirms the original solution is optimal. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# Build model from formulation +model = cp_model.CpModel() +# ... create variables and constraints as per modeling stage ... + +# Solve with status / termination checks +solver = cp_model.CpSolver() +# Set solver parameters +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 + +status = solver.Solve(model) + +# Check status and extract solution +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + solution = {} + for i in I: + solution[i] = { + 'start': solver.Value(start_var[i]), + 'end': solver.Value(end_var[i]) + } + objective_value = solver.ObjectiveValue() + # Optional: verify constraints +else: + print("No feasible solution found.") +``` + +### Common Pitfalls +- Not checking solver status before accessing variable values, which can cause runtime errors. +- Forgetting to set `num_search_workers` for parallel search, leaving performance on the table. +- Adding optimality verification constraints directly to the model object without using a copy or a separate model, which can corrupt the original model for subsequent use. + +# Workflow 2 (MIP with Big-M Disjunction) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Program (MIP) using binary variables to activate disjunctive constraints. This approach is solver-agnostic and uses a large constant (Big-M) to enforce conditional logic, suitable for traditional MIP solvers. + +### Step 1 - Define Core Variables +- Define continuous or integer decision variables `start[i]` and `end[i]` for each interval `i`, with bounds `[0, UB]`. +- Define a continuous variable `max_pos` for the objective. + +### Step 2 - Enforce Interval Lengths +- For each interval `i`, add the constraint `end[i] == start[i] + length[i]`. + +### Step 3 - Enforce Pairwise Non-Overlap via Big-M +- For each non-overlap pair `(i, j)`, create a binary variable `z_ij`. +- Add constraint `end[i] <= start[j] + M * (1 - z_ij)`. When `z_ij = 1`, this enforces `end[i] <= start[j]`. +- Add constraint `end[j] <= start[i] + M * z_ij`. When `z_ij = 0`, this enforces `end[j] <= start[i]`. +- This ensures at least one ordering is active. + +### Step 4 - Define the Objective +- Add constraints `max_pos >= end[i]` for all `i`. +- Set the objective to minimize `max_pos`. + +### Formulation Template +```json +{ + "sets": [ + {"name": "I", "description": "Set of intervals"}, + {"name": "P", "description": "Set of non-overlap pairs (i,j) where i < j"} + ], + "parameters": [ + {"name": "length_i", "for": "i in I", "type": "float", "description": "Fixed length of interval i"}, + {"name": "M", "type": "float", "description": "Sufficiently large constant (Big-M)"} + ], + "decision_variables": [ + {"name": "start_i", "for": "i in I", "type": "continuous", "lb": 0, "ub": "UB"}, + {"name": "end_i", "for": "i in I", "type": "continuous", "lb": 0, "ub": "UB"}, + {"name": "max_pos", "type": "continuous", "lb": 0, "ub": "UB"}, + {"name": "z_ij", "for": "(i,j) in P", "type": "binary", "description": "1 if interval i must finish before j starts"} + ], + "objective": { + "sense": "min", + "expression": "max_pos" + }, + "constraints": [ + {"name": "interval_length", "for": "i in I", "formula": "end_i == start_i + length_i"}, + {"name": "max_pos_bound", "for": "i in I", "formula": "max_pos >= end_i"}, + {"name": "non_overlap_ij", "for": "(i,j) in P", "formula": "end_i <= start_j + M * (1 - z_ij)"}, + {"name": "non_overlap_ji", "for": "(i,j) in P", "formula": "end_j <= start_i + M * z_ij"} + ] +} +``` + +### Common Pitfalls +- Choosing a Big-M value (`M`) that is too small, making feasible solutions infeasible. `M` must be larger than the maximum possible span between any `start` and `end`. +- Choosing a Big-M value that is excessively large, which can cause numerical instability and slow convergence. Use the smallest valid upper bound (e.g., `UB`). +- Defining `z_ij` for both `(i,j)` and `(j,i)`, creating redundant variables and constraints. Define it only for ordered pairs `i < j`. + +## Solving stage + +### Strategy Overview +Use a traditional MIP solver (e.g., CBC, Gurobi, CPLEX). The Big-M formulation is linear and widely supported. Focus on proper Big-M calibration and solver tuning for performance. + +### Step 1 - Solver and Model Setup +- Instantiate the solver and create an empty model. +- Add all variables and constraints as defined. +- Set the objective sense to minimization. + +### Step 2 - Configure Solver Parameters +- Set a time limit (e.g., `[TIME_LIMIT]` seconds). +- Set optimality gap tolerance if appropriate. +- Enable presolve and cutting planes for better performance. +- Set thread count for parallel processing. + +### Step 3 - Solve and Interpret Status +- Invoke the solver's `optimize` method. +- Check the status: `OPTIMAL`, `FEASIBLE`, `INFEASIBLE`, etc. +- If status is not feasible, consider relaxing bounds or increasing `M`. + +### Step 4 - Extract and Verify Solution +- Retrieve variable values for `start_i`, `end_i`, and `max_pos`. +- Verify that `max_pos` equals the maximum retrieved `end_i`. +- Manually check a sample of non-overlap constraints using the retrieved values and the binary `z_ij` to ensure the Big-M logic held. + +### Code Usage +```python +import pulp # or gurobipy, ortools.linear_solver + +# Build model from formulation +prob = pulp.LpProblem("IntervalPacking", pulp.LpMinimize) +# ... create variables and constraints as per modeling stage ... + +# Solve with status / termination checks +solver = pulp.PULP_CBC_CMD(timeLimit=[TIME_LIMIT], threads=8) +prob.solve(solver) + +# Check status and extract solution +status = pulp.LpStatus[prob.status] +if status in ('Optimal', 'Feasible'): + solution = {} + for i in I: + solution[i] = { + 'start': start_var[i].varValue, + 'end': end_var[i].varValue + } + objective_value = pulp.value(prob.objective) + # Optional: verify constraints +else: + print("No feasible solution found. Status:", status) +``` + +### Common Pitfalls +- Not verifying the chosen `M` is sufficient for all possible solutions, leading to incorrect infeasibility. +- Adding feasibility-testing constraints (e.g., `max_pos <= bound`) directly to the model object without copying it first, which alters the original model. +- Ignoring numerical tolerances when checking constraint satisfaction, especially with large `M` values. Use a small epsilon for comparisons. diff --git a/skills/optskills/skill_library/contiguous_interval_span_minimization.md b/skills/optskills/skill_library/contiguous_interval_span_minimization.md new file mode 100644 index 0000000..3f049c7 --- /dev/null +++ b/skills/optskills/skill_library/contiguous_interval_span_minimization.md @@ -0,0 +1,257 @@ +--- +name: contiguous_interval_span_minimization +description: | + Models and solves problems requiring assignment of contiguous intervals to entities with non-overlap and adjacency interference constraints, minimizing the overall span using MILP or constraint programming. +--- + +# Workflow 1 (MILP with Big-M Disjunctive Constraints) + +## Modeling stage + +### Strategy Overview +Model each entity's interval using a single integer start variable. Enforce pairwise non-overlap via binary variables and big-M disjunctive constraints. Minimize the maximum end index across all intervals. + +### Step 1 - Define Interval Variables +- For each entity `i`, define an integer variable `start_index[i]` representing the starting point of its contiguous interval. +- The interval for entity `i` is `[start_index[i], start_index[i] + length[i] - 1]`. + +### Step 2 - Introduce Span Objective +- Define a continuous or integer variable `M` representing the maximum end index across all intervals. +- Add constraints `M >= start_index[i] + length[i] - 1` for all `i`. +- Set the objective to minimize `M`. + +### Step 3 - Encode Non-Overlap with Binary Variables +- For each pair `(i, j)` that must not overlap, introduce a binary variable `y[i,j]` indicating ordering (i before j if `y[i,j]=1`). +- Apply big-M disjunctive constraints: + - `start_index[i] + length[i] <= start_index[j] + K * (1 - y[i,j])` + - `start_index[j] + length[j] <= start_index[i] + K * y[i,j]` +- Choose `K` as a safe upper bound (e.g., sum of all interval lengths or a value larger than any feasible start index). + +### Step 4 - Add Adjacency Interference Constraints +- If adjacency interference is required (e.g., intervals cannot be adjacent), modify the non-overlap constraints to enforce a gap: + - `start_index[i] + length[i] + gap <= start_index[j] + K * (1 - y[i,j])` + - `start_index[j] + length[j] + gap <= start_index[i] + K * y[i,j]` +- Set `gap` to the minimum separation required (e.g., 1 for no adjacency). + +### Formulation Template +```json +{ + "sets": ["I: entities"], + "parameters": ["length[i]: length of interval for entity i", "K: big-M upper bound", "gap: minimum separation between intervals"], + "decision_variables": [ + "start_index[i] (integer): start point of interval for entity i", + "M (continuous): maximum end index", + "y[i,j] (binary): 1 if interval i is before interval j, for i= start_index[i] + length[i] - 1, for all i in I", + "start_index[i] + length[i] + gap <= start_index[j] + K * (1 - y[i,j]), for all i= m.start_index[i] + m.length[i] - 1 + model.max_end_con = pyo.Constraint(model.I, rule=max_end_rule) + + def non_overlap_rule(m, i, j): + if i >= j: + return pyo.Constraint.Skip + return (m.start_index[i] + m.length[i] + m.gap <= m.start_index[j] + m.K * (1 - m.y[i,j])) + model.non_overlap_con1 = pyo.Constraint(model.I, model.I, rule=non_overlap_rule) + + def non_overlap_rule2(m, i, j): + if i >= j: + return pyo.Constraint.Skip + return (m.start_index[j] + m.length[j] + m.gap <= m.start_index[i] + m.K * m.y[i,j]) + model.non_overlap_con2 = pyo.Constraint(model.I, model.I, rule=non_overlap_rule2) + + return model + +def solve_model(model, time_limit=60): + solver = pyo.SolverFactory('gurobi') + solver.options['TimeLimit'] = time_limit + solver.options['MIPGap'] = 0.0 + solver.options['Threads'] = 4 + result = solver.solve(model, tee=False) + if (result.solver.status == pyo.SolverStatus.ok and + result.solver.termination_condition in (pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible)): + print(f"RESULT:{float(pyo.value(model.obj))}") + else: + print('{"status": "error", "reason": "Solver did not find feasible solution"}') +``` + +### Common Pitfalls +- Trusting a non-zero return code or infeasible/unknown status; always check solver status explicitly. +- Outputting pseudo numeric answers when execution fails; use JSON error payloads instead. +- Forgetting to validate constraints after solving, which can catch numerical issues with big-M formulations. + +# Workflow 2 (Constraint Programming with NoOverlap) + +## Modeling stage + +### Strategy Overview +Use constraint programming (CP) with interval variables and a built-in `NoOverlap` constraint. Minimize the makespan (maximum end point) using a CP optimizer. + +### Step 1 - Define Interval Variables +- For each entity `i`, define an interval variable `interval[i]` with fixed size `length[i]`. +- The interval variable represents the contiguous assignment with start and end points determined by the solver. + +### Step 2 - Enforce Non-Overlap +- Use the `NoOverlap` constraint on the set of all interval variables to prevent any two intervals from overlapping. +- For adjacency interference, add a `Distance` constraint between interval ends and starts (e.g., `end_of(interval[i]) + gap <= start_of(interval[j])` for all pairs). + +### Step 3 - Minimize Makespan +- Define an integer variable `M` representing the maximum end point across all intervals. +- Add constraints `end_of(interval[i]) <= M` for all `i`. +- Set the objective to minimize `M`. + +### Step 4 - Optional Search Phase +- Specify a search phase to guide the solver (e.g., `SearchPhase(interval_vars, FirstFail, SelectSmallestValue)`). +- This can improve performance for large instances. + +### Formulation Template +```json +{ + "sets": ["I: entities"], + "parameters": ["length[i]: size of interval for entity i", "gap: minimum separation between intervals"], + "decision_variables": [ + "interval[i]: interval variable with size length[i]", + "M (integer): makespan variable" + ], + "objective": { + "sense": "min", + "expression": "M" + }, + "constraints": [ + "NoOverlap(interval[i] for all i in I)", + "end_of(interval[i]) + gap <= start_of(interval[j]) for all i 0: + for i in entities: + for j in entities: + if i < j: + model.Add(intervals[i].End + gap <= intervals[j].Start) + + # Makespan objective + makespan = model.NewIntVar(0, horizon, 'makespan') + for i in entities: + model.Add(intervals[i].End <= makespan) + model.Minimize(makespan) + + # Solve + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = time_limit + solver.parameters.num_search_workers = 4 + status = solver.Solve(model) + + if status == cp_model.OPTIMAL or status == cp_model.FEASIBLE: + print(f"RESULT:{solver.ObjectiveValue()}") + else: + print('{"status": "infeasible", "reason": "No feasible assignment exists"}') +``` + +### Common Pitfalls +- Not setting a time limit, causing the solver to run indefinitely on large instances. +- Ignoring the solver's log output, which can provide insights into search progress and bottlenecks. +- Assuming optimality without checking the status; always verify `OPTIMAL` or `FEASIBLE` before using results. diff --git a/skills/optskills/skill_library/cutting_stock_pattern_based_integer_optimization.md b/skills/optskills/skill_library/cutting_stock_pattern_based_integer_optimization.md new file mode 100644 index 0000000..1d117e4 --- /dev/null +++ b/skills/optskills/skill_library/cutting_stock_pattern_based_integer_optimization.md @@ -0,0 +1,268 @@ +--- +name: Cutting Stock - Pattern-Based Integer Optimization +description: | + Model and solve one-dimensional cutting stock problems by enumerating feasible patterns, formulating a pattern-based integer program to minimize total stock items used, and solving with a MIP solver. + +--- +# Workflow 1 (Pyomo with CBC/Highs) + +## Modeling stage + +### Strategy Overview +Use Pyomo for model definition, separating pattern generation from optimization. Employ a set-based formulation for clarity and scalability, suitable for problems with a moderate number of feasible patterns. + +### Step 1 - Generate Feasible Patterns +- For each item type `i`, compute `max_count_i = stock_length // item_length_i`. +- Use `itertools.product` over ranges `[0, max_count_i]` to generate all candidate combinations. +- Filter combinations where the total used length is positive (`> 0`) and does not exceed the stock capacity (`<= stock_length`). +- Store each valid pattern as a list of item counts and index them. + +### Step 2 - Define Model Sets and Parameters +- Define a set `P` for pattern indices and a set `I` for item types. +- Create a parameter `a[i, p]` mapping pattern `p` to the number of items of type `i` it produces. +- Initialize this parameter from the pre-generated pattern data using a dictionary comprehension. + +### Step 3 - Define Variables and Objective +- Define non-negative integer variables `x[p]` representing the usage count of each pattern `p`. +- Set the objective to minimize the sum of all `x[p]` (total stock items used). + +### Step 4 - Formulate Demand Constraints +- For each item type `i`, add a constraint ensuring total production meets or exceeds demand: `sum(a[i, p] * x[p] for p in P) >= demand[i]`. + +### Formulation Template +```json +{ + "sets": [ + "P: set of pattern indices", + "I: set of item types" + ], + "parameters": [ + "demand[i]: demand for item type i", + "a[i, p]: number of items of type i produced by pattern p" + ], + "decision_variables": [ + "x[p]: integer, non-negative, usage count of pattern p" + ], + "objective": { + "sense": "min", + "expression": "sum(x[p] for p in P)" + }, + "constraints": [ + "demand_satisfaction[i]: sum(a[i, p] * x[p] for p in P) >= demand[i], for all i in I" + ] +} +``` + +### Common Pitfalls +- Generating an excessive number of patterns for large problems, leading to intractable model size. +- Forgetting to filter out patterns with zero total used length, which are trivially feasible but useless. +- Using float equality checks (`==`) for demand constraints; use `>=` to allow overproduction. +- Not defining `x[p]` as `pyo.Integers` or `pyo.NonNegativeIntegers`, leading to a relaxed LP. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the CBC or Highs solver via the `SolverFactory`. Configure solver options for performance, verify solution status, and extract results with robust post-solution verification. + +### Step 1 - Configure and Execute Solver +- Instantiate the solver: `solver = pyo.SolverFactory('cbc')` or `pyo.SolverFactory('highs')`. +- Set key options: `time_limit`, `mip_rel_gap` (e.g., `0.0` for exact), and `threads`. +- Solve the model with `results = solver.solve(model, tee=False)`. + +### Step 2 - Verify Solution Status +- Check `results.solver.status` and `results.solver.termination_condition`. +- Accept solutions marked as `optimal` or `feasible`. Handle `infeasible` or `unbounded` statuses with appropriate error messages. + +### Step 3 - Extract and Verify Solution +- Iterate over pattern variables `x[p]` and collect those with value > 0.5 (accounting for solver tolerance). +- For each used pattern, record its composition and usage count. +- Recalculate total production per item type from the solution and verify it meets all demands. +- Compute secondary metrics: + - **Total Waste**: `sum(pattern_count * (stock_length - pattern_used_length))`. + - **Utilization Efficiency**: `(total_length_used / (total_stock_items_used * stock_length)) * 100`. + - **Theoretical Lower Bound**: `ceil(total_required_length / stock_length)` where `total_required_length = sum(demand[i] * item_lengths[i])`. Compare to the solution's total items used. + +### Step 4 - Output Interpretable Results +- Print a summary of used patterns, their counts, and associated waste. +- Compare the total items used to the theoretical lower bound to assess optimality. +- Use plain text markers (e.g., "(OK)") for robust output across environments. + +### Code Usage +```python +import pyomo.environ as pyo +import itertools + +# 1. Generate patterns (example structure) +patterns = [] # list of pattern lists +max_counts = [stock_length // length for length in item_lengths] +for counts in itertools.product(*[range(mc+1) for mc in max_counts]): + total = sum(c*l for c,l in zip(counts, item_lengths)) + if 0 < total <= stock_length: + patterns.append(list(counts)) + +# 2. Build model +model = pyo.ConcreteModel() +model.P = pyo.Set(initialize=range(len(patterns))) +model.I = pyo.Set(initialize=range(num_item_types)) + +def a_init(model, i, p): + return patterns[p][i] +model.a = pyo.Param(model.I, model.P, initialize=a_init) + +model.x = pyo.Var(model.P, domain=pyo.NonNegativeIntegers) + +def obj_rule(model): + return sum(model.x[p] for p in model.P) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + +def demand_rule(model, i): + return sum(model.a[i, p] * model.x[p] for p in model.P) >= demand[i] +model.demand_con = pyo.Constraint(model.I, rule=demand_rule) + +# 3. Solve +solver = pyo.SolverFactory('cbc') +solver.options['seconds'] = 30 +results = solver.solve(model) + +# 4. Check status and extract +if results.solver.termination_condition == pyo.TerminationCondition.optimal: + for p in model.P: + if pyo.value(model.x[p]) > 0.5: + print(f"Pattern {p}: Use {pyo.value(model.x[p])} times") +else: + print(f"Solver terminated with status: {results.solver.termination_condition}") +``` + +### Common Pitfalls +- Not checking solver status before extracting variable values, leading to errors. +- Using a loose MIP gap (`mip_rel_gap`) when an exact integer solution is required. +- Misinterpreting variable values due to solver tolerance; use a threshold (e.g., `> 0.5`) for integer variables. +- Forgetting to set `tee=True` during development for debugging solver progress. + +# Workflow 2 (OR-Tools with SCIP/CBC) + +## Modeling stage + +### Strategy Overview +Use Google's OR-Tools CP-SAT or MPSolver for a more procedural API. It is well-suited for integration into larger applications and offers fine-grained control over the solving process. + +### Step 1 - Generate Feasible Patterns +- Identical to Workflow 1: enumerate all item count combinations within stock capacity. +- Store patterns in a list-of-lists structure for efficient coefficient access. + +### Step 2 - Initialize Solver and Variables +- Create a solver instance: `solver = pywraplp.Solver.CreateSolver('SCIP')` or `'CBC'`. +- Define integer variables `x[j]` for each pattern `j` with a lower bound of 0 and no upper bound (`solver.IntVar` or `solver.NumVar` with integer=True). + +### Step 3 - Build Demand Constraints +- For each item type `i`, create a constraint object: `constraint = solver.Constraint(demand[i], solver.infinity())`. +- For each pattern `j`, set the coefficient: `constraint.SetCoefficient(x[j], pattern_item_count[i][j])`. + +### Step 4 - Define Objective Function +- Create the objective: `objective = solver.Objective()`. +- Set all variable coefficients in the objective to 1. +- Set the optimization sense to minimization. + +### Formulation Template +```json +{ + "sets": [ + "P: set of pattern indices", + "I: set of item types" + ], + "parameters": [ + "demand[i]: demand for item type i", + "a[i][p]: number of items of type i produced by pattern p (list of lists)" + ], + "decision_variables": [ + "x[p]: integer, non-negative, usage count of pattern p" + ], + "objective": { + "sense": "min", + "expression": "sum(x[p] for p in P)" + }, + "constraints": [ + "demand_satisfaction[i]: sum(a[i][p] * x[p] for p in P) >= demand[i], for all i in I" + ] +} +``` + +### Common Pitfalls +- Using `solver.NumVar` without setting `integer=True`, resulting in a continuous relaxation. +- Incorrectly setting constraint bounds; the lower bound should be the demand, and the upper bound `solver.infinity()` for a `>=` constraint. +- Not leveraging OR-Tools' pattern for efficient model construction (e.g., building constraints in loops). +- Assuming the solver automatically finds integer solutions; always specify the variable domain. + +## Solving stage + +### Strategy Overview +Solve the model using OR-Tools' solver, which provides a simple `Solve()` call. The focus is on efficient model construction, solution extraction, and verification. + +### Step 1 - Execute Solver +- Call `solver.Solve()` and capture the result status. + +### Step 2 - Interpret Solver Result +- Check the result status: `pywraplp.Solver.OPTIMAL` or `pywraplp.Solver.FEASIBLE` indicate a valid solution. +- Handle `INFEASIBLE` or `UNBOUNDED` statuses appropriately. + +### Step 3 - Extract Solution and Verify +- For each pattern variable `x[j]`, get its solution value using `x[j].solution_value()`. +- Filter patterns with value > 0.5. +- Recalculate total production per item type and verify against demands. +- Compute total waste and utilization efficiency (as defined in Workflow 1). + +### Step 4 - Output Production Instructions +- Print a detailed report listing each used pattern, its composition, usage count, and waste per unit. +- Summarize total stock items used and overall waste. +- Use plain text markers for robust output. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# 1. Generate patterns (example structure) +patterns = [] # list of pattern lists +max_counts = [stock_length // length for length in item_lengths] +for counts in itertools.product(*[range(mc+1) for mc in max_counts]): + total = sum(c*l for c,l in zip(counts, item_lengths)) + if 0 < total <= stock_length: + patterns.append(list(counts)) + +# 2. Initialize solver and variables +solver = pywraplp.Solver.CreateSolver('SCIP') +x = [] +for j in range(len(patterns)): + x.append(solver.IntVar(0, solver.infinity(), f'x_{j}')) + +# 3. Add demand constraints +for i in range(num_item_types): + constraint = solver.Constraint(demand[i], solver.infinity()) + for j in range(len(patterns)): + constraint.SetCoefficient(x[j], patterns[j][i]) + +# 4. Set objective +objective = solver.Objective() +for j in range(len(patterns)): + objective.SetCoefficient(x[j], 1) +objective.SetMinimization() + +# 5. Solve +status = solver.Solve() + +# 6. Extract and verify +if status == pywraplp.Solver.OPTIMAL or status == pywraplp.Solver.FEASIBLE: + used_patterns = [] + for j in range(len(patterns)): + val = x[j].solution_value() + if val > 0.5: + used_patterns.append((j, val, patterns[j])) + # ... verification and output ... +else: + print(f"Solver did not find a solution. Status: {status}") +``` + +### Common Pitfalls +- Not checking the solver status, assuming `Solve()` always returns an optimal solution. +- Forgetting that `solution_value()` returns a float; use a tolerance when checking for integer usage. +- Building the model inefficiently in nested loops for large problems; pre-compute coefficients where possible. +- Misusing `solver.infinity()` for constraint upper bounds when a finite bound exists. diff --git a/skills/optskills/skill_library/degree_constrained_minimum_spanning_tree_via_flow_based_connectivity.md b/skills/optskills/skill_library/degree_constrained_minimum_spanning_tree_via_flow_based_connectivity.md new file mode 100644 index 0000000..80c45e7 --- /dev/null +++ b/skills/optskills/skill_library/degree_constrained_minimum_spanning_tree_via_flow_based_connectivity.md @@ -0,0 +1,325 @@ +--- +name: Degree-Constrained Minimum Spanning Tree via Flow-Based Connectivity +description: | + Build and solve a degree-constrained minimum spanning tree problem using binary edge selection, single-commodity flow for connectivity, and degree constraints, with two solver workflows: one using Pyomo with HiGHS and another using OR-Tools CP-SAT. + +--- +# Workflow 1 (Pyomo with HiGHS) + +## Modeling stage + +### Strategy Overview +Model the degree-constrained minimum spanning tree as a mixed-integer linear program using binary edge selection variables for undirected edges. Enforce tree structure via an exact edge count constraint and single-commodity flow for connectivity. Degree constraints limit incident edges per node. + +### Step 1 - Define Sets and Parameters +- Define the set of nodes `N` and the set of undirected edges `E` as all unordered pairs `(i,j)` with `i < j`. +- Create a parameter `cost[i,j]` for each edge representing the edge weight. +- Define a parameter `max_degree[v]` for each node `v` in `N`. + +### Step 2 - Declare Decision Variables +- Create binary variables `x[i,j]` for each edge `(i,j)` in `E`, indicating whether the edge is selected. +- Create continuous non-negative flow variables `f[i,j]` for each directed arc `(i,j)` with `i != j`. For each undirected edge `(i,j)` in `E`, define both `f[i,j]` and `f[j,i]`. + +### Step 3 - Add Constraints +- **Tree cardinality**: `sum(x[i,j] for (i,j) in E) == |N| - 1` +- **Degree constraints**: For each node `v`, `sum(x[i,j] for (i,j) in E if i==v or j==v) <= max_degree[v]` +- **Flow capacity**: For each edge `(i,j)` in `E`, `f[i,j] <= (|N|-1) * x[i,j]` and `f[j,i] <= (|N|-1) * x[i,j]` +- **Flow conservation at root**: Choose an arbitrary root node `r` (e.g., node 0). `sum(f[r,j] for j != r) - sum(f[i,r] for i != r) == |N|-1` +- **Flow conservation at other nodes**: For each node `v != r`, `sum(f[i,v] for i != v) - sum(f[v,j] for j != v) == 1` + +### Step 4 - Define Objective +- Minimize total edge cost: `sum(cost[i,j] * x[i,j] for (i,j) in E)` + +### Formulation Template +```json +{ + "sets": ["N: nodes", "E: undirected edges (i,j) with i=0 for each directed arc (i,j) with i!=j" + ], + "objective": { + "sense": "min", + "expression": "sum(cost[i,j] * x[i,j] for (i,j) in E)" + }, + "constraints": [ + "sum(x[i,j] for (i,j) in E) == |N|-1", + "for each v in N: sum(x[i,j] for (i,j) in E if i==v or j==v) <= max_degree[v]", + "for each (i,j) in E: f[i,j] <= (|N|-1) * x[i,j]", + "for each (i,j) in E: f[j,i] <= (|N|-1) * x[i,j]", + "root r: sum(f[r,j] for j != r) - sum(f[i,r] for i != r) == |N|-1", + "for each v != r: sum(f[i,v] for i != v) - sum(f[v,j] for j != v) == 1" + ] +} +``` + +### Common Pitfalls +- Forgetting to define flow variables for both directions of each undirected edge, leading to connectivity failures. +- Using `i 0.5]` +- Validate: compute `len(selected_edges) == |N|-1`, check each node's degree against `max_degree`, and run BFS/DFS from root to confirm all nodes are reachable. + +### Code Usage +```python +import pyomo.environ as pyo + +def build_dcmst_model(N, edges, cost, max_degree, root=0): + model = pyo.ConcreteModel() + model.N = pyo.Set(initialize=N) + model.E = pyo.Set(initialize=edges, dimen=2) + + model.cost = pyo.Param(model.E, initialize=cost) + model.max_degree = pyo.Param(model.N, initialize=max_degree) + + model.x = pyo.Var(model.E, domain=pyo.Binary) + # Flow variables for all directed arcs + model.arcs = pyo.Set(initialize=[(i,j) for i in N for j in N if i != j], dimen=2) + model.f = pyo.Var(model.arcs, domain=pyo.NonNegativeReals) + + # Tree cardinality + model.cardinality = pyo.Constraint(expr=sum(model.x[i,j] for (i,j) in model.E) == len(N)-1) + + # Degree constraints + def degree_rule(m, v): + return sum(m.x[i,j] for (i,j) in m.E if i==v or j==v) <= m.max_degree[v] + model.degree = pyo.Constraint(model.N, rule=degree_rule) + + # Flow capacity + def flow_cap_rule(m, i, j): + return m.f[i,j] <= (len(N)-1) * m.x[i,j] + model.flow_cap = pyo.Constraint(model.E, rule=flow_cap_rule) + + def flow_cap_rev_rule(m, i, j): + return m.f[j,i] <= (len(N)-1) * m.x[i,j] + model.flow_cap_rev = pyo.Constraint(model.E, rule=flow_cap_rev_rule) + + # Flow conservation at root + root_out = sum(model.f[root,j] for j in N if j != root) + root_in = sum(model.f[i,root] for i in N if i != root) + model.root_flow = pyo.Constraint(expr=root_out - root_in == len(N)-1) + + # Flow conservation at other nodes + def flow_cons_rule(m, v): + if v == root: + return pyo.Constraint.Skip + inflow = sum(m.f[i,v] for i in N if i != v) + outflow = sum(m.f[v,j] for j in N if j != v) + return inflow - outflow == 1 + model.flow_cons = pyo.Constraint(model.N, rule=flow_cons_rule) + + # Objective + model.obj = pyo.Objective(expr=sum(model.cost[i,j] * model.x[i,j] for (i,j) in model.E), sense=pyo.minimize) + + return model + +# Solve +solver = pyo.SolverFactory("highs") +solver.options["time_limit"] = [TIME_LIMIT] +solver.options["mip_rel_gap"] = 0.0 +solver.options["threads"] = [NUM_THREADS] +solver.options["presolve"] = "on" +result = solver.solve(model, tee=True) + +# Check status +if result.solver.status == pyo.SolverStatus.ok and result.solver.termination_condition in (pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible): + selected = [(i,j) for (i,j) in model.E if pyo.value(model.x[i,j]) > 0.5] + obj_val = pyo.value(model.obj) + # Validate + assert len(selected) == len(N)-1 + # BFS connectivity check + adj = {v: [] for v in N} + for i,j in selected: + adj[i].append(j) + adj[j].append(i) + visited = set() + stack = [root] + while stack: + v = stack.pop() + if v not in visited: + visited.add(v) + stack.extend(adj[v]) + assert len(visited) == len(N) +else: + print("Solver failed:", result.solver.status, result.solver.termination_condition) +``` + +### Common Pitfalls +- Not checking `termination_condition` for `feasible` in addition to `optimal`, which may discard valid solutions when time limit is hit. +- Using `tee=False` during development, making it hard to diagnose solver issues. +- Forgetting to convert `pyo.value()` for flow variables when validating connectivity. + +# Workflow 2 (OR-Tools CP-SAT) + +## Modeling stage + +### Strategy Overview +Model the degree-constrained minimum spanning tree using OR-Tools CP-SAT with boolean variables for edge selection and integer flow variables. Leverage CP-SAT's native support for boolean variables and linear constraints. Use single-commodity flow for connectivity enforcement. + +### Step 1 - Define Data Structures +- Create a list of nodes `range(n)` and a list of undirected edges as tuples `(i,j)` with `i= var_i)` for every entity variable. +- Set objective: `model.Minimize(max_used)`. + +### Formulation Template +```json +{ + "sets": ["entities", "values"], + "parameters": ["allowed_values[entity]", "conflict_pairs"], + "decision_variables": [ + "assignment[entity] (integer, domain=allowed_values[entity])", + "max_used (integer, domain=[min_value, max_value])" + ], + "objective": { + "sense": "min", + "expression": "max_used" + }, + "constraints": [ + "assignment[i] != assignment[j] for each conflict pair (i,j)", + "max_used >= assignment[entity] for all entities" + ] +} +``` + +### Common Pitfalls +- Forgetting to restrict variable domains to allowed values, causing invalid assignments. +- Using `AddAllDifferent` instead of pairwise inequality constraints when conflicts are not all-to-all. +- Setting the auxiliary variable domain too narrow, causing infeasibility. + +## Solving stage + +### Strategy Overview +Use OR-Tools CP-SAT solver with appropriate search parameters. Check solver status and extract assignments with structured output. + +### Step 1 - Configure and Solve +- Create solver: `solver = cp_model.CpSolver()` +- Set parameters: `solver.parameters.max_time_in_seconds = [TIME_LIMIT]`, `solver.parameters.num_search_workers = 8`, optionally `solver.parameters.random_seed = 42` for reproducibility. +- Solve: `status = solver.Solve(model)` + +### Step 2 - Extract and Verify Results +- Check status: `status == cp_model.OPTIMAL` or `status == cp_model.FEASIBLE` +- Retrieve objective: `solver.ObjectiveValue()` +- Extract assignments: `solver.Value(assignment_var)` for each entity +- Validate that each assignment belongs to the entity's allowed set and that all conflict constraints are satisfied. + +### Step 3 - Output Structured Result +- Build dictionary with keys: `status`, `objective`, `assignments` +- Use `json.dumps()` for output + +### Code Usage +```python +from ortools.sat.python import cp_model + +model = cp_model.CpModel() +# Define variables +assignments = {} +for entity in entities: + min_val = min(allowed_values[entity]) + max_val = max(allowed_values[entity]) + var = model.NewIntVar(min_val, max_val, f"assign_{entity}") + model.AddAllowedAssignments([var], [[v] for v in allowed_values[entity]]) + assignments[entity] = var + +# Conflict constraints +for (i, j) in conflict_pairs: + model.Add(assignments[i] != assignments[j]) + +# Min-max objective +max_used = model.NewIntVar(0, max_value, "max_used") +for var in assignments.values(): + model.Add(max_used >= var) +model.Minimize(max_used) + +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 +status = solver.Solve(model) + +if status == cp_model.OPTIMAL or status == cp_model.FEASIBLE: + result = { + "status": "optimal" if status == cp_model.OPTIMAL else "feasible", + "objective": solver.ObjectiveValue(), + "assignments": {e: solver.Value(v) for e, v in assignments.items()} + } +else: + result = {"status": "failed", "solver_status": status} +print(json.dumps(result)) +``` + +### Common Pitfalls +- Not checking for `FEASIBLE` status when optimality is not required. +- Forgetting to set a time limit, causing indefinite runtime on large instances. +- Using default solver parameters without tuning for problem size. + +# Workflow 2 (MIP Solver) + +## Modeling stage + +### Strategy Overview +Use binary assignment variables for each entity-value pair. Enforce exactly-one assignment per entity, conflict avoidance with pairwise constraints, and model the min-max objective with a continuous auxiliary variable. + +### Step 1 - Define Binary Assignment Variables +- Create binary variable `x[i, f]` for each entity `i` and each value `f` in its allowed set. +- Use `pyo.Var(entities, allowed_values, domain=pyo.Binary)`. + +### Step 2 - Enforce Exactly One Assignment +- For each entity `i`, add constraint: `sum(x[i, f] for f in allowed_values[i]) == 1`. + +### Step 3 - Restrict to Allowed Values +- For disallowed entity-value pairs, fix variable to 0: either skip creating the variable or add constraint `x[i, f] == 0`. + +### Step 4 - Model Conflict Avoidance +- For each conflicting pair `(i, j)` and each value `f`, add constraint: `x[i, f] + x[j, f] <= 1`. + +### Step 5 - Model Min-Max Objective +- Create continuous auxiliary variable `z` with domain `pyo.NonNegativeReals`. +- Add constraints: `z >= f * x[i, f]` for all entity-value pairs. +- Set objective: `pyo.Objective(expr=z, sense=pyo.minimize)`. + +### Formulation Template +```json +{ + "sets": ["entities", "values"], + "parameters": ["allowed_values[entity]", "conflict_pairs"], + "decision_variables": [ + "x[entity, value] (binary)", + "z (continuous, non-negative)" + ], + "objective": { + "sense": "min", + "expression": "z" + }, + "constraints": [ + "sum(x[i, f] for f in allowed_values[i]) == 1 for each entity i", + "x[i, f] + x[j, f] <= 1 for each conflict pair (i,j) and each value f", + "z >= f * x[i, f] for all entity-value pairs" + ] +} +``` + +### Common Pitfalls +- Creating variables for all entity-value pairs instead of only allowed ones, increasing model size unnecessarily. +- Using integer variables instead of binary, which may slow down the MIP solver. +- Forgetting to fix disallowed assignments to 0, leading to invalid solutions. + +## Solving stage + +### Strategy Overview +Use a MIP solver (CBC, Gurobi, CPLEX) with appropriate options. Check solver status and termination condition before extracting results. + +### Step 1 - Configure and Solve +- Instantiate solver: `solver = pyo.SolverFactory("cbc")` +- Set options: `solver.options["seconds"] = [TIME_LIMIT]`, `solver.options["ratio"] = 0.0` +- Solve: `results = solver.solve(model, tee=False)` + +### Step 2 - Check Solver Status +- Verify `results.solver.status == pyo.SolverStatus.ok` +- Check termination condition: `results.solver.termination_condition` is `optimal` or `feasible` + +### Step 3 - Extract and Validate Results +- Retrieve objective: `pyo.value(model.z)` +- Decode assignments: for each entity, find value where `pyo.value(model.x[i, f]) > 0.5` +- Validate that each assignment belongs to the entity's allowed set and that all conflict constraints are satisfied. + +### Step 4 - Output Structured Result +- Output structured JSON with status, objective, and assignments + +### Code Usage +```python +import pyomo.environ as pyo +import json + +model = pyo.ConcreteModel() +# Define sets +model.entities = pyo.Set(initialize=entity_list) +model.values = pyo.Set(initialize=value_list) + +# Binary assignment variables +model.x = pyo.Var(model.entities, model.values, domain=pyo.Binary) + +# Exactly one assignment per entity +def exactly_one_rule(m, i): + return sum(m.x[i, f] for f in m.values if (i, f) in allowed_pairs) == 1 +model.exactly_one = pyo.Constraint(model.entities, rule=exactly_one_rule) + +# Conflict avoidance +def conflict_rule(m, i, j, f): + if (i, j) in conflict_pairs: + return m.x[i, f] + m.x[j, f] <= 1 + return pyo.Constraint.Skip +model.conflict = pyo.Constraint(model.entities, model.entities, model.values, rule=conflict_rule) + +# Min-max objective +model.z = pyo.Var(domain=pyo.NonNegativeReals) +def max_rule(m, i, f): + return m.z >= f * m.x[i, f] +model.max_constraint = pyo.Constraint(model.entities, model.values, rule=max_rule) +model.objective = pyo.Objective(expr=model.z, sense=pyo.minimize) + +solver = pyo.SolverFactory("cbc") +solver.options["seconds"] = [TIME_LIMIT] +solver.options["ratio"] = 0.0 +results = solver.solve(model, tee=False) + +if results.solver.status == pyo.SolverStatus.ok: + if results.solver.termination_condition in [pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible]: + assignments = {} + for i in model.entities: + for f in model.values: + if pyo.value(model.x[i, f]) > 0.5: + assignments[i] = f + break + result = { + "status": str(results.solver.termination_condition), + "objective": float(pyo.value(model.z)), + "assignments": assignments + } + else: + result = {"status": "failed", "termination": str(results.solver.termination_condition)} +else: + result = {"status": "failed", "solver_status": str(results.solver.status)} +print(json.dumps(result)) +``` + +### Common Pitfalls +- Not checking both `status` and `termination_condition`, leading to reading invalid results. +- Using `tee=True` in production, which can produce excessive output. +- Forgetting to convert Pyomo values to native Python types before JSON serialization. diff --git a/skills/optskills/skill_library/edge_traversal_minimization_for_required_route_coverage.md b/skills/optskills/skill_library/edge_traversal_minimization_for_required_route_coverage.md new file mode 100644 index 0000000..414fc3e --- /dev/null +++ b/skills/optskills/skill_library/edge_traversal_minimization_for_required_route_coverage.md @@ -0,0 +1,269 @@ +--- +name: Edge Traversal Minimization for Required Route Coverage +description: | + Models and solves routing problems requiring traversal of specified edges with minimum total distance, using integer flow formulations and either constraint programming or mixed-integer programming solvers. +--- + +# Workflow 1 (CP-SAT Integer Flow) + +## Modeling stage + +### Strategy Overview +Model the problem as an integer flow on a directed, fully connected network of nodes. Use CP-SAT to enforce flow conservation and required edge coverage, ensuring a closed Eulerian tour that services all required edges. The vehicle may traverse any arc between nodes; the distance matrix defines the cost for each possible traversal. + +### Step 1 - Define Directed Flow Variables +- For each ordered pair `(i,j)` with `i != j`, create an integer variable `y[(i,j)]` representing the number of traversals from `i` to `j`. +- Set domain to `[0, UB]` where `UB` is a small integer (e.g., 10) to bound the search space. + +### Step 2 - Enforce Flow Conservation +- For each node `k`, add constraint: `sum(y[(k,j)] for j != k) == sum(y[(i,k)] for i != k)`. +- This ensures the tour is a closed walk (Eulerian circuit). + +### Step 3 - Cover Required Edges +- For each required undirected edge `(u,v)`, add constraint: `y[(u,v)] + y[(v,u)] >= 1`. +- This guarantees at least one traversal in either direction. + +### Step 4 - Set Objective +- Minimize total distance: `sum(dist[i][j] * y[(i,j)] for all ordered pairs (i,j) with i != j)`. +- Use a precomputed all-pairs shortest path distance matrix. + +### Formulation Template +```json +{ + "sets": ["NODES", "REQUIRED_EDGES"], + "parameters": ["dist[i][j] for all i,j in NODES, i != j"], + "decision_variables": [ + "y[(i,j)] integer >= 0 for each ordered pair (i,j) with i != j" + ], + "objective": { + "sense": "min", + "expression": "sum(dist[i][j] * y[(i,j)] for all i != j)" + }, + "constraints": [ + "sum(y[(k,j)] for j != k) == sum(y[(i,k)] for i != k) for each k in NODES", + "y[(u,v)] + y[(v,u)] >= 1 for each undirected required edge (u,v) in REQUIRED_EDGES" + ] +} +``` + +### Common Pitfalls +- **Incorrectly restricting traversals to physical edges.** The model must allow flow on all arcs; required edges define service obligations, not traversal restrictions. +- Using binary variables instead of integer variables, which prevents modeling multiple traversals of the same arc. +- Omitting the required edge coverage constraint, leaving edges unserviced. +- **Mismatching cost calculation with allowed arcs.** The objective must sum over all ordered pairs `(i,j)` where `i != j`, using the distance matrix for cost. Do not limit the sum to a subset of edges. + +## Solving stage + +### Strategy Overview +Use OR-Tools CP-SAT solver to handle the integer programming model. Configure for parallel search and time-limited execution, then extract and verify the solution. + +### Step 1 - Initialize Solver and Model +- Import `cp_model` from `ortools.sat.python`. +- Create `CpModel()` instance and add all variables with appropriate domains. + +### Step 2 - Add Constraints and Objective +- Add flow conservation constraints using `model.Add()`. +- Add required edge coverage constraints. +- Set objective using `model.Minimize()` with the full distance matrix over all ordered pairs. + +### Step 3 - Configure and Solve +- Create `CpSolver()` instance. +- Set parameters: `max_time_in_seconds = [TIME_LIMIT]`, `num_search_workers = 8`, `random_seed = 42`. +- Call `solver.Solve(model)` and capture status. + +### Step 4 - Extract and Verify Results +- Check status: `OPTIMAL` or `FEASIBLE` indicate valid solutions. +- Print non-zero directed flows for verification. +- **Verify that all required edges are covered:** For each `(u,v)` in required edges, ensure `solver.Value(y[(u,v)]) + solver.Value(y[(v,u)]) >= 1`. +- **Verify flow conservation per node** by recomputing inflow and outflow. +- **Verify objective consistency:** Recompute total cost from the solution flows and the full distance matrix to ensure it matches the solver's reported objective. + +### Code Usage +```python +from ortools.sat.python import cp_model + +def solve_route_coverage_cpsat(nodes, required_edges, dist, time_limit=60, ub=10): + model = cp_model.CpModel() + y = {} + # Create directed flow variables for all ordered pairs + for i in nodes: + for j in nodes: + if i != j: + y[(i, j)] = model.NewIntVar(0, ub, f'y_{i}_{j}') + + # Flow conservation + for k in nodes: + outflow = sum(y[(k, j)] for j in nodes if j != k) + inflow = sum(y[(i, k)] for i in nodes if i != k) + model.Add(outflow == inflow) + + # Required edge coverage + for (u, v) in required_edges: + model.Add(y[(u, v)] + y[(v, u)] >= 1) + + # Objective: sum over all ordered pairs + model.Minimize(sum(dist[i][j] * y[(i, j)] for i in nodes for j in nodes if i != j)) + + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = time_limit + solver.parameters.num_search_workers = 8 + solver.parameters.random_seed = 42 + status = solver.Solve(model) + + if status == cp_model.OPTIMAL or status == cp_model.FEASIBLE: + result = {"status": "OPTIMAL" if status == cp_model.OPTIMAL else "FEASIBLE", + "objective": solver.ObjectiveValue()} + # Verification checks + for (u, v) in required_edges: + if not (solver.Value(y[(u, v)]) + solver.Value(y[(v, u)]) >= 1): + raise ValueError(f"Required edge ({u},{v}) not covered.") + # Print non-zero flows + for (i, j) in y: + if solver.Value(y[(i, j)]) > 0: + print(f"Arc ({i}->{j}): {solver.Value(y[(i, j)])} traversals") + return result + else: + return {"status": "INFEASIBLE", "objective": None} +``` + +### Common Pitfalls +- Not checking for `FEASIBLE` status in addition to `OPTIMAL`, missing valid suboptimal solutions. +- Forgetting to set `random_seed` for reproducibility across runs. +- **Failing to verify required edge coverage and objective consistency**, potentially accepting solutions that violate problem constraints or have mismatched costs. + +# Workflow 2 (HiGHS MIP Flow) + +## Modeling stage + +### Strategy Overview +Model the problem directly with directed integer flow variables on all arcs in a fully connected network. Use a MIP solver to minimize total traversal distance while enforcing flow conservation and required edge coverage. The distance matrix provides the cost for each possible arc traversal. + +### Step 1 - Define Directed Flow Variables +- For each ordered pair `(i,j)` with `i != j`, create an integer variable `x[i,j]` representing the number of traversals from `i` to `j`. +- Set domain to `NonNegativeIntegers` to allow multiple passes. + +### Step 2 - Enforce Flow Conservation +- For each node `k`, add constraint: `sum(x[k,j] for j != k) == sum(x[i,k] for i != k)`. +- This ensures a closed tour starting and ending at the same node. + +### Step 3 - Cover Required Edges +- For each required undirected edge `(u,v)`, add constraint: `x[u,v] + x[v,u] >= 1`. +- This guarantees at least one traversal in either direction. + +### Step 4 - Set Objective +- Minimize total distance: `sum(dist[i,j] * x[i,j] for all ordered pairs (i,j) with i != j)`. +- Use a precomputed all-pairs shortest path distance matrix. + +### Formulation Template +```json +{ + "sets": ["NODES", "ARCS = {(i,j) for i,j in NODES, i != j}", "REQUIRED_EDGES"], + "parameters": ["dist[i][j] for all (i,j) in ARCS"], + "decision_variables": [ + "x[i,j] integer >= 0 for each (i,j) in ARCS" + ], + "objective": { + "sense": "min", + "expression": "sum(dist[i][j] * x[i,j] for all (i,j) in ARCS)" + }, + "constraints": [ + "sum(x[k,j] for j != k) == sum(x[i,k] for i != k) for each k in NODES", + "x[u,v] + x[v,u] >= 1 for each undirected required edge (u,v) in REQUIRED_EDGES" + ] +} +``` + +### Common Pitfalls +- Using binary variables instead of integer variables, preventing multiple traversals of the same arc. +- Forgetting to add both directions for required edge coverage, leaving the edge potentially uncovered. +- **Restricting the arc set to physical edges.** The model must use the fully connected arc set `ARCS`; required edges only impose coverage constraints. +- **Mismatching the objective sum with the arc set.** Ensure the objective sums over all arcs in `ARCS`, not a subset. + +## Solving stage + +### Strategy Overview +Use HiGHS solver via Pyomo to solve the MIP model. Configure for exact optimality with time limit and parallel threads, then extract and verify the solution. + +### Step 1 - Build Pyomo Model +- Import `pyomo.environ` as `pyo`. +- Create `ConcreteModel()` with sets for nodes and arcs (all ordered pairs `i != j`). +- Add `Var` for each arc with domain `NonNegativeIntegers`. + +### Step 2 - Add Constraints and Objective +- Add flow conservation constraints using `Constraint` for each node. +- Add required edge coverage constraints. +- Set objective using `Objective` with `sense=minimize`, summing over all arcs. + +### Step 3 - Configure and Solve +- Create solver instance: `SolverFactory("highs")`. +- Set options: `time_limit=[TIME_LIMIT]`, `mip_rel_gap=0.0`, `threads=4`. +- Call `solver.solve(model)` and capture status. + +### Step 4 - Extract and Verify Results +- Check `SolverStatus.ok` and `TerminationCondition.optimal` or `feasible`. +- **Verify required edge coverage:** For each `(u,v)` in required edges, ensure `model.x[u,v].value + model.x[v,u].value >= 1`. +- **Verify flow conservation per node** by recomputing inflow and outflow. +- **Verify objective consistency:** Recompute total cost from solution flows and the distance matrix. +- Iterate over all arcs to print non-zero flows for verification. + +### Code Usage +```python +import pyomo.environ as pyo + +def solve_route_coverage_highs(nodes, required_edges, dist, time_limit=60): + model = pyo.ConcreteModel() + model.NODES = pyo.Set(initialize=nodes) + # Arcs: all ordered pairs i != j + model.ARCS = pyo.Set(initialize=[(i, j) for i in nodes for j in nodes if i != j], dimen=2) + + # Decision variables + model.x = pyo.Var(model.ARCS, domain=pyo.NonNegativeIntegers) + + # Flow conservation + def flow_conservation_rule(model, k): + outflow = sum(model.x[k, j] for j in model.NODES if (k, j) in model.ARCS) + inflow = sum(model.x[i, k] for i in model.NODES if (i, k) in model.ARCS) + return outflow == inflow + model.flow_cons = pyo.Constraint(model.NODES, rule=flow_conservation_rule) + + # Required edge coverage + def required_edge_rule(model, u, v): + return model.x[u, v] + model.x[v, u] >= 1 + model.req_edges = pyo.Constraint(required_edges, rule=required_edge_rule) + + # Objective: sum over all arcs + def obj_rule(model): + return sum(dist[i][j] * model.x[i, j] for (i, j) in model.ARCS) + model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + + # Solve + solver = pyo.SolverFactory("highs") + solver.options["time_limit"] = time_limit + solver.options["mip_rel_gap"] = 0.0 + solver.options["threads"] = 4 + result = solver.solve(model, tee=False) + + # Check status + if (result.solver.status == pyo.SolverStatus.ok and + result.solver.termination_condition in + [pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible]): + obj_val = pyo.value(model.obj) + # Verification checks + for (u, v) in required_edges: + if not (pyo.value(model.x[u, v]) + pyo.value(model.x[v, u]) >= 1): + raise ValueError(f"Required edge ({u},{v}) not covered.") + # Print non-zero flows + for (i, j) in model.ARCS: + val = pyo.value(model.x[i, j]) + if val > 0: + print(f"Arc ({i}->{j}): {val} traversals") + return {"status": "OPTIMAL" if result.solver.termination_condition == pyo.TerminationCondition.optimal else "FEASIBLE", + "objective": obj_val} + else: + return {"status": "INFEASIBLE", "objective": None} +``` + +### Common Pitfalls +- Not checking both `SolverStatus.ok` and termination condition, potentially accepting failed solves. +- Setting `mip_rel_gap` too high, accepting suboptimal solutions when exact optimality is required. +- **Failing to verify required edge coverage and objective consistency**, which can hide model-solution mismatches. diff --git a/skills/optskills/skill_library/employee_shift_assignment_with_unfulfilled_demand.md b/skills/optskills/skill_library/employee_shift_assignment_with_unfulfilled_demand.md new file mode 100644 index 0000000..4bc47f1 --- /dev/null +++ b/skills/optskills/skill_library/employee_shift_assignment_with_unfulfilled_demand.md @@ -0,0 +1,229 @@ +--- +name: Employee Shift Assignment with Unfulfilled Demand +description: | + Models and solves a resource-to-demand assignment problem where unfulfilled demand is penalized, using either CP-SAT or MIP solvers with binary assignment variables and integer slack variables. +--- + +# Workflow 1 (CP-SAT with OR-Tools) + +## Modeling stage + +### Strategy Overview +Model the assignment problem using boolean variables for each employee-location-shift combination and integer slack variables for unfulfilled demand. Enforce demand coverage with equality constraints, employee availability by fixing disallowed assignments to zero, and at-most-one-assignment per employee. Minimize a weighted sum of assignment costs and penalty costs for unmet demand. + +### Step 1 - Define Assignment Variables +- Create a binary variable for each employee, location, and shift combination using `model.NewBoolVar()`. +- Name each variable with a descriptive string (e.g., `f"x_{e}_{r}_{s}"`) for debugging. + +### Step 2 - Define Unfulfilled Demand Variables +- Create an integer variable for each location-shift pair using `model.NewIntVar(0, demand[r][s], name)`. +- Set the upper bound to the demand value for that pair to prevent unbounded solutions. + +### Step 3 - Enforce Demand Coverage +- For each location-shift pair, add an equality constraint: sum of assignment variables for that pair plus the unfulfilled variable equals the required demand. +- Use `model.Add(sum(x[e, r, s] for e in employees) + u[r, s] == demand[r, s])`. + +### Step 4 - Model Employee Availability +- For each unavailable employee, add a constraint fixing the sum of all their assignment variables to zero: `model.Add(sum(x[employee, ...]) == 0)`. +- For available employees, no constraint is needed. + +### Step 5 - Limit Each Employee to One Assignment +- For each employee, add a constraint that the sum of all their assignment variables is less than or equal to 1: `model.Add(sum(x[e, ...]) <= 1)`. + +### Step 6 - Construct Objective +- Minimize the sum of assignment costs (cost[e] * x[e, r, s]) plus penalty costs (penalty * u[r, s]). +- Use `model.Minimize()` with the weighted sum expression. + +### Formulation Template +```json +{ + "sets": ["Employees", "Locations", "Shifts"], + "parameters": ["demand[location, shift]", "cost[employee]", "penalty"], + "decision_variables": [ + "x[employee, location, shift] ∈ {0, 1}", + "u[location, shift] ∈ [0, demand[location, shift]]" + ], + "objective": { + "sense": "min", + "expression": "sum(cost[e] * x[e, r, s]) + sum(penalty * u[r, s])" + }, + "constraints": [ + "sum(x[e, r, s] for e) + u[r, s] == demand[r, s]", + "sum(x[e, r, s] for r, s) <= 1", + "sum(x[unavailable_e, r, s] for r, s) == 0" + ] +} +``` + +### Common Pitfalls +- Setting the penalty too low, causing the solver to leave demand unfulfilled even when employees are available. +- Forgetting to set an upper bound on unfulfilled demand variables, which can lead to unbounded solutions. +- Using `model.AddBoolOr()` or other logical constraints when simple linear constraints suffice. + +## Solving stage + +### Strategy Overview +Use OR-Tools CP-SAT solver with parallel search and a time limit. Check solver status for both optimal and feasible solutions before extracting results. Output objective value and variable assignments in a parseable format. + +### Step 1 - Configure Solver +- Create a `CpSolver()` instance. +- Set `solver.parameters.max_time_in_seconds = [TIME_LIMIT]` for predictable runtime. +- Enable parallel search with `solver.parameters.num_search_workers = [NUM_WORKERS]`. +- Fix a random seed with `solver.parameters.random_seed = [SEED]` for reproducibility. + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve(model)`. +- Check if status is `cp_model.OPTIMAL` or `cp_model.FEASIBLE` before reading results. +- If neither, output a failure message with the status code. + +### Step 3 - Extract Results +- Read the objective value using `float(solver.ObjectiveValue())`. +- Iterate over all assignment variables and collect those with `solver.Value(var) == 1`. +- Iterate over unfulfilled demand variables and collect those with `solver.Value(var) > 0`. +- Print results in a parseable format (e.g., `RESULT:{obj}` or JSON). + +### Code Usage +```python +from ortools.sat.python import cp_model + +model = cp_model.CpModel() +# ... build model ... + +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30 +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 + +status = solver.Solve(model) +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + obj = float(solver.ObjectiveValue()) + assignments = [(e, r, s) for e, r, s in x if solver.Value(x[e, r, s]) == 1] + unfulfilled = [(r, s) for r, s in u if solver.Value(u[r, s]) > 0] + print(f"RESULT:{obj}") +else: + print("FAILED") +``` + +### Common Pitfalls +- Not checking for `FEASIBLE` status, missing near-optimal solutions found within the time limit. +- Using `solver.Value()` on variables that were not solved (e.g., after infeasible status), causing runtime errors. +- Setting the time limit too short for large instances, resulting in no feasible solution found. + +# Workflow 2 (MIP with Pyomo) + +## Modeling stage + +### Strategy Overview +Model the assignment problem using Pyomo with binary assignment variables and non-negative integer slack variables. Enforce demand coverage with equality constraints, employee availability by fixing disallowed assignments to zero, and at-most-one-assignment per employee. Minimize a weighted sum of assignment costs and penalty costs for unmet demand. + +### Step 1 - Define Sets and Parameters +- Create Pyomo sets for employees, locations, and shifts using `pyo.Set(initialize=...)`. +- Define parameters for demand, cost, and penalty using `pyo.Param()`. + +### Step 2 - Define Assignment Variables +- Create binary variables for each employee-location-shift combination using `pyo.Var(domain=pyo.Binary)`. +- Use a multi-index set (e.g., `model.employee_set * model.location_set * model.shift_set`) for the indexing domain. + +### Step 3 - Define Unfulfilled Demand Variables +- Create non-negative integer variables for each location-shift pair using `pyo.Var(domain=pyo.NonNegativeIntegers)`. +- Optionally set an upper bound equal to the demand value. + +### Step 4 - Enforce Demand Coverage +- For each location-shift pair, add a constraint: sum of assignment variables plus unfulfilled variable equals demand. +- Use `model.Add(expr=sum(x[e, r, s] for e in employees) + u[r, s] == demand[r, s])`. + +### Step 5 - Model Employee Availability +- For each unavailable employee, add a constraint fixing the sum of all their assignment variables to zero. +- Use `pyo.Constraint.Skip` for available employees to avoid unnecessary constraints. + +### Step 6 - Limit Each Employee to One Assignment +- For each employee, add a constraint that the sum of all their assignment variables is less than or equal to 1. + +### Step 7 - Construct Objective +- Minimize the sum of assignment costs plus penalty costs for unfulfilled demand. +- Use `pyo.Objective(expr=..., sense=pyo.minimize)`. + +### Formulation Template +```json +{ + "sets": ["Employees", "Locations", "Shifts"], + "parameters": ["demand[location, shift]", "cost[employee]", "penalty"], + "decision_variables": [ + "x[employee, location, shift] ∈ {0, 1}", + "u[location, shift] ∈ ℤ⁺" + ], + "objective": { + "sense": "min", + "expression": "sum(cost[e] * x[e, r, s]) + sum(penalty * u[r, s])" + }, + "constraints": [ + "sum(x[e, r, s] for e) + u[r, s] == demand[r, s]", + "sum(x[e, r, s] for r, s) <= 1", + "sum(x[unavailable_e, r, s] for r, s) == 0" + ] +} +``` + +### Common Pitfalls +- Using `pyo.NonNegativeReals` instead of `pyo.NonNegativeIntegers` for unfulfilled demand, which can lead to fractional shortages. +- Forgetting to use `pyo.Constraint.Skip` for available employees, creating unnecessary constraints that slow down the solver. +- Setting the penalty too low, causing the solver to leave demand unfulfilled even when employees are available. + +## Solving stage + +### Strategy Overview +Use a standard MIP solver (CBC or GLPK) with a time limit and optimality gap. Check solver status and termination condition before extracting results. Handle floating-point rounding for binary variables with a threshold check. + +### Step 1 - Configure Solver +- Create a solver instance with `pyo.SolverFactory("cbc")` or `pyo.SolverFactory("glpk")`. +- Set time limit: `solver.options["seconds"] = [TIME_LIMIT]` (CBC) or `solver.options["tmlim"] = [TIME_LIMIT]` (GLPK). +- Set optimality gap: `solver.options["ratio"] = [GAP]` (CBC) or `solver.options["mipgap"] = [GAP]` (GLPK). +- Enable parallelism: `solver.options["threads"] = [NUM_WORKERS]` (CBC). + +### Step 2 - Solve and Check Status +- Call `results = solver.solve(model, tee=False)`. +- Check `results.solver.status == SolverStatus.ok` and `results.solver.termination_condition` is either `TerminationCondition.optimal` or `TerminationCondition.feasible`. +- If not, output a failure JSON payload with status and termination condition strings. + +### Step 3 - Extract Results +- Read the objective value using `pyo.value(model.objective)`. +- Iterate over assignment variables and collect those with `pyo.value(var) > 0.5` (threshold for binary variables). +- Iterate over unfulfilled demand variables and collect those with `pyo.value(var) > 0`. +- Package results in a JSON payload with keys: `status`, `objective`, `assignments`, and `unfulfilled`. +- Print the JSON with a `RESULT_JSON:` prefix for easy parsing. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition +import json + +model = pyo.ConcreteModel() +# ... build model ... + +solver = pyo.SolverFactory("cbc") +solver.options["seconds"] = 30 +solver.options["ratio"] = 0.0 +solver.options["threads"] = 4 + +results = solver.solve(model, tee=False) +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in (TerminationCondition.optimal, + TerminationCondition.feasible)): + obj = pyo.value(model.objective) + assignments = [(e, r, s) for e, r, s in model.x if pyo.value(model.x[e, r, s]) > 0.5] + unfulfilled = [(r, s) for r, s in model.u if pyo.value(model.u[r, s]) > 0] + payload = {"status": "success", "objective": obj, + "assignments": assignments, "unfulfilled": unfulfilled} + print(f"RESULT_JSON:{json.dumps(payload)}") +else: + payload = {"status": "failed", + "solver_status": str(results.solver.status), + "termination": str(results.solver.termination_condition)} + print(f"RESULT_JSON:{json.dumps(payload)}") +``` + +### Common Pitfalls +- Using `pyo.value()` on variables before checking solver status, causing runtime errors on failed solves. +- Using a threshold of `0.0` for binary variables, which can pick up floating-point noise near zero. +- Not setting a time limit, causing the solver to run indefinitely on large instances. diff --git a/skills/optskills/skill_library/facility_location_coverage_optimizer.md b/skills/optskills/skill_library/facility_location_coverage_optimizer.md new file mode 100644 index 0000000..a204b40 --- /dev/null +++ b/skills/optskills/skill_library/facility_location_coverage_optimizer.md @@ -0,0 +1,289 @@ +--- +name: Facility Location Coverage Optimizer +description: | + Models and solves a facility location problem with mandatory coverage and premium service tiers, minimizing net cost using binary activation and coverage variables. +--- + +# Workflow 1 (OR-Tools MIP Solver) + +## Modeling stage + +### Strategy Overview +Use Google OR-Tools' linear solver to formulate a mixed-integer program with binary activation variables for facilities and binary premium coverage variables for demand points. Precompute coverage sets to keep constraints sparse and linear. + +### Step 1 - Precompute Coverage Sets +- For each demand point `i`, compute `mandatory_cover[i]`: list of candidate facilities within mandatory distance `R1`. +- For each demand point `i`, compute `premium_cover[i]`: list of candidate facilities within premium distance `R2`. +- **Feasibility Check**: Verify `mandatory_cover[i]` is non-empty for all `i`. If empty, the problem is infeasible with the given candidate sites and distance threshold. + +### Step 2 - Define Binary Activation Variables +- Create `IntVar(0, 1)` for each candidate facility site `j` to represent activation `x[j]`. +- Use a dictionary indexed by facility IDs for easy access. + +### Step 3 - Define Premium Coverage Variables +- Create `IntVar(0, 1)` for each demand point `i` to represent premium coverage `y[i]`. + +### Step 4 - Enforce Mandatory Coverage Constraints +- For each demand point `i`, add constraint: `sum(x[j] for j in mandatory_cover[i]) >= 1`. + +### Step 5 - Link Premium Coverage to Activation +- For each demand point `i`: + - If `premium_cover[i]` is non-empty: add constraint `y[i] <= sum(x[j] for j in premium_cover[i])`. + - If `premium_cover[i]` is empty: fix `y[i] == 0` to prevent solver errors and model infeasibility. + +### Step 6 - Define Net Cost Objective +- Set objective to minimize: `sum(construction_cost[j] * x[j]) - sum(benefit[i] * y[i])`. +- Use `objective.SetCoefficient()` for each term and `objective.SetMinimization()`. + +### Formulation Template +```json +{ + "sets": ["I: demand points", "J: candidate facility sites"], + "parameters": ["construction_cost[j]", "benefit[i]", "mandatory_cover[i]", "premium_cover[i]"], + "decision_variables": [ + "x[j] ∈ {0,1}: facility activation", + "y[i] ∈ {0,1}: premium coverage indicator" + ], + "objective": { + "sense": "min", + "expression": "sum_{j∈J} construction_cost[j] * x[j] - sum_{i∈I} benefit[i] * y[i]" + }, + "constraints": [ + "sum_{j∈mandatory_cover[i]} x[j] >= 1, ∀i∈I", + "y[i] <= sum_{j∈premium_cover[i]} x[j], ∀i∈I where premium_cover[i] ≠ ∅", + "y[i] == 0, ∀i∈I where premium_cover[i] = ∅" + ] +} +``` + +### Common Pitfalls +- Forgetting to fix `y[i] = 0` for demand points with empty premium coverage sets, which can cause solver errors or unintended behavior. +- Using distance calculations inside constraint loops instead of precomputing coverage sets, leading to performance degradation. +- Not verifying mandatory coverage feasibility before solving. + +## Solving stage + +### Strategy Overview +Use OR-Tools' SCIP solver for mixed-integer problems. Set time limits and thread counts for practical solve times. Parse results with status checks and output structured JSON. + +### Step 1 - Configure Solver +- Create solver with `pywraplp.Solver.CreateSolver("SCIP")`. +- Set time limit: `solver.SetTimeLimit([TIME_LIMIT_MS])` (e.g., 60000 for 60 seconds). +- Enable multi-threading: `solver.SetNumThreads([THREAD_COUNT])`. + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve()`. +- Accept both `pywraplp.Solver.OPTIMAL` and `pywraplp.Solver.FEASIBLE` as valid outcomes. + +### Step 3 - Extract Solution +- For binary variables, check `variable.solution_value() > 0.5` to determine activation. +- Collect activated facility IDs and premium-covered demand points. + +### Step 4 - Verify and Output +- **Verification Loop**: Independently verify mandatory coverage constraints by checking distances against chosen sites. Verify each premium-covered demand point has at least one activated facility within `R2`. +- Print results as JSON prefixed with `RESULT_JSON:` including status, objective value, and decision summaries. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# Precompute coverage sets (Step 1) +mandatory_cover = {i: [j for j in facilities if dist[i][j] <= R1] for i in demands} +premium_cover = {i: [j for j in facilities if dist[i][j] <= R2] for i in demands} + +# Feasibility check +for i in demands: + if not mandatory_cover[i]: + raise ValueError(f"Demand point {i} has no mandatory coverage candidate.") + +solver = pywraplp.Solver.CreateSolver("SCIP") +solver.SetTimeLimit([TIME_LIMIT_MS]) +solver.SetNumThreads([THREAD_COUNT]) + +# Decision variables +x = {j: solver.IntVar(0, 1, f"x_{j}") for j in facilities} +y = {i: solver.IntVar(0, 1, f"y_{i}") for i in demands} + +# Mandatory coverage constraints +for i in demands: + solver.Add(sum(x[j] for j in mandatory_cover[i]) >= 1) + +# Premium coverage linkage +for i in demands: + if premium_cover[i]: + solver.Add(y[i] <= sum(x[j] for j in premium_cover[i])) + else: + solver.Add(y[i] == 0) + +# Objective +objective = solver.Objective() +for j in facilities: + objective.SetCoefficient(x[j], construction_cost[j]) +for i in demands: + objective.SetCoefficient(y[i], -benefit[i]) # Negative for benefit +objective.SetMinimization() + +status = solver.Solve() +if status in (pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE): + activated = [j for j in facilities if x[j].solution_value() > 0.5] + premium_covered = [i for i in demands if y[i].solution_value() > 0.5] + # Verification + for i in demands: + if not any(dist[i][j] <= R1 for j in activated): + raise RuntimeError(f"Mandatory coverage violated for demand point {i}") + for i in premium_covered: + if not any(dist[i][j] <= R2 for j in activated): + raise RuntimeError(f"Premium coverage violated for demand point {i}") + result = { + "status": "OPTIMAL" if status == pywraplp.Solver.OPTIMAL else "FEASIBLE", + "objective": objective.Value(), + "activated_facilities": activated, + "premium_covered": premium_covered + } + print(f"RESULT_JSON:{result}") +else: + print(f'RESULT_JSON:{{"status": "INFEASIBLE", "solver_status": {status}}}') +``` + +### Common Pitfalls +- Not checking solver status before accessing solution values, causing runtime errors on infeasible models. +- Using `solution_value()` directly for integer comparison without tolerance (e.g., `> 0.5` instead of `== 1.0`). + +# Workflow 2 (Pyomo with CBC/HiGHS) + +## Modeling stage + +### Strategy Overview +Use Pyomo's algebraic modeling language to formulate the facility location problem with binary variables. Leverage constraint rules and precomputed distance sets for clean, maintainable code. + +### Step 1 - Precompute Coverage Sets +- Compute `mandatory_cover[i]` and `premium_cover[i]` as immutable data structures (e.g., tuples or frozensets) before model construction. +- Verify mandatory coverage feasibility. + +### Step 2 - Define Sets and Parameters +- Create Pyomo `Set` objects for demand points and candidate facilities. +- Define parameters for construction costs, premium benefits, and the precomputed coverage sets. + +### Step 3 - Declare Binary Decision Variables +- Use `pyo.Var(domain=pyo.Binary)` for facility activation (`x[j]`). +- Use `pyo.Var(domain=pyo.Binary)` for premium coverage indicators (`y[i]`). + +### Step 4 - Implement Mandatory Coverage Rule +- Write a constraint rule that sums activation variables over facilities in the mandatory coverage set for each demand point. +- Enforce the sum to be at least 1. + +### Step 5 - Implement Premium Coverage Linkage +- For each demand point, add constraint: `y[i] <= sum(x[j] for j in premium_cover_set[i])` if the set is non-empty. +- For demand points with empty premium coverage sets, fix `y[i] = 0` explicitly. + +### Step 6 - Define Net Cost Objective +- Express objective as: `sum(construction_cost[j] * x[j] for j in facilities) - sum(benefit[i] * y[i] for i in demands)`. +- Use `pyo.Objective(sense=pyo.minimize, rule=objective_rule)`. + +### Formulation Template +```json +{ + "sets": ["I: demand points", "J: candidate facility sites"], + "parameters": ["construction_cost[J]", "benefit[I]", "mandatory_cover[I]", "premium_cover[I]"], + "decision_variables": [ + "x[J] ∈ {0,1}: facility activation", + "y[I] ∈ {0,1}: premium coverage indicator" + ], + "objective": { + "sense": "min", + "expression": "sum_{j∈J} construction_cost[j] * x[j] - sum_{i∈I} benefit[i] * y[i]" + }, + "constraints": [ + "sum_{j∈mandatory_cover[i]} x[j] >= 1, ∀i∈I", + "y[i] <= sum_{j∈premium_cover[i]} x[j], ∀i∈I where premium_cover[i] ≠ ∅", + "y[i] == 0, ∀i∈I where premium_cover[i] = ∅" + ] +} +``` + +### Common Pitfalls +- Using mutable lists inside constraint rules that change during construction; always use immutable data or deep copies. +- Forgetting to handle empty premium coverage sets, which can cause Pyomo to generate trivial constraints that break the model. + +## Solving stage + +### Strategy Overview +Use Pyomo's solver interface with CBC or HiGHS for MILP solving. Configure solver options for time limits and MIP gaps. Parse results with proper status checks and structured output. + +### Step 1 - Configure Solver +- Create solver with `pyo.SolverFactory("cbc")` or `pyo.SolverFactory("highs")`. +- Set options: `solver.options["seconds"] = [TIME_LIMIT_S]` (e.g., 60), `solver.options["ratio"] = 0.0` for optimality. + +### Step 2 - Solve and Check Status +- Call `results = solver.solve(model, tee=False)`. +- Check `results.solver.status` and `results.solver.termination_condition`. +- Accept both `TerminationCondition.optimal` and `TerminationCondition.feasible`. + +### Step 3 - Extract Solution +- Read variable values using `pyo.value(model.x[j])` and `pyo.value(model.y[i])`. +- For binary variables, compare with `> 0.5` to determine activation. + +### Step 4 - Verify and Output +- **Verification Loop**: Verify mandatory coverage by checking distances against activated facilities. Verify each premium-covered demand point has at least one activated facility within `R2`. +- Print objective value with `print(f"RESULT:{float(pyo.value(model.obj))}")`. +- On failure, print JSON with status, reason, and solver details. + +### Code Usage +```python +import pyomo.environ as pyo + +# Precompute coverage sets (Step 1) +mandatory_cover = {i: tuple(j for j in facilities if dist[i][j] <= R1) for i in demands} +premium_cover = {i: tuple(j for j in facilities if dist[i][j] <= R2) for i in demands} + +# Feasibility check +for i in demands: + if not mandatory_cover[i]: + raise ValueError(f"Demand point {i} has no mandatory coverage candidate.") + +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=demand_ids) +model.J = pyo.Set(initialize=facility_ids) + +model.x = pyo.Var(model.J, domain=pyo.Binary) +model.y = pyo.Var(model.I, domain=pyo.Binary) + +def mandatory_rule(m, i): + return sum(m.x[j] for j in mandatory_cover[i]) >= 1 +model.mandatory_con = pyo.Constraint(model.I, rule=mandatory_rule) + +def premium_rule(m, i): + if premium_cover[i]: + return m.y[i] <= sum(m.x[j] for j in premium_cover[i]) + else: + return m.y[i] == 0 +model.premium_con = pyo.Constraint(model.I, rule=premium_rule) + +def obj_rule(m): + return sum(construction_cost[j] * m.x[j] for j in model.J) - sum(benefit[i] * m.y[i] for i in model.I) +model.obj = pyo.Objective(sense=pyo.minimize, rule=obj_rule) + +solver = pyo.SolverFactory("cbc") +solver.options["seconds"] = [TIME_LIMIT_S] +solver.options["ratio"] = 0.0 + +results = solver.solve(model, tee=False) +if results.solver.termination_condition in (pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible): + activated = [j for j in model.J if pyo.value(model.x[j]) > 0.5] + premium_covered = [i for i in model.I if pyo.value(model.y[i]) > 0.5] + # Verification + for i in model.I: + if not any(dist[i][j] <= R1 for j in activated): + raise RuntimeError(f"Mandatory coverage violated for demand point {i}") + for i in premium_covered: + if not any(dist[i][j] <= R2 for j in activated): + raise RuntimeError(f"Premium coverage violated for demand point {i}") + print(f"RESULT:{float(pyo.value(model.obj))}") + print(f"Activated: {activated}, Premium covered: {premium_covered}") +else: + print(f'{{"status": "FAIL", "reason": "{results.solver.termination_condition}"}}') +``` + +### Common Pitfalls +- Not setting `tee=False` in production, which can flood output with solver logs. +- Using `pyo.value()` on variables before solving, which raises an error; always check solver status first. diff --git a/skills/optskills/skill_library/facility_location_problem_solver.md b/skills/optskills/skill_library/facility_location_problem_solver.md new file mode 100644 index 0000000..2583a73 --- /dev/null +++ b/skills/optskills/skill_library/facility_location_problem_solver.md @@ -0,0 +1,220 @@ +--- +name: Facility Location Problem Solver +description: | + Models and solves uncapacitated facility location problems using binary decision variables for facility opening and customer assignment, with fixed opening costs and variable assignment costs. +--- + +# Workflow 1 (Pyomo with CBC Solver) + +## Modeling stage + +### Strategy Overview +Formulate the uncapacitated facility location problem (UFLP) as a mixed-integer linear program using Pyomo's ConcreteModel. Define binary variables for facility opening and customer assignment, enforce each customer is served exactly once and only by open facilities, and minimize the sum of fixed opening costs and variable assignment costs. + +### Step 1 - Define Sets and Parameters +- Create Pyomo Set objects for customers (`model.I`) and facilities (`model.J`) using `pyo.Set(initialize=...)`. +- Define parameter dictionaries for fixed opening costs per facility (`fixed_cost[j]`) and variable assignment costs per customer-facility pair (`assign_cost[i,j]`). Use precomputed data (e.g., a cost matrix) directly. + +### Step 2 - Declare Decision Variables +- Create binary variable `y[j]` for each facility indicating whether it is opened: `pyo.Var(model.J, domain=pyo.Binary)`. +- Create binary variable `x[i,j]` for each customer-facility pair indicating assignment: `pyo.Var(model.I, model.J, domain=pyo.Binary)`. + +### Step 3 - Build Objective and Constraints +- Define objective as `sum(fixed_cost[j] * y[j] for j in model.J) + sum(assign_cost[i,j] * x[i,j] for i in model.I for j in model.J)` with `sense=pyo.minimize`. +- Add **assignment_coverage** constraint: each customer assigned to exactly one facility: `sum(x[i,j] for j in model.J) == 1` for each customer `i`. +- Add **facility_activation** constraint: assignments only to open facilities: `x[i,j] <= y[j]` for all customer-facility pairs `(i,j)`. + +### Formulation Template +```json +{ + "sets": ["I: customers", "J: facilities"], + "parameters": ["f_j: fixed cost for facility j", "c_ij: assignment cost for customer i to facility j"], + "decision_variables": ["y_j: binary, 1 if facility j opened", "x_ij: binary, 1 if customer i assigned to facility j"], + "objective": { + "sense": "min", + "expression": "sum_j f_j * y_j + sum_i sum_j c_ij * x_ij" + }, + "constraints": [ + "sum_j x_ij == 1 for all i", + "x_ij <= y_j for all i, j" + ] +} +``` + +### Common Pitfalls +- Forgetting to set `domain=pyo.Binary` on both variable types, leading to continuous relaxation instead of MILP. +- Using mutable parameter objects when static dictionaries suffice, adding unnecessary complexity. + +## Solving stage + +### Strategy Overview +Solve the MILP using CBC solver via Pyomo's SolverFactory. Configure solver options for time limit and optimality gap, then parse results with proper status checks and threshold-based binary variable extraction. + +### Step 1 - Configure and Solve +- Instantiate solver: `solver = pyo.SolverFactory("cbc")`. +- Set solver options: `solver.options["seconds"] = [TIME_LIMIT]` and `solver.options["ratio"] = 0.0` for exact optimal solution. +- Call solve: `results = solver.solve(model, tee=False)`. + +### Step 2 - Parse Results +- Check solver status: `results.solver.status == SolverStatus.ok` and `results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}`. +- Extract objective value: `float(pyo.value(model.obj))`. +- Retrieve open facilities: `[j for j in model.J if pyo.value(model.y[j]) > 0.5]`. +- Retrieve assignments: `{(i, j) for i in model.I for j in model.J if pyo.value(model.x[i, j]) > 0.5}`. +- **Calculate cost breakdown**: Compute fixed cost component (`sum(fixed_cost[j] for j in open_facilities)`) and assignment cost component (`obj_val - fixed_cost_sum`) for solution verification. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Build model +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=customer_list) +model.J = pyo.Set(initialize=facility_list) + +model.y = pyo.Var(model.J, domain=pyo.Binary) +model.x = pyo.Var(model.I, model.J, domain=pyo.Binary) + +def obj_rule(m): + return sum(fixed_cost[j] * m.y[j] for j in m.J) + sum(assign_cost[i,j] * m.x[i,j] for i in m.I for j in m.J) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + +def assign_rule(m, i): + return sum(m.x[i,j] for j in m.J) == 1 +model.assign_con = pyo.Constraint(model.I, rule=assign_rule) + +def open_rule(m, i, j): + return m.x[i,j] <= m.y[j] +model.open_con = pyo.Constraint(model.I, model.J, rule=open_rule) + +# Solve +solver = pyo.SolverFactory("cbc") +solver.options["seconds"] = [TIME_LIMIT] +solver.options["ratio"] = 0.0 +results = solver.solve(model, tee=False) + +# Parse results +if results.solver.status == SolverStatus.ok and results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}: + obj_val = float(pyo.value(model.obj)) + open_facilities = [j for j in model.J if pyo.value(model.y[j]) > 0.5] + assignments = {(i, j) for i in model.I for j in model.J if pyo.value(model.x[i, j]) > 0.5} + # Cost breakdown for verification + fixed_cost_sum = sum(fixed_cost[j] for j in open_facilities) + assign_cost_sum = obj_val - fixed_cost_sum + result_json = {"status": "optimal", "objective": obj_val, "open_facilities": open_facilities, "assignments": assignments, "fixed_cost": fixed_cost_sum, "assignment_cost": assign_cost_sum} +else: + result_json = {"status": "infeasible"} +``` + +### Common Pitfalls +- Not checking termination condition for feasible solutions when optimal is not guaranteed, missing valid results. +- Using threshold too low (e.g., 0.1) for binary variable extraction, potentially including numerical noise. + +# Workflow 2 (OR-Tools with SCIP Solver) + +## Modeling stage + +### Strategy Overview +Formulate the uncapacitated facility location problem using Google OR-Tools' pywraplp solver interface. Define binary variables explicitly, add constraints using solver methods, and set objective coefficients directly for efficient MILP construction. + +### Step 1 - Initialize Solver and Variables +- Create solver instance: `solver = pywraplp.Solver.CreateSolver("SCIP")`. +- Create binary variables for facilities: `y = [solver.BoolVar(f"y_{j}") for j in range(num_facilities)]`. +- Create binary variables for assignments: `x = [[solver.BoolVar(f"x_{i}_{j}") for j in range(num_facilities)] for i in range(num_customers)]`. + +### Step 2 - Add Constraints +- Add **assignment_coverage** constraints: `solver.Add(sum(x[i][j] for j in range(num_facilities)) == 1)` for each customer `i`. +- Add **facility_activation** constraints: `solver.Add(x[i][j] <= y[j])` for all customer-facility pairs `(i, j)`. + +### Step 3 - Build Objective +- Create objective: `objective = solver.Objective()`. +- Set coefficients: for each facility `j`, `objective.SetCoefficient(y[j], fixed_cost[j])`; for each pair `(i,j)`, `objective.SetCoefficient(x[i][j], assign_cost[i][j])`. +- Set minimization: `objective.SetMinimization()`. + +### Formulation Template +```json +{ + "sets": ["I: customers (0..n-1)", "J: facilities (0..m-1)"], + "parameters": ["f_j: fixed cost for facility j", "c_ij: assignment cost for customer i to facility j"], + "decision_variables": ["y_j: BoolVar, 1 if facility j opened", "x_ij: BoolVar, 1 if customer i assigned to facility j"], + "objective": { + "sense": "min", + "expression": "sum_j f_j * y_j + sum_i sum_j c_ij * x_ij" + }, + "constraints": [ + "sum_j x_ij == 1 for all i", + "x_ij <= y_j for all i, j" + ] +} +``` + +### Common Pitfalls +- Using `IntVar(0, 1)` instead of `BoolVar()` for binary variables, which may cause solver inefficiency. +- Forgetting to call `objective.SetMinimization()` after setting coefficients, defaulting to maximization. + +## Solving stage + +### Strategy Overview +Solve the MILP using OR-Tools' SCIP backend with configurable time limit and parallelism. Check solver status against predefined constants and extract solution values using threshold-based filtering. + +### Step 1 - Configure and Solve +- Set time limit: `solver.SetTimeLimit([TIME_LIMIT_MS])` (time in milliseconds). +- Set thread count: `solver.SetNumThreads([NUM_THREADS])`. +- Call solve: `status = solver.Solve()`. + +### Step 2 - Parse Results +- Check status: `status == pywraplp.Solver.OPTIMAL` or `status == pywraplp.Solver.FEASIBLE`. +- Extract objective value: `solver.Objective().Value()`. +- Retrieve open facilities: `[j for j in range(num_facilities) if y[j].solution_value() > 0.5]`. +- Retrieve assignments: `[(i, j) for i in range(num_customers) for j in range(num_facilities) if x[i][j].solution_value() > 0.5]`. +- **Calculate cost breakdown**: Compute fixed cost component (`sum(fixed_cost[j] for j in open_facilities)`) and assignment cost component (`obj_val - fixed_cost_sum`) for solution verification. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# Build model +solver = pywraplp.Solver.CreateSolver("SCIP") +num_customers = len(customer_list) +num_facilities = len(facility_list) + +y = [solver.BoolVar(f"y_{j}") for j in range(num_facilities)] +x = [[solver.BoolVar(f"x_{i}_{j}") for j in range(num_facilities)] for i in range(num_customers)] + +# Constraints +for i in range(num_customers): + solver.Add(sum(x[i][j] for j in range(num_facilities)) == 1) +for i in range(num_customers): + for j in range(num_facilities): + solver.Add(x[i][j] <= y[j]) + +# Objective +objective = solver.Objective() +for j in range(num_facilities): + objective.SetCoefficient(y[j], fixed_cost[j]) +for i in range(num_customers): + for j in range(num_facilities): + objective.SetCoefficient(x[i][j], assign_cost[i][j]) +objective.SetMinimization() + +# Solve +solver.SetTimeLimit([TIME_LIMIT_MS]) +solver.SetNumThreads([NUM_THREADS]) +status = solver.Solve() + +# Parse results +if status == pywraplp.Solver.OPTIMAL or status == pywraplp.Solver.FEASIBLE: + obj_val = solver.Objective().Value() + open_facilities = [j for j in range(num_facilities) if y[j].solution_value() > 0.5] + assignments = [(i, j) for i in range(num_customers) for j in range(num_facilities) if x[i][j].solution_value() > 0.5] + # Cost breakdown for verification + fixed_cost_sum = sum(fixed_cost[j] for j in open_facilities) + assign_cost_sum = obj_val - fixed_cost_sum + result_json = {"status": "optimal", "objective": obj_val, "open_facilities": open_facilities, "assignments": assignments, "fixed_cost": fixed_cost_sum, "assignment_cost": assign_cost_sum} +else: + result_json = {"status": "infeasible"} +``` + +### Common Pitfalls +- Not converting time limit to milliseconds (OR-Tools expects milliseconds, not seconds). +- Assuming `solution_value()` returns exactly 0 or 1 for binary variables; always use a threshold for robustness. diff --git a/skills/optskills/skill_library/fixed_charge_production_planning.md b/skills/optskills/skill_library/fixed_charge_production_planning.md new file mode 100644 index 0000000..2b7b2b9 --- /dev/null +++ b/skills/optskills/skill_library/fixed_charge_production_planning.md @@ -0,0 +1,249 @@ +--- +name: Fixed-Charge Production Planning +description: | + Model and solve production planning problems with fixed activation costs and variable production costs using mixed-integer linear programming (MILP) with big-M constraints. +--- + +# Workflow 1 (OR-Tools MILP with SCIP/CBC) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) using the OR-Tools `pywraplp` API. This approach directly builds a solver model with binary activation variables linked to continuous production variables via big-M constraints, suitable for direct solving with SCIP or CBC. + +### Step 1 - Define Sets and Parameters +- Define clear index sets for production sources (e.g., `factories`) and time periods (e.g., `periods`). +- Store all problem data in dictionaries or lists indexed by these sets for easy reference during model building. This includes `fixed_cost`, `variable_cost`, `min_production`, `max_production`, and `demand`. + +### Step 2 - Create Decision Variables +- For each source `i` and period `t`, create a binary variable `run[i, t]` (0/1) representing the activation decision. +- For each source `i` and period `t`, create a continuous, non-negative variable `production[i, t]` representing the production quantity. + +### Step 3 - Link Activation and Production (Big-M) +- Add a constraint enforcing minimum production if active: `production[i, t] >= min_production[i] * run[i, t]`. +- Add a constraint enforcing maximum production (and zero if inactive): `production[i, t] <= max_production[i] * run[i, t]`. This couples the binary and continuous variables. + +### Step 4 - Enforce Demand Satisfaction +- For each time period `t`, add a linear constraint ensuring total production meets demand: `sum(production[i, t] for i in sources) >= demand[t]`. + +### Step 5 - Define Linear Cost Objective +- Define the objective to minimize total cost: `sum(fixed_cost[i] * run[i, t] + variable_cost[i] * production[i, t] for all i, t)`. + +### Formulation Template +```json +{ + "sets": [ + "sources", + "periods" + ], + "parameters": [ + "fixed_cost[sources]", + "variable_cost[sources]", + "min_production[sources]", + "max_production[sources]", + "demand[periods]" + ], + "decision_variables": [ + "run[sources, periods] ∈ {0, 1}", + "production[sources, periods] ≥ 0" + ], + "objective": { + "sense": "min", + "expression": "Σ_i Σ_t (fixed_cost[i] * run[i, t] + variable_cost[i] * production[i, t])" + }, + "constraints": [ + "production_lower[i,t]: production[i, t] ≥ min_production[i] * run[i, t]", + "production_upper[i,t]: production[i, t] ≤ max_production[i] * run[i, t]", + "demand_satisfaction[t]: Σ_i production[i, t] ≥ demand[t]" + ] +} +``` + +### Common Pitfalls +- Forgetting to set an upper bound for the continuous `production` variable, which can default to infinity. +- Using an excessively large "M" value in the upper bound constraint; the parameter `max_production[i]` serves as a tight, data-driven big-M. +- Creating variables and constraints with manual loops that risk missing indices; use nested loops over the defined sets. + +## Solving stage + +### Strategy Overview +Solve the built MILP model using the OR-Tools wrapper for SCIP or CBC. Configure solver limits for performance, check the solution status rigorously, and extract results into a structured format for verification and analysis. + +### Step 1 - Initialize Solver and Set Parameters +- Create the solver instance (e.g., `solver = pywraplp.Solver.CreateSolver("SCIP")`). Provide a fallback to "CBC" if preferred solver is unavailable. +- Set practical limits: `solver.SetTimeLimit(time_limit_ms)` and `solver.SetNumThreads(num_threads)` to balance speed and resource use. + +### Step 2 - Solve and Check Status +- Invoke `solver.Solve()`. +- Check the result status: `status = solver.optimal()` or `solver.feasible()`. Proceed only if status is `True`; otherwise, handle infeasible or non-optimal outcomes. + +### Step 3 - Extract and Verify Solution +- Extract the objective value: `total_cost = solver.Objective().Value()`. +- Extract variable values by iterating over all indices: `run_val = run[i, t].solution_value()` and `prod_val = production[i, t].solution_value()`. +- Implement a post-solve verification: recompute aggregated production per period and ensure it meets demand within a small tolerance. Validate that production is zero when `run` is 0 and within bounds when `run` is 1. + +### Step 4 - Output Structured Results +- Compile results into a dictionary or JSON object containing the status, objective value, activation decisions, production levels, and aggregated totals (e.g., monthly production). +- This structured output aids in debugging, reporting, and further analysis. + +### Code Usage +```python +# build model from formulation +from ortools.linear_solver import pywraplp + +solver = pywraplp.Solver.CreateSolver("SCIP") +if not solver: + solver = pywraplp.Solver.CreateSolver("CBC") + +# Define sets, parameters, and create variables as per modeling steps. +# ... (variable creation loops) +# ... (constraint addition loops) +# ... (objective definition) + +# solve with status / termination checks +solver.SetTimeLimit(30000) # 30 seconds +solver.SetNumThreads(4) +status = solver.Solve() + +if status == pywraplp.Solver.OPTIMAL or status == pywraplp.Solver.FEASIBLE: + total_cost = solver.Objective().Value() + solution = {"status": "optimal" if status == pywraplp.Solver.OPTIMAL else "feasible", + "objective": total_cost, + "decisions": {}} + # Extract variable values into solution['decisions'] + # ... (extraction loops) + # Post-solve verification + # ... (verification logic) +else: + solution = {"status": "infeasible_or_unbounded", "objective": None} +``` + +### Common Pitfalls +- Not checking solver status before extracting variable values, which can cause runtime errors. +- Assuming the solver found an optimal solution; always check for `OPTIMAL` or `FEASIBLE` status. +- Ignoring numerical precision when verifying constraints; use a small tolerance (e.g., `1e-6`) for comparisons. + +# Workflow 2 (Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +Model the problem using Pyomo's abstract or concrete modeling environment. This approach separates the problem formulation from the solver interface, improving readability and maintainability. The model is then solved using an external MILP solver like HiGHS or CBC. + +### Step 1 - Define Pyomo Sets and Parameters +- Use `pyo.Set` objects to define index sets for sources and periods (e.g., `model.SOURCES`, `model.PERIODS`). +- Use `pyo.Param` objects or Python dictionaries to store all cost, capacity, and demand parameters, indexed by the defined sets. + +### Step 2 - Declare Decision Variables +- Declare a binary variable `model.run` indexed over sources and periods, with domain `pyo.Binary`. +- Declare a continuous, non-negative variable `model.production` indexed over the same sets, with domain `pyo.NonNegativeReals`. + +### Step 3 - Implement Constraint Rules +- Define a rule function for the production lower bound: `model.production_lower = pyo.Constraint(model.SOURCES, model.PERIODS, rule=prod_lower_rule)`. Inside the rule, return `model.production[i, t] >= min_prod[i] * model.run[i, t]`. +- Similarly, define rules for the production upper bound and demand satisfaction constraints. + +### Step 4 - Define Objective Rule +- Define an objective rule function that sums the fixed and variable costs across all indices: `model.total_cost = pyo.Objective(rule=obj_rule, sense=pyo.minimize)`. + +### Formulation Template +```json +{ + "sets": [ + "SOURCES", + "PERIODS" + ], + "parameters": [ + "fixed_cost[SOURCES]", + "variable_cost[SOURCES]", + "min_production[SOURCES]", + "max_production[SOURCES]", + "demand[PERIODS]" + ], + "decision_variables": [ + "run[SOURCES, PERIODS] ∈ Binary", + "production[SOURCES, PERIODS] ≥ 0" + ], + "objective": { + "sense": "min", + "expression": "sum(fixed_cost[i] * run[i, t] + variable_cost[i] * production[i, t] for i in SOURCES, t in PERIODS)" + }, + "constraints": [ + "production_lower[i,t]: production[i, t] ≥ min_production[i] * run[i, t]", + "production_upper[i,t]: production[i, t] ≤ max_production[i] * run[i, t]", + "demand_satisfaction[t]: sum(production[i, t] for i in SOURCES) ≥ demand[t]" + ] +} +``` + +### Common Pitfalls +- Defining constraint or objective rules with incorrect indexing, leading to `KeyError` or missing constraints. +- Using mutable default arguments (like lists) inside Pyomo rule functions. +- Confusing Pyomo's `value()` function with variable values during model construction; `pyo.value()` is for evaluating expressions *after* solving. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model by instantiating a solver object (e.g., HiGHS, CBC) via `SolverFactory`. Configure solver options for performance and gap tolerance, enable output for debugging, and rigorously check the termination condition before extracting and verifying the solution. + +### Step 1 - Select and Configure Solver +- Create a solver instance: `solver = pyo.SolverFactory("highs")`. Implement a fallback check (e.g., if `solver is None`, try `"cbc"`). +- Set key options: `solver.options["time_limit"] = time_limit`, `solver.options["mip_rel_gap"] = 0.0` (for exact solution), and `solver.options["threads"] = num_threads`. + +### Step 2 - Solve with Diagnostics +- Invoke the solver with `results = solver.solve(model, tee=True)`. The `tee=True` flag streams solver output, which is invaluable for debugging and understanding progress. + +### Step 3 - Check Solver Status and Termination +- Check if the solver run completed: `assert results.solver.status == pyo.SolverStatus.ok`. +- Check the termination condition: `if results.solver.termination_condition in [pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible]:`. Only extract results under these conditions. + +### Step 4 - Extract and Validate Solution +- Extract the objective value: `total_cost = pyo.value(model.total_cost)`. +- Iterate through variables to extract values: `run_val = pyo.value(model.run[i, t])`, `prod_val = pyo.value(model.production[i, t])`. For binary variables, interpret values using a threshold (e.g., `> 0.5`). +- Programmatically verify all constraints: check production bounds against activation status and ensure demand is met. Recalculate the total cost from extracted values to confirm numerical consistency. + +### Step 5 - Serialize and Output Results +- Compile results into a structured format. Convert tuple keys (e.g., `(i, t)`) to strings (e.g., `f"source{i}_period{t}"`) for easy JSON serialization. +- Include cost breakdowns (fixed vs. variable) and aggregated production totals in the output. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo + +model = pyo.ConcreteModel() +model.SOURCES = pyo.Set(initialize=sources_list) +model.PERIODS = pyo.Set(initialize=periods_list) +# Define parameters (as Pyomo Param or dictionaries) +# ... (parameter definitions) +model.run = pyo.Var(model.SOURCES, model.PERIODS, domain=pyo.Binary) +model.production = pyo.Var(model.SOURCES, model.PERIODS, domain=pyo.NonNegativeReals) +# Define constraints via rules +# ... (constraint definitions) +model.total_cost = pyo.Objective(rule=objective_rule, sense=pyo.minimize) + +# solve with status / termination checks +solver = pyo.SolverFactory("highs") +if solver is None: + solver = pyo.SolverFactory("cbc") +solver.options["time_limit"] = 30 +solver.options["mip_rel_gap"] = 0.0 +solver.options["threads"] = 4 + +results = solver.solve(model, tee=True) + +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in [pyo.TerminationCondition.optimal, + pyo.TerminationCondition.feasible]): + total_cost = pyo.value(model.total_cost) + solution_summary = {"status": results.solver.termination_condition.name, + "objective": total_cost} + # Extract and verify variable values + # ... (extraction and verification loops) +else: + solution_summary = {"status": "failed", "termination_condition": results.solver.termination_condition.name} +``` + +### Common Pitfalls +- Not verifying `solver is None` after `SolverFactory`, which can lead to cryptic errors if the requested solver is not installed. +- Accessing variable values via `pyo.value()` before checking the termination condition, which may load invalid or stale values. +- Forgetting to handle the case where the solver finds a feasible but non-optimal solution; the extraction logic should still proceed. diff --git a/skills/optskills/skill_library/flexible_job_shop_scheduling_with_arbitrary_routing.md b/skills/optskills/skill_library/flexible_job_shop_scheduling_with_arbitrary_routing.md new file mode 100644 index 0000000..0825fa9 --- /dev/null +++ b/skills/optskills/skill_library/flexible_job_shop_scheduling_with_arbitrary_routing.md @@ -0,0 +1,278 @@ +--- +name: Flexible Job Shop Scheduling with Arbitrary Routing +description: | + Model and solve flexible job shop problems where each job must visit all machines exactly once in any order, minimizing makespan using either CP-SAT with permutation variables or MILP with disjunctive constraints. + +--- + +# Workflow 1 (CP-SAT with Permutation Variables) + +## Modeling stage + +### Strategy Overview +This approach models the problem as a bipartite matching between jobs and machines using permutation variables, enforcing no-overlap via conditional precedence constraints. It is well-suited for CP-SAT solvers which handle `AllDifferent` and `OnlyEnforceIf` efficiently. + +### Step 1 - Define Core Variables +- Define integer variables `position[j, m]` representing the sequence order of job `j` on machine `m`. Domain is `[0, n_jobs - 1]`. +- Define integer variables `machine_order[j, m]` representing the processing order of machine `m` for job `j`. Domain is `[0, n_machines - 1]`. +- Define continuous or integer variables `start_time[j, m]` for the start time of each operation. + +### Step 2 - Enforce Bijection and Permutation Constraints +- For each job `j`, enforce `AllDifferent` on `position[j, m]` across all machines `m`. This ensures each job has a unique position on each machine. +- For each machine `m`, enforce `AllDifferent` on `position[j, m]` across all jobs `j`. This ensures each position on a machine is occupied by exactly one job. +- For each job `j`, enforce `AllDifferent` on `machine_order[j, m]` across all machines `m`. This defines a total order of machines for the job. + +### Step 3 - Model Machine No-Overlap with Conditional Precedence +- For each machine `m` and each unordered pair of distinct jobs `(i, j)`, create a boolean variable `precedes[i, j, m]`. +- Link the boolean variable to the permutation: `position[i, m] < position[j, m]` implies `precedes[i, j, m] = True`. Use `OnlyEnforceIf`. +- Enforce the disjunctive constraint: `start_time[i, m] + processing_time[i, m] <= start_time[j, m]` only if `precedes[i, j, m]` is true. + +### Step 4 - Model Job No-Overlap with Machine Order Variables +- For each job `j` and each unordered pair of distinct machines `(a, b)`, create a boolean variable `a_before_b[j, a, b]`. +- Link the boolean variable to the machine order: `machine_order[j, a] < machine_order[j, b]` implies `a_before_b[j, a, b] = True`. Use `OnlyEnforceIf`. +- Enforce the precedence constraint: `start_time[j, a] + processing_time[j, a] <= start_time[j, b]` only if `a_before_b[j, a, b]` is true. + +### Step 5 - Define Makespan Objective +- Create a makespan variable `C_max`. +- For each job `j` and machine `m`, add constraint: `start_time[j, m] + processing_time[j, m] <= C_max`. +- Set the objective to minimize `C_max`. + +### Formulation Template +```json +{ + "sets": [ + "Jobs", + "Machines" + ], + "parameters": [ + "processing_time[j in Jobs][m in Machines]" + ], + "decision_variables": [ + "position[j in Jobs][m in Machines] ∈ {0, ..., |Jobs|-1}", + "machine_order[j in Jobs][m in Machines] ∈ {0, ..., |Machines|-1}", + "start_time[j in Jobs][m in Machines] ≥ 0", + "precedes[i in Jobs, j in Jobs, m in Machines where i < j] ∈ {0,1}", + "a_before_b[j in Jobs, a in Machines, b in Machines where a < b] ∈ {0,1}", + "C_max ≥ 0" + ], + "objective": { + "sense": "min", + "expression": "C_max" + }, + "constraints": [ + "AllDifferent(position[j, :]) for all j in Jobs", + "AllDifferent(position[:, m]) for all m in Machines", + "AllDifferent(machine_order[j, :]) for all j in Jobs", + "position[i,m] < position[j,m] ↔ precedes[i,j,m] for all i= end[j][m]`. +- Set the objective to minimize the `makespan` variable. + +### Formulation Template +```json +{ + "sets": [ + "jobs", + "machines" + ], + "parameters": [ + "processing_time[job][machine]" + ], + "decision_variables": [ + "start[job][machine] (integer)", + "end[job][machine] (integer)", + "interval[job][machine] (interval)", + "makespan (integer)" + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + "end[j][m] == start[j][m] + processing_time[j][m]", + "NoOverlap([interval[j][m] for m in machines]) for each job j", + "NoOverlap([interval[j][m] for j in jobs]) for each machine m", + "makespan >= end[j][m] for all j, m" + ] +} +``` + +### Common Pitfalls +- Forgetting to define the `interval` variable with the correct start, end, and size, leading to incorrect `NoOverlap` behavior. +- Setting an overly loose upper bound for `makespan` (e.g., sum of all processing times) can slow down search; always use the tighter bound `max(max_machine_load, max_job_load)`. +- Not utilizing solver parallelism; CP-SAT benefits from multiple search workers. + +## Solving stage + +### Strategy Overview +Solve the CP model using OR-Tools CP-SAT solver with appropriate time limits and parallel search. Extract and validate the schedule from the solution. + +### Step 1 - Configure Solver +- Instantiate the `CpSolver` and set key parameters: a time limit (`max_time_in_seconds`), number of parallel workers (`num_search_workers`), and a random seed for reproducibility. +- Optionally, set `relative_gap_limit = 0.0` to aim for proven optimality. + +### Step 2 - Solve and Check Status +- Call `solver.Solve(model)` and capture the status. +- Check if the status is `OPTIMAL` or `FEASIBLE` before proceeding to extract the solution. + +### Step 3 - Extract and Validate Schedule +- For each variable `start[j][m]` and `end[j][m]`, retrieve its value using `solver.Value()`. +- Reconstruct the sequence of operations for each job and each machine by sorting operations by their start times. +- Verify that no overlaps exist in the extracted schedule as a sanity check. +- For verification, calculate the lower bound `max(max_machine_load, max_job_load)`. If the solution value equals this bound, optimality is proven. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# Build model +model = cp_model.CpModel() +# ... (create variables and constraints as per Modeling Stage) + +# Solve +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = [NUM_WORKERS] +solver.parameters.random_seed = [SEED] +solver.parameters.relative_gap_limit = 0.0 + +status = solver.Solve(model) + +# Check status and extract solution +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + makespan_value = solver.Value(makespan) + schedule = {} + for j in jobs: + for m in machines: + schedule[(j, m)] = (solver.Value(start[j][m]), solver.Value(end[j][m])) + # ... (process schedule) +else: + print("No solution found.") +``` + +### Common Pitfalls +- Not checking solver status before accessing variable values, which can cause runtime errors. +- Interpreting `FEASIBLE` status as optimal; if optimality is required, explicitly check for `OPTIMAL`. +- Using default solver parameters for large instances; always set a time limit and consider enabling logging (`solver.parameters.log_search_progress = True`) for debugging. + +# Workflow 2 (MILP with Big-M Disjunctions) + +## Modeling stage + +### Strategy Overview +Model the problem as a Mixed-Integer Linear Program (MILP) using a Big-M formulation with explicit binary sequencing variables. This provides explicit control over precedence relationships and is suitable for solvers like Gurobi or CBC. + +### Step 1 - Define Core Variables +- Create continuous variables `start[j][m]` and `completion[j][m]` for each job `j` and machine `m`. +- Create binary variables `prec[j][m1][m2]` for each job `j` and distinct machine pair `(m1, m2)`, where `1` indicates `m1` precedes `m2` for job `j`. +- Create binary variables `seq[i][j][m]` for each machine `m` and distinct job pair `(i, j)`, where `1` indicates job `i` precedes job `j` on machine `m`. +- Create a continuous variable `makespan` to be minimized. + +### Step 2 - Link Times and Define Processing +- Add constraints: `completion[j][m] == start[j][m] + processing_time[j][m]` for all `j, m`. + +### Step 3 - Enforce Job Non-Overlap via Precedence +- For each job `j` and each distinct machine pair `(m1, m2)`, add a Big-M constraint: `start[j][m2] >= completion[j][m1] - BigM * (1 - prec[j][m1][m2])`. +- Add antisymmetry constraints: `prec[j][m1][m2] + prec[j][m2][m1] == 1` for all `j, m1, m2` where `m1 != m2`. + +### Step 4 - Enforce Machine Non-Overlap via Sequencing +- For each machine `m` and each distinct job pair `(i, j)`, add a Big-M constraint: `start[j][m] >= completion[i][m] - BigM * (1 - seq[i][j][m])`. +- Add antisymmetry constraints: `seq[i][j][m] + seq[j][i][m] == 1` for all `i, j, m` where `i != j`. + +### Step 5 - Define Makespan and Objective +- Add constraints: `makespan >= completion[j][m]` for all `j, m`. +- Set the objective to minimize `makespan`. + +### Formulation Template +```json +{ + "sets": [ + "jobs", + "machines" + ], + "parameters": [ + "processing_time[job][machine]", + "BigM (large constant)" + ], + "decision_variables": [ + "start[job][machine] (continuous)", + "completion[job][machine] (continuous)", + "prec[job][machine1][machine2] (binary)", + "seq[job1][job2][machine] (binary)", + "makespan (continuous)" + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + "completion[j][m] == start[j][m] + processing_time[j][m]", + "start[j][m2] >= completion[j][m1] - BigM * (1 - prec[j][m1][m2]) for j, m1≠m2", + "prec[j][m1][m2] + prec[j][m2][m1] == 1 for j, m1≠m2", + "start[j][m] >= completion[i][m] - BigM * (1 - seq[i][j][m]) for m, i≠j", + "seq[i][j][m] + seq[j][i][m] == 1 for m, i≠j", + "makespan >= completion[j][m] for all j, m" + ] +} +``` + +### Common Pitfalls +- Choosing a `BigM` value that is too large, leading to numerical instability and weak LP relaxations; use a tight bound like `max(max_machine_load, max_job_load)`. +- Forgetting the antisymmetry constraints for precedence/sequence variables, resulting in incomplete or invalid orders. +- Creating redundant variables or constraints (e.g., `prec[j][m][m]`) which increase model size unnecessarily. + +## Solving stage + +### Strategy Overview +Solve the MILP model using a MIP solver via a modeling framework like Pyomo. Configure solver parameters for performance, verify termination status, and extract the schedule. + +### Step 1 - Build Model and Configure Solver +- Instantiate a concrete model in Pyomo, defining sets, parameters, variables, constraints, and the objective as per the formulation. +- Create a solver object (e.g., for Gurobi, CBC) and set parameters: time limit, MIP gap tolerance (`MIPGap`), number of threads, and a random seed if applicable. + +### Step 2 - Solve and Verify Termination +- Call the solver on the model and capture the results. +- Check the solver status (`SolverStatus.ok`) and termination condition (`TerminationCondition.optimal` or `.feasible`) to ensure a valid solution is available. + +### Step 3 - Extract and Analyze Solution +- Retrieve variable values using `pyo.value()` for `start`, `completion`, and the binary precedence/sequence variables. +- Reconstruct the operation sequence per job and per machine by inspecting the binary variable values. +- Validate the schedule by checking for constraint violations (e.g., overlaps, precedence satisfaction). + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Build model +model = pyo.ConcreteModel() +# ... (define sets, parameters, variables, constraints, objective as per Modeling Stage) + +# Solve +solver = pyo.SolverFactory('[SOLVER_NAME]') # e.g., 'gurobi' or 'cbc' +solver.options['TimeLimit'] = [TIME_LIMIT] +solver.options['MIPGap'] = [GAP_TOLERANCE] +solver.options['Threads'] = [NUM_THREADS] +results = solver.solve(model, tee=True) # tee=True prints solver log + +# Check status and extract solution +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in (TerminationCondition.optimal, + TerminationCondition.feasible)): + makespan_value = pyo.value(model.makespan) + schedule = {} + for j in model.jobs: + for m in model.machines: + schedule[(j, m)] = (pyo.value(model.start[j, m]), + pyo.value(model.completion[j, m])) + # ... (process schedule) +else: + print("No solution found.") +``` + +### Common Pitfalls +- Not checking both solver status and termination condition, leading to extraction attempts from infeasible or error states. +- Using a solver without a proper license or installation (e.g., Gurobi); have a fallback like CBC. +- Ignoring solver logs; setting `tee=True` helps monitor progress and identify early issues like numerical instability. diff --git a/skills/optskills/skill_library/flow_shop_scheduling_with_makespan_minimization.md b/skills/optskills/skill_library/flow_shop_scheduling_with_makespan_minimization.md new file mode 100644 index 0000000..dec01c1 --- /dev/null +++ b/skills/optskills/skill_library/flow_shop_scheduling_with_makespan_minimization.md @@ -0,0 +1,229 @@ +--- +name: Flow Shop Scheduling with Makespan Minimization +description: | + Model permutation flow shop problems with unary machine resources and precedence chains, then solve using either CP-SAT for exact solutions or MILP with symmetry breaking for heuristic/optimal results. + +--- +# Workflow 1 (CP-SAT with Sequence Variables) + +## Modeling stage + +### Strategy Overview +Model the problem using Constraint Programming (CP-SAT) with explicit binary precedence variables for each machine, enforcing disjunctive constraints via big-M and adding transitivity for stronger propagation. This approach directly captures the permutation sequence. + +### Step 1 - Define Time and Precedence Variables +- Create integer variables `start[j][m]` and `end[j][m]` for each job `j` and machine `m` within a global horizon. +- Create binary variables `precedes[i][j][m]` for each pair of distinct jobs `i, j` and each machine `m`, where `1` indicates job `i` is processed before job `j` on machine `m`. + +### Step 2 - Enforce Job Precedence Chains +- For each job `j` and for each machine `m` except the last, add constraint: `end[j][m] <= start[j][m+1]`. This ensures the operation sequence for each job follows the machine order. + +### Step 3 - Enforce Machine Unary Resources +- For each machine `m` and each pair of distinct jobs `i, j`, add a disjunctive constraint using big-M: `start[j][m] >= end[i][m] - M * (1 - precedes[i][j][m])`. Set `M` to a large constant (e.g., sum of all processing times). +- Add mutual exclusivity: `precedes[i][j][m] + precedes[j][i][m] == 1` for all `i < j` and each machine `m`. + +### Step 4 - Strengthen with Transitivity (Optional) +- To improve solver performance, add transitivity constraints for each machine `m` and all distinct `i, j, k`: `precedes[i][j][m] + precedes[j][k][m] - precedes[i][k][m] <= 1`. + +### Step 5 - Define Makespan Objective +- Create an integer variable `makespan`. +- For each job `j`, add constraint: `makespan >= end[j][last_machine]`. +- Set the objective to minimize `makespan`. + +### Formulation Template +```json +{ + "sets": [ + "Jobs", + "Machines" + ], + "parameters": [ + "processing_time[j][m]", + "horizon", + "big_M" + ], + "decision_variables": [ + "start[j][m] (integer, [0, horizon])", + "end[j][m] (integer, [0, horizon])", + "precedes[i][j][m] (binary)" + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + "end[j][m] == start[j][m] + processing_time[j][m]", + "end[j][m] <= start[j][m+1] for m < last_machine", + "start[j][m] >= end[i][m] - big_M * (1 - precedes[i][j][m]) for all i != j, m", + "precedes[i][j][m] + precedes[j][i][m] == 1 for all i < j, m", + "makespan >= end[j][last_machine] for all j" + ] +} +``` + +### Common Pitfalls +- Setting `big_M` too small, which can cut off feasible solutions. +- Forgetting to enforce the precedence chain for all jobs, leading to invalid schedules. +- Omitting transitivity constraints can result in weaker propagation and longer solve times. + +## Solving stage + +### Strategy Overview +Use the OR-Tools CP-SAT solver, configured for parallel search and a time limit, to find optimal or feasible solutions. Extract and validate the schedule. + +### Step 1 - Solver Configuration +- Instantiate the CP-SAT solver (`cp_model.CpSolver()`). +- Set `solver.parameters.max_time_in_seconds` to a reasonable limit (e.g., `[TIME_LIMIT]`). +- Set `solver.parameters.num_search_workers` to the number of available CPU cores. +- Set `solver.parameters.random_seed` for reproducibility. + +### Step 2 - Solve and Check Status +- Call `solver.Solve(model)` and capture the status. +- If status is `OPTIMAL`, the solution is proven optimal. If `FEASIBLE`, it is a valid but not necessarily optimal solution. Handle `UNKNOWN` or `INFEASIBLE` status appropriately. + +### Step 3 - Solution Extraction +- If a solution was found, use `solver.Value(variable)` to retrieve the value of each `start[j][m]` and `precedes[i][j][m]` variable. +- Compute `end[j][m]` as `start[j][m] + processing_time[j][m]`. +- The makespan is `solver.Value(makespan)`. +- To get the job sequence per machine, sort jobs by their `start` time on that machine. + +### Step 4 - Validation for Small Instances +- For problems with a small number of jobs (e.g., <= 8), validate the CP-SAT solution by enumerating all permutations and calculating the makespan via dynamic programming to verify correctness. + +### Code Usage +```python +# build model from formulation +model = cp_model.CpModel() +# ... define variables, constraints, objective as per modeling stage + +# solve with status / termination checks +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = [NUM_WORKERS] +status = solver.Solve(model) + +if status in [cp_model.OPTIMAL, cp_model.FEASIBLE]: + makespan = solver.Value(makespan_var) + schedule = {(j, m): solver.Value(start_var[j][m]) for j in jobs for m in machines} + # ... process schedule +else: + print("No solution found.") +``` + +### Common Pitfalls +- Not checking solver status before extracting variable values, leading to errors. +- Misinterpreting `FEASIBLE` status as optimal. +- Forgetting to set a time limit for potentially large instances. + +# Workflow 2 (MILP with Disjunctive Sequencing) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) using disjunctive constraints with big-M and explicit sequencing variables. This approach is suitable for solvers like Gurobi or CBC and benefits from strong symmetry-breaking constraints. + +### Step 1 - Define Continuous Start Variables +- Create continuous (or integer) variables `s[j][m]` representing the start time of job `j` on machine `m`. + +### Step 2 - Enforce Precedence Chains +- For each job `j` and machine `m` except the last, add constraint: `s[j][m] + p[j][m] <= s[j][m+1]`, where `p` is the processing time. + +### Step 3 - Model Machine Disjunctions +- For each machine `m` and each pair of distinct jobs `i, j`, create a binary variable `y[i][j][m]` where `1` means job `i` precedes job `j` on machine `m`. +- Add disjunctive constraints: `s[i][m] + p[i][m] <= s[j][m] + M * (1 - y[i][j][m])` and `s[j][m] + p[j][m] <= s[i][m] + M * y[i][j][m]`. `M` is a large constant. + +### Step 4 - Add Symmetry-Breaking Constraints +- For each machine `m` and each pair `i < j`, add constraint: `y[i][j][m] + y[j][i][m] == 1`. This ensures a total order. +- Optionally, add constraints to break symmetry across identical machines or jobs. + +### Step 5 - Define Makespan Objective +- Create a continuous variable `C_max`. +- For each job `j`, add constraint: `C_max >= s[j][last_machine] + p[j][last_machine]`. +- Set the objective to minimize `C_max`. + +### Formulation Template +```json +{ + "sets": [ + "Jobs", + "Machines" + ], + "parameters": [ + "processing_time[j][m]", + "big_M" + ], + "decision_variables": [ + "s[j][m] (continuous, >=0)", + "y[i][j][m] (binary)", + "C_max (continuous)" + ], + "objective": { + "sense": "min", + "expression": "C_max" + }, + "constraints": [ + "s[j][m] + processing_time[j][m] <= s[j][m+1] for m < last_machine", + "s[i][m] + processing_time[i][m] <= s[j][m] + big_M * (1 - y[i][j][m]) for all i != j, m", + "s[j][m] + processing_time[j][m] <= s[i][m] + big_M * y[i][j][m] for all i != j, m", + "y[i][j][m] + y[j][i][m] == 1 for all i < j, m", + "C_max >= s[j][last_machine] + processing_time[j][last_machine] for all j" + ] +} +``` + +### Common Pitfalls +- Using an insufficiently large `big_M`, which can make the model infeasible. +- Creating an excessive number of binary variables for large job sets, impacting performance. +- Neglecting symmetry-breaking constraints, leading to a bloated search space. + +## Solving stage + +### Strategy Overview +Use a MILP solver (e.g., Gurobi, CBC) with tuned parameters to find optimal or good feasible solutions within a time limit. Extract and interpret the schedule. + +### Step 1 - Solver Configuration and Model Submission +- Instantiate the solver and load the model. +- Set a time limit (e.g., `model.setParam('TimeLimit', [TIME_LIMIT])`). +- Set optimality tolerance (e.g., `model.setParam('MIPGap', [TOLERANCE])`). +- Set the number of threads (e.g., `model.setParam('Threads', [NUM_THREADS])`). +- Set a random seed for reproducibility if supported. + +### Step 2 - Solve and Check Termination Status +- Call `model.optimize()`. +- Check the status: `OPTIMAL`, `FEASIBLE`, `TIME_LIMIT`, or `INFEASIBLE`. +- If `OPTIMAL` or `FEASIBLE`, proceed to extract solution. For `TIME_LIMIT`, the best solution found is still valid. + +### Step 3 - Extract and Interpret Solution +- Retrieve the objective value `model.objVal` (makespan). +- Get variable values: `s[j][m].X` for start times, `y[i][j][m].X` for sequencing. +- The sequence on each machine is determined by jobs where `y[i][j][m].X` is (approximately) 1. + +### Step 4 - Validation and Reporting +- For small instances, validate the MILP solution via full permutation enumeration. +- Generate a human-readable schedule listing job sequences per machine and start/end times. + +### Code Usage +```python +# build model from formulation +import gurobipy as gp +model = gp.Model() +# ... define variables, constraints, objective as per modeling stage + +# solve with status / termination checks +model.setParam('TimeLimit', [TIME_LIMIT]) +model.setParam('MIPGap', [TOLERANCE]) +model.setParam('Threads', [NUM_THREADS]) +model.optimize() + +if model.status in [gp.GRB.OPTIMAL, gp.GRB.TIME_LIMIT, gp.GRB.SUBOPTIMAL]: + makespan = model.objVal + schedule = {(j, m): s[j][m].X for j in jobs for m in machines} + # ... process schedule +else: + print("No feasible solution found.") +``` + +### Common Pitfalls +- Not handling `TIME_LIMIT` status, which may still provide a useful incumbent solution. +- Assuming binary variables are exactly 0 or 1; use a tolerance (e.g., `if var.X > 0.5`). +- Forgetting to suppress solver log output, cluttering the console. diff --git a/skills/optskills/skill_library/flowshopscheduling.md b/skills/optskills/skill_library/flowshopscheduling.md new file mode 100644 index 0000000..c6faff6 --- /dev/null +++ b/skills/optskills/skill_library/flowshopscheduling.md @@ -0,0 +1,201 @@ +--- +name: FlowShopScheduling +description: | + Model and solve flow shop scheduling problems with disjunctive resources and precedence constraints to minimize makespan using either CP-SAT with interval variables or MIP with big-M disjunctive constraints. + +--- +# Workflow 1 (CP-SAT with Interval Variables) + +## Modeling stage + +### Strategy Overview +This workflow uses Google OR-Tools CP-SAT solver, modeling jobs as interval variables on each machine. It leverages native `AddNoOverlap` constraints for disjunctive resources and enforces precedence via linear constraints. The model is concise and benefits from CP-SAT's dedicated scheduling algorithms. + +### Step 1 - Define Problem Sets and Horizon +- Define sets for `jobs` and `machines`. The sequence of machines is fixed for all jobs. +- Compute a horizon upper bound as the sum of all processing times across all jobs and machines. +- Create a dictionary `processing_time[j][m]` for the duration of job `j` on machine `m`. + +### Step 2 - Create Interval Variables +- For each job `j` and machine `m`, create an interval variable `interval[j][m]`. Its start is `start_var[j][m]`, its size is fixed to `processing_time[j][m]`, and its end is `end_var[j][m]`. All variables are defined within the horizon. + +### Step 3 - Enforce Precedence Constraints +- For each job `j` and for each machine `m` (except the last), add a constraint: `end_var[j][m] <= start_var[j][m+1]`. This ensures the job's operation on machine `m` finishes before it starts on machine `m+1`. + +### Step 4 - Enforce Disjunctive Resources +- For each machine `m`, add a `AddNoOverlap` constraint over the list of interval variables for all jobs on that machine: `solver.AddNoOverlap([interval[j][m] for j in jobs])`. This ensures no two jobs overlap on the same machine. + +### Step 5 - Define Makespan and Objective +- Create a makespan variable. For each job `j`, add a constraint: `makespan >= end_var[j][last_machine]`. +- Set the objective to minimize the makespan variable. + +### Formulation Template +```json +{ + "sets": ["jobs", "machines"], + "parameters": ["processing_time[j][m]"], + "decision_variables": ["start_var[j][m]", "interval[j][m]", "makespan"], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + "precedence: end_var[j][m] <= start_var[j][m+1] for all j, m < last_machine", + "disjunctive: NoOverlap([interval[j][m] for j in jobs]) for all m", + "makespan_definition: makespan >= end_var[j][last_machine] for all j" + ] +} +``` + +### Common Pitfalls +- Forgetting to set an upper bound for the horizon, which can lead to inefficient solving. +- Misindexing machines when adding precedence constraints, leading to an invalid sequence. +- Using `AddNoOverlap` on the wrong list of intervals (e.g., mixing machines), which does not correctly model machine capacity. + +## Solving stage + +### Strategy Overview +Configure the CP-SAT solver with time and parallel search settings, then solve and extract the schedule. Solution verification includes checking constraint satisfaction and, for small instances, validating optimality via enumeration. + +### Step 1 - Configure Solver Parameters +- Set `solver.parameters.max_time_in_seconds` to a reasonable limit (e.g., 30). +- Set `solver.parameters.num_search_workers` to the number of CPU cores (e.g., 8). +- Set `solver.parameters.random_seed` for reproducibility (e.g., 42). +- For exact optimization, set `solver.parameters.relative_gap_limit = 0.0`. + +### Step 2 - Solve and Check Status +- Invoke `solver.Solve(model)`. +- Check the status: `OPTIMAL`, `FEASIBLE`, or `INFEASIBLE`. Handle each case appropriately in output. + +### Step 3 - Extract and Validate Schedule +- If a solution is found, collect the values of `start_var[j][m]` and `end_var[j][m]`. +- Programmatically verify all constraints: precedence and no-overlap per machine. +- For problems with `n <= 6` jobs, perform an exhaustive permutation check to confirm global optimality. + +### Step 4 - Output Schedule and Metrics +- Print the makespan value. +- Output a Gantt chart or a table listing job start/end times per machine. +- Report machine utilization and idle times for analysis. + +### Code Usage +```python +# build model from formulation +model = cp_model.CpModel() +# ... (create variables, add constraints, set objective) +# solve with status / termination checks +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30 +solver.parameters.num_search_workers = 8 +status = solver.Solve(model) +# check status and extract solution +if status in [cp_model.OPTIMAL, cp_model.FEASIBLE]: + makespan = solver.Value(makespan_var) + # ... extract start/end times +``` + +### Common Pitfalls +- Not checking solver status, leading to errors when trying to access values from an infeasible model. +- Assuming `FEASIBLE` status means optimal; always report the status clearly. +- Forgetting to scale the `max_time_in_seconds` parameter with problem size, potentially cutting off the search too early. + +# Workflow 2 (MIP with Big-M Disjunctive Constraints) + +## Modeling stage + +### Strategy Overview +This workflow formulates the problem as a Mixed-Integer Program (MIP), using binary sequencing variables and Big-M constraints to model disjunctive resources. It provides explicit control over the disjunctive logic and is suitable for solvers like Gurobi or CBC. + +### Step 1 - Define Problem Sets and Parameters +- Define sets for `jobs` and `machines`. +- Define parameter `processing_time[j][m]`. +- Compute a large constant `M` (Big-M), e.g., as the sum of all processing times. + +### Step 2 - Create Continuous Decision Variables +- Create continuous variable `start[j][m]` for the start time of job `j` on machine `m`. +- Create variable `makespan` to capture the completion time. + +### Step 3 - Enforce Processing and Precedence +- For each job `j` and machine `m`, enforce the processing relation implicitly through precedence: `start[j][m] + processing_time[j][m] <= start[j][m+1]` for `m < last_machine`. +- This ensures the operation on machine `m` finishes before the job starts on `m+1`. + +### Step 4 - Model Disjunctive Resources with Sequencing Variables +- For each machine `m` and for each pair of jobs `i, j` where `i < j`, create a binary variable `seq[i][j][m]`. +- If `seq[i][j][m] = 1`, job `i` must finish before job `j` starts on machine `m`. If `0`, job `j` must finish before job `i` starts. +- Add Big-M constraints: + - `start[j][m] >= start[i][m] + processing_time[i][m] - M * (1 - seq[i][j][m])` + - `start[i][m] >= start[j][m] + processing_time[j][m] - M * seq[i][j][m]` + +### Step 5 - Define Makespan and Objective +- For each job `j`, add constraint: `makespan >= start[j][last_machine] + processing_time[j][last_machine]`. +- Set the objective to minimize `makespan`. + +### Formulation Template +```json +{ + "sets": ["jobs", "machines"], + "parameters": ["processing_time[j][m]", "M (Big-M constant)"], + "decision_variables": ["start[j][m]", "seq[i][j][m] (binary, i= start[i][m] + processing_time[i][m] - M*(1 - seq[i][j][m]) for all m, i= start[j][m] + processing_time[j][m] - M*seq[i][j][m] for all m, i= start[j][last_machine] + processing_time[j][last_machine] for all j" + ] +} +``` + +### Common Pitfalls +- Setting `M` too small, which can cut off valid solutions. It must be larger than the latest possible start time. +- Creating sequencing variables for all job pairs `(i,j)` without enforcing `i 0`, compare total qualified/available employees against demand to confirm the penalty is structurally necessary. + +### Code Usage +```python +# build model from formulation +model = cp_model.CpModel() +# ... build variables, constraints, objective ... +solver = cp_model.CpSolver() +# apply parameter configuration +solver.parameters.max_time_in_seconds = 30 +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 +solver.parameters.relative_gap_limit = 0.0 + +# solve with status / termination checks +status = solver.Solve(model) +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + # extract solution + assignments = {(e,r,s): solver.Value(assign_var) for ...} + unfulfilled = {(r,s): solver.Value(unfulfilled_var) for ...} + total_cost = solver.ObjectiveValue() + # verify constraints and calculate cost breakdown +else: + print(f"Solver failed with status: {status}") +``` + +### Common Pitfalls +- Assuming `OPTIMAL` is the only acceptable status; `FEASIBLE` is also valid for satisficing. +- Not verifying the solution against original constraints, potentially missing solver errors. +- Misinterpreting floating-point objective values; round to nearest integer if necessary. + +# Workflow 2 (Pyomo with MIP Solver) + +## Modeling stage + +### Strategy Overview +Build a Pyomo `ConcreteModel` using `Var`, `Constraint`, and `Objective` components. Use `Param` objects for data to separate model logic from instance values. Structure constraints in a logical hierarchy for clarity. + +### Step 1 - Declare Sets and Parameters +- Define Pyomo `Set` objects for `employees`, `restaurants`, `shifts`. +- Define `Param` objects for `demand`, `preference_cost`, `availability`, `has_skill`, `penalty_cost`, initialized from data dictionaries. + +### Step 2 - Define Decision Variables +- Create `Var(..., domain=Binary)` for `model.x[employee, restaurant, shift]`. +- Create `Var(..., domain=NonNegativeIntegers)` for `model.u[restaurant, shift]`. + +### Step 3 - Build Constraint Rules +- Implement `demand_coverage` rule returning `sum(model.x[e,r,s] for e in employees) + model.u[r,s] == demand[r,s]`. +- Implement `availability` rule returning `model.x[e,r,s] <= availability[e,s]`. +- Implement `skill_requirement` rule returning `model.x[e,r,s] <= has_skill[e]`. +- Implement `one_assignment` rule returning `sum(model.x[e,r,s] for r in restaurants, s in shifts) <= 1`. + +### Step 4 - Define the Objective +- Create an `Objective` rule minimizing `sum(preference_cost[e,r,s] * model.x[e,r,s] for all e,r,s) + penalty_cost * sum(model.u[r,s] for all r,s)`. + +### Step 5 - Return Model and Data for Post-Processing +- Structure the model-building function to return the `model` object and the data dictionaries to avoid scope issues during solution analysis. + +### Formulation Template +```json +{ + "sets": ["employees", "restaurants", "shifts"], + "parameters": ["demand", "preference_cost", "availability", "has_skill", "penalty_cost"], + "decision_variables": ["x[employees, restaurants, shifts]", "u[restaurants, shifts]"], + "objective": { + "sense": "minimize", + "expression": "sum(preference_cost * x) + penalty_cost * sum(u)" + }, + "constraints": [ + "demand_coverage: sum(x[e, r, s]) + u[r, s] == demand[r, s] for all r, s", + "availability: x[e, r, s] <= availability[e, s] for all e, r, s", + "skill_requirement: x[e, r, s] <= has_skill[e] for all e, r, s", + "one_assignment_per_employee: sum(x[e, r, s]) <= 1 for all e" + ] +} +``` + +### Common Pitfalls +- Defining `Param` objects without initializing them, causing build errors. +- Using Python's global variables inside Pyomo rule functions, leading to unpredictable behavior. +- Not defining variable domains correctly (e.g., `NonNegativeIntegers` for slack variables). + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a MIP solver (e.g., Gurobi, CBC) with deterministic settings. Check termination condition and solution status rigorously, then extract and verify results. + +### Step 1 - Configure Solver for Deterministic Optimality +- Instantiate the solver (e.g., `SolverFactory('gurobi')`). +- Set options: `'TimeLimit'=30`, `'MIPGap'=0.0`, `'Threads'=4`, `'Seed'=42` to ensure reproducibility and a push for proven optimality. + +### Step 2 - Solve and Check Termination Status +- Call `results = solver.solve(model, tee=True)` to solve with log output. +- Check `results.solver.status` is `SolverStatus.ok`. +- Check `results.solver.termination_condition` is `optimal` or `feasible`. Handle other conditions as failures. + +### Step 3 - Extract Solution and Verify +- If solved successfully, load solution into model: `model.solutions.load_from(results)`. +- Extract variable values via `model.x[emp, rest, shift].value` and `model.u[rest, shift].value`. +- Programmatically verify all constraints are satisfied using the extracted values. +- Calculate total cost and its breakdown (preference vs. penalty) for reporting. + +### Step 4 - Perform Bottleneck Analysis +- If any `model.u[...].value > 0`, analyze the corresponding demand and qualified/available employee count to confirm infeasibility is structural. + +### Step 5 - Output Structured Results +- Print a summary table of assignments by shift and restaurant. +- Print employee assignment details including cost contribution. +- Print constraint checks to confirm model correctness. + +### Code Usage +```python +# build model from formulation +def build_model(data_dict): + model = ConcreteModel() + # ... define sets, params, variables, constraints, objective ... + return model, data_dict + +model, data = build_model(instance_data) +solver = SolverFactory('gurobi') +solver.options['TimeLimit'] = 30 +solver.options['MIPGap'] = 0.0 +solver.options['Threads'] = 4 +solver.options['Seed'] = 42 + +# solve with status / termination checks +results = solver.solve(model, tee=True) +if results.solver.status == SolverStatus.ok and results.solver.termination_condition in ['optimal', 'feasible']: + model.solutions.load_from(results) + # extract solution and verify + assignments = {(e,r,s): model.x[e,r,s].value for ...} + unfulfilled = {(r,s): model.u[r,s].value for ...} + # perform verification and analysis +else: + print(f"Solver failed: {results.solver.termination_condition}") +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition`, potentially accepting incomplete solutions. +- Forgetting to load the solution before accessing variable values, resulting in `None`. +- Using `tee=False` in final runs, missing solver logs that confirm optimality and zero MIP gap. diff --git a/skills/optskills/skill_library/generalized_bin_packing_container_minimization.md b/skills/optskills/skill_library/generalized_bin_packing_container_minimization.md new file mode 100644 index 0000000..2dd3d7a --- /dev/null +++ b/skills/optskills/skill_library/generalized_bin_packing_container_minimization.md @@ -0,0 +1,251 @@ +--- +name: Generalized Bin Packing / Container Minimization +description: | + Model and solve assignment problems where items must be assigned to containers with capacity limits, minimizing the number of containers used, using binary assignment and container usage variables. + +--- +# Workflow 1 (MILP with Pyomo and CBC/Gurobi) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) using Pyomo, suitable for open-source (CBC) or commercial (Gurobi) solvers. The model uses a clear separation of assignment and activation variables with capacity constraints deactivated for unused containers via a big-M formulation. + +### Step 1 - Define Sets and Parameters +- Define the set of items `I` and the set of containers `J` (generated as an upper bound, e.g., number of items). +- Define parameters for item weights `weight[i][d]` (for each resource dimension `d`) and container capacity `capacity[d]` per dimension. + +### Step 2 - Create Decision Variables +- Create binary assignment variables `x[i, j]` for each item `i` and container `j`. +- Create binary container usage variables `y[j]` for each container `j`. + +### Step 3 - Formulate Constraints +- **Assignment**: Each item must be assigned to exactly one container: `sum(x[i, j] for j in J) == 1` for all `i`. +- **Capacity**: For each container `j` and resource dimension `d`, total consumption cannot exceed capacity multiplied by the container's usage variable: `sum(weight[i][d] * x[i, j] for i in I) <= capacity[d] * y[j]`. +- **Linking**: If an item is assigned to a container, that container must be marked as used: `y[j] >= x[i, j]` for all `i, j`. This strengthens the formulation. + +### Step 4 - Define Objective +- Minimize the total number of containers used: `minimize sum(y[j] for j in J)`. + +### Formulation Template +```json +{ + "sets": [ + "I (items)", + "J (containers)" + ], + "parameters": [ + "weight[I][dimensions]", + "capacity[dimensions]" + ], + "decision_variables": [ + "x[I, J] ∈ {0, 1}", + "y[J] ∈ {0, 1}" + ], + "objective": { + "sense": "min", + "expression": "sum(y[j] for j in J)" + }, + "constraints": [ + "assignment: sum(x[i, j] for j in J) == 1, for all i in I", + "capacity: sum(weight[i][d] * x[i, j] for i in I) <= capacity[d] * y[j], for all j in J, for all dimensions d", + "linking: y[j] >= x[i, j], for all i in I, j in J" + ] +} +``` + +### Common Pitfalls +- Ensure the number of containers in set `J` is at least the trivial lower bound: `ceil(total weight / capacity)` for a single dimension, or the maximum over dimensions for multi-dimensional. +- For multi-dimensional capacity, add a separate capacity constraint for each dimension, each multiplied by `y[j]`. +- Omitting explicit linking constraints can lead to fractional solutions where `y[j]` is set to a small fraction instead of 1. +- Avoid using Pyomo reserved attribute names (e.g., `items`). Use distinct names like `I` for items and `J` for containers. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a configured MILP solver, extract the solution, and perform validation checks to ensure feasibility and assess solution quality. + +### Step 1 - Instantiate Solver and Set Parameters +- Instantiate the solver factory (e.g., `'cbc'` or `'gurobi'`). +- Configure key parameters: time limit (`[TIME_LIMIT]`), optimality gap tolerance (set `ratio` to `0.0` for exact optimum), number of threads, and random seed for reproducibility. + +### Step 2 - Solve and Check Status +- Execute the solve command. +- Check the solver termination condition. Accept solutions marked as `optimal` or `feasible`. + +### Step 3 - Extract and Structure Solution +- Retrieve the objective value. +- Identify used containers: `[j for j in model.J if pyo.value(model.y[j]) > 0.5]`. +- For each used container, list assigned items and calculate total load per resource dimension. +- Package results into a structured dictionary (e.g., JSON) for downstream use. + +### Step 4 - Validate Solution and Assess Quality +- **Feasibility Verification**: + - Verify each item is assigned exactly once. + - Verify the load in each used container does not exceed capacity for each resource dimension. + - Verify the objective value equals the count of used containers. +- **Optimality Assessment**: + - Compute a theoretical lower bound: for a single dimension, use `ceil(total weight / capacity)`. For multiple dimensions, compute the bound per dimension and take the maximum: `max(ceil(total_dim1 / capacity_dim1), ceil(total_dim2 / capacity_dim2), ...)`. + - Compare the objective to this bound. An optimal solution meets this bound. + +### Code Usage +```python +import pyomo.environ as pyo + +# Build model from formulation (model defined as per Modeling Stage) +model = pyo.ConcreteModel() +# ... (model construction code) + +# Solve with status / termination checks +solver = pyo.SolverFactory('cbc') # or 'gurobi' +solver.options['seconds'] = [TIME_LIMIT] +solver.options['ratio'] = 0.0 +results = solver.solve(model, tee=False) + +# Check status +status = results.solver.termination_condition +if status in [pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible]: + # Extract solution + used_containers = [j for j in model.J if pyo.value(model.y[j]) > 0.5] + assignments = {} + for j in used_containers: + items_in_j = [i for i in model.I if pyo.value(model.x[i, j]) > 0.5] + assignments[j] = items_in_j + # ... (proceed with validation and output) +else: + raise Exception(f"Solver did not find a feasible solution. Status: {status}") +``` + +### Common Pitfalls +- Not checking solver status can lead to using invalid solutions. +- Using a loose optimality gap (`ratio` > 0) may return suboptimal solutions when an exact optimum is required. +- Forgetting to set a time limit for large instances can cause excessively long runtimes. + +# Workflow 2 (CP-SAT with OR-Tools) + +## Modeling stage + +### Strategy Overview +Use Google's OR-Tools CP-SAT solver, a constraint programming solver optimized for Boolean and integer variables. Model the problem with binary variables and linear constraints, leveraging efficient search for combinatorial assignments. + +### Step 1 - Define Data Structures +- Define lists for items `I` and containers `J`. +- Define a list or dictionary for item weights `weight[i][d]` (for each resource dimension `d`). + +### Step 2 - Create CP-SAT Variables +- Create a Boolean variable `assign[i, c]` for each item-container assignment pair. +- Create a Boolean variable `used[c]` for each container to indicate usage. + +### Step 3 - Add Constraints to the Model +- **Assignment**: For each item, exactly one container must be selected: `sum(assign[i, c] for c in J) == 1`. +- **Capacity**: For each container `c` and each resource dimension `d`, total weight of assigned items must be less than or equal to capacity: `sum(weight[i][d] * assign[i, c] for i in I) <= capacity[d]`. +- **Linking**: Container usage variable must be greater than or equal to each assignment variable for that container: `used[c] >= assign[i, c]` for all `i, c`. This is crucial for the objective to work properly. + +### Step 4 - Define Objective +- Minimize the sum of container usage variables: `minimize sum(used[c] for c in J)`. + +### Formulation Template +```json +{ + "sets": [ + "I (items)", + "J (containers)" + ], + "parameters": [ + "weight[I][dimensions]", + "capacity[dimensions]" + ], + "decision_variables": [ + "assign[I, J] ∈ Bool", + "used[J] ∈ Bool" + ], + "objective": { + "sense": "min", + "expression": "sum(used[c] for c in J)" + }, + "constraints": [ + "assignment: sum(assign[i, c] for c in J) == 1, for all i in I", + "capacity: sum(weight[i][d] * assign[i, c] for i in I) <= capacity[d], for all c in J, for all dimensions d", + "linking: used[c] >= assign[i, c], for all i in I, c in J" + ] +} +``` + +### Common Pitfalls +- Creating an excessively large number of containers (the upper bound) can increase model size unnecessarily; balance between feasibility and performance. +- Not using `used[c] >= assign[i, c]` and relying solely on the capacity constraint may be insufficient for the CP-SAT solver's propagation. + +## Solving stage + +### Strategy Overview +Configure and run the CP-SAT solver, extract the Boolean variable values, and assemble the assignment solution. + +### Step 1 - Configure Solver Parameters +- Instantiate `CpSolver()`. +- Set parameters: maximum time allowed (`[TIME_LIMIT]`), number of parallel search workers, random seed, and optionally an absolute or relative optimality gap. + +### Step 2 - Solve and Interpret Status +- Execute the solver on the model. +- Check the status (`OPTIMAL`, `FEASIBLE`, `INFEASIBLE`, etc.). Proceed if status indicates a solution was found. + +### Step 3 - Extract Solution Values +- Use `solver.Value(variable)` to get the value (0 or 1) of each Boolean variable. +- Build a mapping of used containers and the items assigned to them by iterating through the assignment variables. + +### Step 4 - Validate and Report +- **Feasibility Verification**: + - Verify each item is assigned exactly once. + - Verify the load in each used container does not exceed capacity for each resource dimension. + - Verify the objective value equals the count of used containers. +- **Optimality Assessment**: + - Compute a theoretical lower bound as `ceil(total weight / capacity)` for a single dimension, or the maximum over dimensions for multi-dimensional. + - Compare the objective to this bound; if equal, the solution is proven optimal. +- Output the solution in a structured format. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# Build model from formulation +model = cp_model.CpModel() +assign = {} +for i in I: + for c in J: + assign[(i, c)] = model.NewBoolVar(f'assign_{i}_{c}') +used = [model.NewBoolVar(f'used_{c}') for c in J] + +# Constraints +for i in I: + model.Add(sum(assign[(i, c)] for c in J) == 1) +for c in J: + for d in dimensions: + model.Add(sum(weight[i][d] * assign[(i, c)] for i in I) <= capacity[d]) + for i in I: + model.Add(used[c] >= assign[(i, c)]) + +# Objective +model.Minimize(sum(used)) + +# Solve with status / termination checks +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 +status = solver.Solve(model) + +if status in [cp_model.OPTIMAL, cp_model.FEASIBLE]: + # Extract solution + used_containers = [c for c in J if solver.Value(used[c]) == 1] + assignments = {} + for c in used_containers: + items_in_c = [i for i in I if solver.Value(assign[(i, c)]) == 1] + assignments[c] = items_in_c + # ... (proceed with validation and output) +else: + raise Exception("Solver did not find a feasible solution.") +``` + +### Common Pitfalls +- Not setting `max_time_in_seconds` can lead to indefinite runs on difficult instances. +- Interpreting the status incorrectly; `FEASIBLE` is acceptable when a time limit is set, but `OPTIMAL` is not guaranteed. +- Forgetting that `solver.Value()` returns an integer (0 or 1) for Boolean variables. diff --git a/skills/optskills/skill_library/generalizedassignmentsolver.md b/skills/optskills/skill_library/generalizedassignmentsolver.md new file mode 100644 index 0000000..ae74cea --- /dev/null +++ b/skills/optskills/skill_library/generalizedassignmentsolver.md @@ -0,0 +1,190 @@ +--- +name: GeneralizedAssignmentSolver +description: | + Solves assignment problems with capacity constraints by formulating as a Mixed-Integer Linear Program (MILP) with binary decision variables, assignment and knapsack constraints, and a linear objective, then solving with a configured MILP solver. + +--- +# Workflow 1 (Pyomo-HiGHS) + +## Modeling stage + +### Strategy Overview +Model the problem as a Generalized Assignment Problem (GAP) using Pyomo's abstract modeling capabilities. Define sets for agents and tasks, binary decision variables for assignment, and linear constraints for assignment and capacity. The objective is to minimize total assignment cost. + +### Step 1 - Problem Recognition and Set Definition +- Recognize the core structure as assigning tasks to agents subject to agent capacity limits. +- Define Pyomo Sets: `model.A` for agents (e.g., crews, machines) and `model.T` for tasks (e.g., zones, jobs). + +### Step 2 - Parameter Declaration +- Declare parameters using `pyo.Param(model.A, model.T, within=pyo.NonNegativeReals)` for cost and resource consumption matrices (e.g., `model.cost`, `model.resource_use`). +- Declare agent capacity parameters using `pyo.Param(model.A, within=pyo.NonNegativeReals)`. + +### Step 3 - Decision Variable Definition +- Define binary decision variables `model.x = pyo.Var(model.A, model.T, within=pyo.Binary)` where `model.x[a, t] = 1` indicates task `t` is assigned to agent `a`. + +### Step 4 - Objective Function Formulation +- Formulate a linear objective to minimize total cost: `sum(model.cost[a, t] * model.x[a, t] for a in model.A for t in model.T)`. + +### Step 5 - Constraint Formulation +- Add assignment constraints: For each task `t` in `model.T`, enforce `sum(model.x[a, t] for a in model.A) == 1`. +- Add capacity constraints: For each agent `a` in `model.A`, enforce `sum(model.resource_use[a, t] * model.x[a, t] for t in model.T) <= model.capacity[a]`. + +### Formulation Template +```json +{ + "sets": ["A", "T"], + "parameters": ["cost[A,T]", "resource_use[A,T]", "capacity[A]"], + "decision_variables": ["x[A,T]"], + "objective": { + "sense": "min", + "expression": "sum(cost[a,t] * x[a,t] for a in A for t in T)" + }, + "constraints": [ + "sum(x[a,t] for a in A) == 1 for all t in T", + "sum(resource_use[a,t] * x[a,t] for t in T) <= capacity[a] for all a in A" + ] +} +``` + +### Common Pitfalls +- Forgetting to initialize all required parameters before model instantiation, leading to runtime errors. +- Incorrectly indexing parameters within constraint rules, causing KeyErrors. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the HiGHS MILP solver via the `solverfactory`. Configure solver options for performance and reliability, then check the solution status and termination condition before extracting results. + +### Step 1 - Solver Selection and Configuration +- Select solver: `solver = SolverFactory('highs')`. +- Configure options: Set `solver.options['time_limit'] = [TIME_LIMIT]`, `solver.options['threads'] = [NUM_THREADS]`, and `solver.options['mip_rel_gap'] = 0.0` for optimality. + +### Step 2 - Model Solving and Status Check +- Execute `results = solver.solve(model, tee=False)`. +- Check solution status: `if results.solver.status == SolverStatus.ok`. +- Check termination condition: `if results.solver.termination_condition == TerminationCondition.optimal`. + +### Step 3 - Solution Extraction and Validation +- Extract objective value: `pyo.value(model.obj)`. +- Extract assignments by iterating over `model.x` and checking `pyo.value(model.x[a, t]) > 0.5`. +- Compute realized resource usage per agent to verify capacity constraints are satisfied. + +### Step 4 - Results Reporting +- Print the optimal cost and assignment mapping. +- Optionally, print agent utilization (total resource used / capacity). + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# build model from formulation +model = pyo.ConcreteModel() +# ... define sets, params, variables, objective, constraints + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = [TIME_LIMIT] +results = solver.solve(model, tee=False) + +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition == TerminationCondition.optimal): + # Process solution + print(f'Optimal cost: {pyo.value(model.obj)}') + # ... extract assignments +else: + print('Solver did not find optimal solution.') +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition`, potentially processing infeasible or suboptimal results. +- Misinterpreting variable values due to solver tolerances; always use a tolerance (e.g., > 0.5) when checking binary variables. + +# Workflow 2 (OR-Tools CP-SAT) + +## Modeling stage + +### Strategy Overview +Model the problem using Google's OR-Tools CP-SAT solver. Define linear expressions for the objective and constraints using the solver's native interface. This approach is suitable for medium-sized problems and provides good performance with logical constraints. + +### Step 1 - Solver and Model Initialization +- Create a CP-SAT model: `model = cp_model.CpModel()`. + +### Step 2 - Variable Creation +- Create binary decision variables using `model.NewBoolVar(name)` for each agent-task pair. Store them in a dictionary `x[(a, t)]`. + +### Step 3 - Objective Function Definition +- Create a linear objective expression: `objective = sum(cost[a, t] * x[(a, t)] for all pairs)`. +- Set the model to minimize this objective: `model.Minimize(objective)`. + +### Step 4 - Constraint Addition +- Add assignment constraints: For each task `t`, enforce `sum(x[(a, t)] for all a) == 1`. +- Add capacity constraints: For each agent `a`, enforce `sum(resource_use[a, t] * x[(a, t)] for all t) <= capacity[a]`. + +### Formulation Template +```json +{ + "sets": ["A", "T"], + "parameters": ["cost[A,T]", "resource_use[A,T]", "capacity[A]"], + "decision_variables": ["x[A,T]"], + "objective": { + "sense": "min", + "expression": "sum(cost[a,t] * x[a,t] for a in A for t in T)" + }, + "constraints": [ + "sum(x[a,t] for a in A) == 1 for all t in T", + "sum(resource_use[a,t] * x[a,t] for t in T) <= capacity[a] for all a in A" + ] +} +``` + +### Common Pitfalls +- Exceeding the CP-SAT solver's limit on variable or constraint count for very large instances. +- Incorrectly building linear expressions by mixing Python types; ensure all terms are OR-Tools linear expressions. + +## Solving stage + +### Strategy Overview +Solve the CP-SAT model with configured time and optional solution callback. Check the solver's status and extract the variable assignments if an optimal or feasible solution is found. + +### Step 1 - Solver Configuration and Solving +- Create a solver instance: `solver = cp_model.CpSolver()`. +- Set solver parameters: `solver.parameters.max_time_in_seconds = [TIME_LIMIT]`, `solver.parameters.num_search_workers = [NUM_THREADS]`. +- Execute the solve: `status = solver.Solve(model)`. + +### Step 2 - Status Verification +- Check the solve status: `if status in (cp_model.OPTIMAL, cp_model.FEASIBLE)`. + +### Step 3 - Solution Extraction +- If status is acceptable, extract assignments by evaluating `solver.Value(x[(a, t)]) == 1`. +- Compute the objective value from the solver: `objective_value = solver.ObjectiveValue()`. +- Compute realized resource usage per agent for validation. + +### Step 4 - Results Reporting +- Print the status, objective value, and assignment list. +- Optionally, print a summary of agent utilization. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# build model from formulation +model = cp_model.CpModel() +# ... create variables, objective, constraints + +# solve with status / termination checks +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = [NUM_THREADS] +status = solver.Solve(model) + +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + print(f'Objective value: {solver.ObjectiveValue()}') + # ... extract assignments by checking solver.Value(x_var) +else: + print('Solver did not find a solution.') +``` + +### Common Pitfalls +- Confusing `cp_model.OPTIMAL` with `cp_model.FEASIBLE`; the former guarantees optimality, the latter only feasibility. +- Not setting `num_search_workers` for parallel search, potentially missing performance gains on multi-core machines. diff --git a/skills/optskills/skill_library/generator_commitment_and_dispatch.md b/skills/optskills/skill_library/generator_commitment_and_dispatch.md new file mode 100644 index 0000000..71921f9 --- /dev/null +++ b/skills/optskills/skill_library/generator_commitment_and_dispatch.md @@ -0,0 +1,272 @@ +--- +name: Generator Commitment and Dispatch +description: | + Model and solve unit commitment problems with startup logic, capacity constraints, and cost minimization using mixed-integer linear programming. + +--- + +# Workflow 1 (Pyomo with Gurobi) + +## Modeling stage + +### Strategy Overview +This workflow uses Pyomo's concrete model paradigm with explicit index sets and parameter dictionaries. It employs integer variables for discrete activation decisions and continuous variables for power output, linked via linear constraints to enforce operational limits and startup logic. + +### Step 1 - Define Sets and Parameters +- Declare index sets for generators and time periods. +- Organize all cost coefficients and operational limits into parameter dictionaries (e.g., `base_cost`, `variable_cost`, `startup_cost`, `max_active`, `min_output`, `max_output`, `max_startups`). +- Define time-series parameters for system demand and reserve requirements. + +### Step 2 - Design Decision Variables +- Create integer variables `num_active[g,t]` for the count of active units per generator type and period. +- Create continuous variables `power_output[g,t]` for the total power output per generator type and period. +- Create integer variables `num_startup[g,t]` for the count of units started per generator type and period. + +### Step 3 - Formulate Objective Function +- Construct a minimizing objective summing three cost components across all generators and periods: base cost (`base_cost[g] * num_active[g,t]`), variable cost (`variable_cost[g] * power_output[g,t]`), and startup cost (`startup_cost[g] * num_startup[g,t]`). + +### Step 4 - Implement Core Operational Constraints +- **Demand Satisfaction**: Sum of `power_output` across all generators for each period must meet or exceed the period's demand. +- **Reserve Requirement**: Sum of maximum possible output (`max_output[g] * num_active[g,t]`) across all generators for each period must meet or exceed the demand plus a reserve threshold. +- **Output Bounds**: For each generator and period, enforce `min_output[g] * num_active[g,t] <= power_output[g,t] <= max_output[g] * num_active[g,t]`. +- **Capacity Limit**: For each generator and period, enforce `num_active[g,t] <= max_active[g]`. + +### Step 5 - Enforce Startup Logic +- For the initial period, enforce `num_startup[g,0] <= num_active[g,0]`. +- For subsequent periods, enforce `num_startup[g,t] <= num_active[g,t] - num_active[g,t-1]` and `num_startup[g,t] >= 0`. +- For generators where startups are prohibited, add equality constraint `num_startup[g,t] == 0`. + +### Formulation Template +```json +{ + "sets": [ + "generators", + "time_periods" + ], + "parameters": [ + "base_cost[g]", + "variable_cost[g]", + "startup_cost[g]", + "max_active[g]", + "min_output[g]", + "max_output[g]", + "max_startups[g]", + "demand[t]", + "reserve_requirement[t]" + ], + "decision_variables": [ + "num_active[g,t] (integer)", + "power_output[g,t] (continuous)", + "num_startup[g,t] (integer)" + ], + "objective": { + "sense": "min", + "expression": "sum_{g,t} base_cost[g]*num_active[g,t] + variable_cost[g]*power_output[g,t] + startup_cost[g]*num_startup[g,t]" + }, + "constraints": [ + "demand_satisfaction[t]: sum_g power_output[g,t] >= demand[t]", + "reserve_requirement[t]: sum_g max_output[g]*num_active[g,t] >= demand[t] + reserve_requirement[t]", + "output_lower_bound[g,t]: power_output[g,t] >= min_output[g]*num_active[g,t]", + "output_upper_bound[g,t]: power_output[g,t] <= max_output[g]*num_active[g,t]", + "capacity_limit[g,t]: num_active[g,t] <= max_active[g]", + "startup_initial[g]: num_startup[g,0] <= num_active[g,0]", + "startup_subsequent[g,t]: num_startup[g,t] <= num_active[g,t] - num_active[g,t-1]", + "startup_nonneg[g,t]: num_startup[g,t] >= 0", + "no_startup_if_prohibited[g,t]: num_startup[g,t] == 0 (if max_startups[g]==0)" + ] +} +``` + +### Common Pitfalls +- Using Pyomo reserved keywords (e.g., `active`) as variable or parameter names, causing syntax errors. +- Omitting the non-negativity constraint on the difference in the startup logic, which can lead to incorrect model behavior. +- Failing to handle edge cases like generators with zero allowed startups or initial conditions. + +## Solving stage + +### Strategy Overview +This stage configures the Gurobi solver via Pyomo's `SolverFactory`, sets performance and reproducibility options, rigorously checks the solution status, extracts and verifies results, and provides structured output for both success and failure cases. + +### Step 1 - Configure Solver and Solve +- Instantiate the solver using `SolverFactory("gurobi")`. +- Set solver options such as time limit, MIP gap tolerance, thread count, and random seed for reproducibility. +- Call the solver's `solve` method on the model instance. + +### Step 2 - Validate Solution Status +- Check that `solver.status == SolverStatus.ok`. +- Check that the model's termination condition is either `TerminationCondition.optimal` or `TerminationCondition.feasible`. +- If checks fail, proceed to error handling without extracting variable values. + +### Step 3 - Extract and Verify Solution +- Extract the objective value using `pyo.value(model.obj)`. +- Access variable values via their `.value` attribute. +- Optionally, implement post-solution verification logic to confirm key constraints (demand, reserve) are satisfied within a small tolerance. + +### Step 4 - Format and Output Results +- For a successful solve, print the objective value in a parseable format (e.g., `RESULT:{objective_value}`). +- Output a detailed breakdown of the solution (active counts, outputs, startups per period) and cost decomposition for debugging. +- For infeasible or error cases, output a structured JSON payload containing the status, reason, solver status, and termination condition. + +### Code Usage +```python +import pyomo.environ as pyo + +# build model from formulation +model = pyo.ConcreteModel() +# ... (model construction code as per Modeling Stage) + +# solve with status / termination checks +solver = pyo.SolverFactory('gurobi') +solver.options['TimeLimit'] = 30 +solver.options['MIPGap'] = 0.0001 +solver.options['Threads'] = 4 +solver.options['Seed'] = 42 + +results = solver.solve(model) + +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in [pyo.TerminationCondition.optimal, + pyo.TerminationCondition.feasible]): + objective_value = pyo.value(model.obj) + print(f"RESULT:{objective_value}") + # ... extract and print detailed solution +else: + error_payload = { + "status": "error", + "reason": "Solver did not return an optimal or feasible solution.", + "solver_status": str(results.solver.status), + "termination_condition": str(results.solver.termination_condition) + } + print(json.dumps(error_payload)) +``` + +### Common Pitfalls +- Trusting a non-zero return code or an `unknown` termination condition as a valid solution. +- Attempting to access `.value` attributes on variables before confirming a successful solve, which may raise exceptions. +- Outputting pseudo-numeric answers or partial results when the solver execution has failed. + +# Workflow 2 (PuLP with CBC) + +## Modeling stage + +### Strategy Overview +This workflow uses PuLP's Pythonic API for rapid prototyping of MILP models. It employs binary variables for individual unit commitment decisions, aggregating them to model counts, and uses list comprehensions for concise constraint definition. + +### Step 1 - Define Problem and Data Structures +- Instantiate a `pulp.LpProblem` with a name and sense (`LpMinimize`). +- Store cost coefficients and operational limits in Python dictionaries or lists indexed by generator and period. +- Define time-series data for demand and reserve as lists. + +### Step 2 - Design Decision Variables +- Create binary variables `is_active[g,u,t]` for each individual unit `u` of generator type `g` in each period `t`. +- Create continuous variables `unit_output[g,u,t]` for the power output of each individual unit. +- Create binary variables `unit_startup[g,u,t]` for the startup decision of each individual unit. + +### Step 3 - Formulate Objective Function +- Construct the objective by summing: base cost per active unit, variable cost per unit output, and startup cost per unit started, across all units, generators, and periods. + +### Step 4 - Implement Aggregated Operational Constraints +- **Demand Satisfaction**: Sum of all `unit_output` variables for each period must meet or exceed demand. +- **Reserve Requirement**: Sum of `max_output[g] * is_active[g,u,t]` across all units for each period must meet demand plus reserve. +- **Output Bounds**: For each unit, enforce `min_output[g] * is_active[g,u,t] <= unit_output[g,u,t] <= max_output[g] * is_active[g,u,t]`. +- **Unit Count Limit**: For each generator type and period, the sum of `is_active` variables for its units must be less than or equal to `max_active[g]`. + +### Step 5 - Enforce Per-Unit Startup Logic +- For the initial period, enforce `unit_startup[g,u,0] <= is_active[g,u,0]`. +- For subsequent periods, enforce `unit_startup[g,u,t] <= is_active[g,u,t] - is_active[g,u,t-1]`. +- Add non-negativity for startup variables. + +### Formulation Template +```json +{ + "sets": [ + "generators", + "units[g]", + "time_periods" + ], + "parameters": [ + "base_cost[g]", + "variable_cost[g]", + "startup_cost[g]", + "max_active[g]", + "min_output[g]", + "max_output[g]", + "demand[t]", + "reserve_requirement[t]" + ], + "decision_variables": [ + "is_active[g,u,t] (binary)", + "unit_output[g,u,t] (continuous)", + "unit_startup[g,u,t] (binary)" + ], + "objective": { + "sense": "min", + "expression": "sum_{g,u,t} base_cost[g]*is_active[g,u,t] + variable_cost[g]*unit_output[g,u,t] + startup_cost[g]*unit_startup[g,u,t]" + }, + "constraints": [ + "demand_satisfaction[t]: sum_{g,u} unit_output[g,u,t] >= demand[t]", + "reserve_requirement[t]: sum_{g,u} max_output[g]*is_active[g,u,t] >= demand[t] + reserve_requirement[t]", + "output_lower_bound[g,u,t]: unit_output[g,u,t] >= min_output[g]*is_active[g,u,t]", + "output_upper_bound[g,u,t]: unit_output[g,u,t] <= max_output[g]*is_active[g,u,t]", + "capacity_limit[g,t]: sum_{u in units[g]} is_active[g,u,t] <= max_active[g]", + "startup_initial[g,u]: unit_startup[g,u,0] <= is_active[g,u,0]", + "startup_subsequent[g,u,t]: unit_startup[g,u,t] <= is_active[g,u,t] - is_active[g,u,t-1]", + "startup_nonneg[g,u,t]: unit_startup[g,u,t] >= 0" + ] +} +``` + +### Common Pitfalls +- Creating an excessively large number of binary variables if unit counts are very high, impacting solve time. +- Incorrectly aggregating unit-level variables in constraints (e.g., confusing generator-level and unit-level indices). +- Forgetting to enforce the binary nature of startup variables in the per-unit formulation. + +## Solving stage + +### Strategy Overview +This stage uses PuLP's default CBC solver or an installed alternative. It focuses on extracting solution information directly from PuLP's variable objects, providing basic status checks, and generating a clear solution summary. + +### Step 1 - Solve the Problem +- Call the `solve()` method on the `pulp.LpProblem` instance. PuLP will use the default solver (CBC) unless specified otherwise. +- Optionally, pass solver-specific options via the `solve` method if using a different solver like GLPK. + +### Step 2 - Check Solution Status +- Check the problem's status attribute (`LpStatus`) for `'Optimal'` or `'Feasible'`. +- Do not proceed with value extraction if the status is `'Infeasible'`, `'Unbounded'`, or `'Not Solved'`. + +### Step 3 - Extract Solution Values +- Access the objective value via `pulp.value(problem.objective)`. +- Iterate through the problem's variables and retrieve their `varValue` attribute. +- Aggregate unit-level results to generator-level totals for reporting. + +### Step 4 - Generate Output +- Print the objective value in a consistent format. +- Output a summary table showing aggregated active units, total output, and startup counts per generator type and period. +- For unsuccessful solves, print the problem status and avoid outputting numerical results. + +### Code Usage +```python +import pulp + +# build model from formulation +prob = pulp.LpProblem('Generator_Commitment', pulp.LpMinimize) +# ... (model construction code as per Modeling Stage) + +# solve with status / termination checks +solver = pulp.PULP_CBC_CMD(msg=False, timeLimit=30) # Optional: use and configure CBC +prob.solve(solver) + +status = pulp.LpStatus[prob.status] +if status in ['Optimal', 'Feasible']: + objective_value = pulp.value(prob.objective) + print(f"RESULT:{objective_value}") + # ... iterate through variables, aggregate, and print summary +else: + print(f"STATUS:{status}") + # Do not output variable values +``` + +### Common Pitfalls +- Assuming the solve was successful without checking `LpStatus`, leading to errors when accessing `varValue`. +- Outputting variable values from an infeasible model, which may be `None` or misleading. +- Neglecting to set a time limit or other solver options, potentially resulting in very long run times for large instances. diff --git a/skills/optskills/skill_library/index.json b/skills/optskills/skill_library/index.json new file mode 100644 index 0000000..912d818 --- /dev/null +++ b/skills/optskills/skill_library/index.json @@ -0,0 +1,623 @@ +{ + "version": "1.0", + "skills": [ + { + "skill_id": "maximum_cardinality_bipartite_matching", + "name": "Maximum Cardinality Bipartite Matching", + "description": "Model and solve one-to-one assignment problems between two disjoint sets with preference compatibility, aiming to maximize the total number of matches.", + "path": "maximum_cardinality_bipartite_matching.md" + }, + { + "skill_id": "binary_network_flow_routing", + "name": "Binary Network Flow Routing", + "description": "Model and solve network flow problems with binary arc selection variables, enforcing flow conservation and minimizing linear costs, using either a direct solver API or a modeling framework.", + "path": "binary_network_flow_routing.md" + }, + { + "skill_id": "multi_resource_assignment_optimization", + "name": "Multi-Resource Assignment Optimization", + "description": "Model and solve linear cost-minimization problems where discrete or continuous resources with capacity contributions must be assigned to tasks to meet demands, subject to supply limits.", + "path": "multi_resource_assignment_optimization.md" + }, + { + "skill_id": "quadratic_portfolio_optimization", + "name": "Quadratic Portfolio Optimization", + "description": "Formulate and solve portfolio variance minimization as a quadratic program with linear constraints, handling incomplete covariance data and verifying solution feasibility.", + "path": "quadratic_portfolio_optimization.md" + }, + { + "skill_id": "multiindexflowassignmentlp", + "name": "MultiIndexFlowAssignmentLP", + "description": "Model and solve multi-index flow assignment problems with linear profit maximization and exact demand satisfaction using continuous variables, implemented via either direct solver APIs or algebraic modeling frameworks.", + "path": "multiindexflowassignmentlp.md" + }, + { + "skill_id": "multi_index_flow_allocation", + "name": "Multi-Index Flow Allocation", + "description": "Model and solve linear profit maximization problems for allocating flows across multiple origins, types, and destinations with exact demand satisfaction.", + "path": "multi_index_flow_allocation.md" + }, + { + "skill_id": "multi_commodity_flow_allocation", + "name": "Multi-Commodity Flow Allocation", + "description": "Model and solve multi-source, multi-sink, multi-commodity flow problems with linear profit objectives and exact demand satisfaction using linear programming.", + "path": "multi_commodity_flow_allocation.md" + }, + { + "skill_id": "weighted_set_cover_solver", + "name": "Weighted Set Cover Solver", + "description": "Model and solve weighted set covering problems with binary selection variables, coverage constraints, and fixed-cost minimization using either CP-SAT or MILP frameworks.", + "path": "weighted_set_cover_solver.md" + }, + { + "skill_id": "cardinality_constrained_pairwise_selection", + "name": "Cardinality-Constrained Pairwise Selection", + "description": "Model and solve combinatorial problems where exactly K elements are selected from a set, and the objective maximizes the sum of pairwise benefits between selected elements, using linearized binary variables for pairwise activation.", + "path": "cardinality_constrained_pairwise_selection.md" + }, + { + "skill_id": "weighted_set_cover_optimization", + "name": "Weighted Set Cover Optimization", + "description": "Model binary selection problems with coverage requirements as weighted set cover MILP, then solve with open-source solvers via Pyomo or OR-Tools.", + "path": "weighted_set_cover_optimization.md" + }, + { + "skill_id": "multi_commodity_resource_allocation_with_shared_capacity", + "name": "Multi-Commodity Resource Allocation with Shared Capacity", + "description": "Model and solve integer or linear programs for allocating multiple products across shared, capacity-constrained resources to maximize linear profit, using explicit demand limits and resource usage matrices.", + "path": "multi_commodity_resource_allocation_with_shared_capacity.md" + }, + { + "skill_id": "fixed_charge_production_planning", + "name": "Fixed-Charge Production Planning", + "description": "Model and solve production planning problems with fixed activation costs and variable production costs using mixed-integer linear programming (MILP) with big-M constraints.", + "path": "fixed_charge_production_planning.md" + }, + { + "skill_id": "minimum_cost_network_flow", + "name": "Minimum Cost Network Flow", + "description": "Model and solve capacitated network flow problems with supply/demand nodes and linear transportation costs using structured data and solver-aware implementations.", + "path": "minimum_cost_network_flow.md" + }, + { + "skill_id": "transportationproblemsolver", + "name": "TransportationProblemSolver", + "description": "Model and solve balanced or capacitated transportation problems with linear costs using either direct LP solver APIs or algebraic modeling frameworks.", + "path": "transportationproblemsolver.md" + }, + { + "skill_id": "binary_selection_with_knapsack_constraint", + "name": "Binary Selection with Knapsack Constraint", + "description": "Model and solve binary selection problems with a single capacity constraint using either a dedicated knapsack solver or a general-purpose MILP solver, ensuring robust solution extraction and verification.", + "path": "binary_selection_with_knapsack_constraint.md" + }, + { + "skill_id": "supplier_selection_with_minimum_quantity_and_count", + "name": "Supplier Selection with Minimum Quantity and Count", + "description": "Model and solve assignment problems with minimum quantity requirements if selected and minimum supplier count constraints using mixed-integer linear programming.", + "path": "supplier_selection_with_minimum_quantity_and_count.md" + }, + { + "skill_id": "set_cover_cost_minimization", + "name": "Set Cover Cost Minimization", + "description": "Model and solve binary set cover problems with linear costs using either a direct matrix formulation or a sparse coverage mapping, and implement robust solving with verification.", + "path": "set_cover_cost_minimization.md" + }, + { + "skill_id": "tsp_mtz_mip", + "name": "TSP-MTZ-MIP", + "description": "Model and solve the Traveling Salesperson Problem (TSP) as a Mixed-Integer Program (MIP) using binary arc selection and Miller-Tucker-Zemlin (MTZ) subtour elimination constraints, with workflows for both commercial and open-source solver backends.", + "path": "tsp_mtz_mip.md" + }, + { + "skill_id": "integer_linear_programming_with_multiple_knapsack_constraints", + "name": "Integer Linear Programming with Multiple Knapsack Constraints", + "description": "Model and solve integer linear programs with bounded, nonnegative integer variables, a linear objective to maximize profit, and multiple linear inequality (knapsack) constraints using structured formulations and robust solver backends.", + "path": "integer_linear_programming_with_multiple_knapsack_constraints.md" + }, + { + "skill_id": "bipartiteassignmentflow", + "name": "BipartiteAssignmentFlow", + "description": "Model and solve linear bipartite assignment problems with flow quantities, demand satisfaction, and unconstrained supply to maximize total profit or minimize total cost.", + "path": "bipartiteassignmentflow.md" + }, + { + "skill_id": "multicommodityflowoptimization", + "name": "MultiCommodityFlowOptimization", + "description": "Model multi-commodity flow problems with shared arc capacities and solve them using linear programming, with robust status checking and solution validation.", + "path": "multicommodityflowoptimization.md" + }, + { + "skill_id": "schedulewithprecedenceanddeviations", + "name": "ScheduleWithPrecedenceAndDeviations", + "description": "Model and solve scheduling problems with precedence decisions, time windows, and piecewise-linear deviation penalties using MILP formulations and modern solvers.", + "path": "schedulewithprecedenceanddeviations.md" + }, + { + "skill_id": "binary_arc_selection_with_flow_conservation", + "name": "Binary Arc Selection with Flow Conservation", + "description": "Model and solve network optimization problems where arcs are selected via binary variables and flow conservation enforces connectivity from source to sink, minimizing total cost.", + "path": "binary_arc_selection_with_flow_conservation.md" + }, + { + "skill_id": "weighted_set_cover_optimization_2", + "name": "Weighted Set Cover Optimization", + "description": "Model and solve weighted set cover problems using binary selection variables, coverage constraints, and weighted sum minimization, with implementation options for both CP-SAT and MIP solvers.", + "path": "weighted_set_cover_optimization_2.md" + }, + { + "skill_id": "constrained_cardinality_assignment", + "name": "Constrained Cardinality Assignment", + "description": "Model and solve binary assignment problems with cardinality constraints, assignment limits, conditional exclusions, and linear cost minimization using MILP solvers.", + "path": "constrained_cardinality_assignment.md" + }, + { + "skill_id": "assignmentflowwithparticipationrequirements", + "name": "AssignmentFlowWithParticipationRequirements", + "description": "Model and solve mixed-integer linear programs for assignment-flow problems with minimum contributor counts and conditional minimum delivery amounts, using either direct solver APIs or algebraic modeling frameworks.", + "path": "assignmentflowwithparticipationrequirements.md" + }, + { + "skill_id": "cardinality_constrained_pairwise_interaction_maximization", + "name": "Cardinality-Constrained Pairwise Interaction Maximization", + "description": "Model and solve selection problems where exactly K items are chosen to maximize the sum of directed pairwise interaction scores between selected items, using linearized binary variables and logical constraints.", + "path": "cardinality_constrained_pairwise_interaction_maximization.md" + }, + { + "skill_id": "capacitated_facility_location_milp", + "name": "Capacitated Facility Location MILP", + "description": "Model and solve capacitated facility location problems with fixed opening costs and linear transportation costs using mixed-integer linear programming (MILP), with robust solver integration and solution validation.", + "path": "capacitated_facility_location_milp.md" + }, + { + "skill_id": "binary_coverage_maximization_with_budget", + "name": "Binary Coverage Maximization with Budget", + "description": "Model and solve binary selection problems to maximize weighted coverage subject to a budget constraint, using two-layer binary variables and activation constraints.", + "path": "binary_coverage_maximization_with_budget.md" + }, + { + "skill_id": "multi_item_allocation_with_capacity_constraints", + "name": "Multi-Item Allocation with Capacity Constraints", + "description": "Model and solve integer linear programs for allocating items under individual demand limits and multiple linear capacity constraints to maximize linear revenue.", + "path": "multi_item_allocation_with_capacity_constraints.md" + }, + { + "skill_id": "multicommodityflowallocation", + "name": "MultiCommodityFlowAllocation", + "description": "Model and solve multi-source, multi-destination, multi-commodity allocation problems with exact demand satisfaction and linear profit maximization using structured LP formulations.", + "path": "multicommodityflowallocation.md" + }, + { + "skill_id": "multiperiodproductioninventorylp", + "name": "MultiPeriodProductionInventoryLP", + "description": "Model and solve multi-period production-inventory problems with resource capacity constraints and sales limits using linear programming, with workflows for both direct solver APIs and algebraic modeling languages.", + "path": "multiperiodproductioninventorylp.md" + }, + { + "skill_id": "multicommoditynetworkflow", + "name": "MultiCommodityNetworkFlow", + "description": "Model and solve multi-commodity flow problems with shared arc capacities and commodity-specific limits, minimizing linear transportation cost.", + "path": "multicommoditynetworkflow.md" + }, + { + "skill_id": "generalized_bin_packing_container_minimization", + "name": "Generalized Bin Packing / Container Minimization", + "description": "Model and solve assignment problems where items must be assigned to containers with capacity limits, minimizing the number of containers used, using binary assignment and container usage variables.", + "path": "generalized_bin_packing_container_minimization.md" + }, + { + "skill_id": "multi_period_production_inventory_planning", + "name": "Multi-Period Production-Inventory Planning", + "description": "Formulate and solve multi-period production planning problems with inventory balance, capacity constraints, and terminal conditions, using both continuous and integer variable approaches.", + "path": "multi_period_production_inventory_planning.md" + }, + { + "skill_id": "weighted_set_cover_solver_2", + "name": "Weighted Set Cover Solver", + "description": "Model and solve weighted set cover problems by selecting subsets to cover all elements at minimum cost using binary variables, coverage constraints, and MIP solvers.", + "path": "weighted_set_cover_solver_2.md" + }, + { + "skill_id": "job_shop_scheduling_with_disjunctive_constraints", + "name": "Job Shop Scheduling with Disjunctive Constraints", + "description": "Model job shop scheduling problems using binary precedence variables and big-M constraints to sequence operations on shared machines, then solve via MIP with careful numerical tolerance settings and solution verification.", + "path": "job_shop_scheduling_with_disjunctive_constraints.md" + }, + { + "skill_id": "set_covering_with_logical_or_constraints", + "name": "Set Covering with Logical OR Constraints", + "description": "Model and solve binary selection problems where elements must be covered by at least one chosen set, minimizing total selection cost.", + "path": "set_covering_with_logical_or_constraints.md" + }, + { + "skill_id": "tsp_mtz_formulation", + "name": "TSP_MTZ_Formulation", + "description": "Model and solve the Traveling Salesperson Problem using the Miller-Tucker-Zemlin (MTZ) formulation with binary arc and integer position variables, producing exact or feasible tours via MIP/CP-SAT solvers.", + "path": "tsp_mtz_formulation.md" + }, + { + "skill_id": "binary_subset_selection_with_pairwise_interactions", + "name": "Binary Subset Selection with Pairwise Interactions", + "description": "Model and solve subset selection problems with cardinality constraints and pairwise interaction objectives using linearized binary variables.", + "path": "binary_subset_selection_with_pairwise_interactions.md" + }, + { + "skill_id": "multi_commodity_transportation_network_flow", + "name": "Multi-Commodity Transportation Network Flow", + "description": "Model and solve multi-product flow problems on capacitated networks, minimizing total transportation cost while respecting supply limits, demand requirements, and shared arc capacities.", + "path": "multi_commodity_transportation_network_flow.md" + }, + { + "skill_id": "capacitated_transportation_problem_solver", + "name": "Capacitated Transportation Problem Solver", + "description": "Model and solve balanced capacitated transportation problems with linear costs, supply/demand equality, and per-route capacity limits using continuous flow variables.", + "path": "capacitated_transportation_problem_solver.md" + }, + { + "skill_id": "generator_commitment_and_dispatch", + "name": "Generator Commitment and Dispatch", + "description": "Model and solve unit commitment problems with startup logic, capacity constraints, and cost minimization using mixed-integer linear programming.", + "path": "generator_commitment_and_dispatch.md" + }, + { + "skill_id": "integer_resource_allocation_with_demand_satisfaction", + "name": "Integer Resource Allocation with Demand Satisfaction", + "description": "Model and solve integer linear programs for minimizing total resource usage while satisfying demand requirements and respecting individual usage limits.", + "path": "integer_resource_allocation_with_demand_satisfaction.md" + }, + { + "skill_id": "assignment_problem_with_capacity_and_demand", + "name": "Assignment Problem with Capacity and Demand", + "description": "Model and solve linear assignment problems with resource availability, task demand, and per-assignment capacity limits using continuous variables and linear programming solvers.", + "path": "assignment_problem_with_capacity_and_demand.md" + }, + { + "skill_id": "tsp_with_position_variables", + "name": "TSP with Position Variables", + "description": "Model and solve the Traveling Salesperson Problem using binary arc selection and integer position assignment variables, with explicit subtour elimination constraints, to produce a minimal-distance Hamiltonian cycle.", + "path": "tsp_with_position_variables.md" + }, + { + "skill_id": "rooted_tree_optimization_with_subtree_size_constraints", + "name": "Rooted Tree Optimization with Subtree Size Constraints", + "description": "Models and solves minimum-cost rooted tree problems with connectivity and subtree size limits using single-commodity flow formulations, with two solver backends (CBC and GLPK).", + "path": "rooted_tree_optimization_with_subtree_size_constraints.md" + }, + { + "skill_id": "edge_traversal_minimization_for_required_route_coverage", + "name": "Edge Traversal Minimization for Required Route Coverage", + "description": "Models and solves routing problems requiring traversal of specified edges with minimum total distance, using integer flow formulations and either constraint programming or mixed-integer programming solvers.", + "path": "edge_traversal_minimization_for_required_route_coverage.md" + }, + { + "skill_id": "facility_location_coverage_optimizer", + "name": "Facility Location Coverage Optimizer", + "description": "Models and solves a facility location problem with mandatory coverage and premium service tiers, minimizing net cost using binary activation and coverage variables.", + "path": "facility_location_coverage_optimizer.md" + }, + { + "skill_id": "lexicographic_max_flow_min_cost_solver", + "name": "Lexicographic Max-Flow Min-Cost Solver", + "description": "Models and solves a two-stage lexicographic optimization problem that first maximizes total flow from source to sink, then minimizes total cost among all maximum-flow solutions.", + "path": "lexicographic_max_flow_min_cost_solver.md" + }, + { + "skill_id": "stable_matching_egalitarian_optimizer", + "name": "Stable Matching Egalitarian Optimizer", + "description": "Builds and solves a one-to-one stable matching problem with no blocking pairs, minimizing the egalitarian score (sum of mutual preference ranks) using either constraint programming or mixed-integer linear programming.", + "path": "stable_matching_egalitarian_optimizer.md" + }, + { + "skill_id": "maximum_weight_assignment_with_capacity_constraints", + "name": "Maximum Weight Assignment with Capacity Constraints", + "description": "Models and solves a maximum weight assignment problem where each resource has a capacity limit on the number of incident assignments it can participate in, using either CP-SAT or MILP solvers.", + "path": "maximum_weight_assignment_with_capacity_constraints.md" + }, + { + "skill_id": "2d_bin_packing_with_disjunctive_constraints", + "name": "2D Bin Packing with Disjunctive Constraints", + "description": "Models and solves 2D rectangular packing problems with non-overlap, containment, and fixed-orientation constraints, minimizing the number of sheets used via either iterative feasibility or direct MIP optimization.", + "path": "2d_bin_packing_with_disjunctive_constraints.md" + }, + { + "skill_id": "minimum_cost_flow_with_route_constraints", + "name": "Minimum Cost Flow with Route Constraints", + "description": "Model and solve a minimum cost flow problem with flow balance, minimum flow, and capacity constraints on routes using either Pyomo or OR-Tools.", + "path": "minimum_cost_flow_with_route_constraints.md" + }, + { + "skill_id": "minimax_assignment_with_absolute_difference_constraints", + "name": "Minimax Assignment with Absolute Difference Constraints", + "description": "Build and solve optimization models that assign integer values to entities while enforcing minimum absolute differences between pairs and minimizing the maximum assigned value.", + "path": "minimax_assignment_with_absolute_difference_constraints.md" + }, + { + "skill_id": "multi_commodity_flow_min_max_utilization", + "name": "Multi-Commodity Flow Min-Max Utilization", + "description": "Models and solves a multi-commodity flow problem where the objective is to minimize the maximum arc utilization across the network, using either direct LP solver APIs or algebraic modeling frameworks.", + "path": "multi_commodity_flow_min_max_utilization.md" + }, + { + "skill_id": "maximum_cardinality_matching_via_binary_edge_assignment", + "name": "Maximum Cardinality Matching via Binary Edge Assignment", + "description": "Models and solves the maximum cardinality matching problem on a general graph using binary edge assignment variables, degree constraints, and edge set restrictions, with two distinct solver backends (CP-SAT and MILP).", + "path": "maximum_cardinality_matching_via_binary_edge_assignment.md" + }, + { + "skill_id": "facility_location_problem_solver", + "name": "Facility Location Problem Solver", + "description": "Models and solves uncapacitated facility location problems using binary decision variables for facility opening and customer assignment, with fixed opening costs and variable assignment costs.", + "path": "facility_location_problem_solver.md" + }, + { + "skill_id": "minimum_group_assignment_with_conflict_separation", + "name": "Minimum Group Assignment with Conflict Separation", + "description": "Models and solves the problem of assigning items to the minimum number of groups such that conflicting pairs are never placed in the same group, using either binary assignment variables with group usage indicators or integer assignment variables with reified constraints.", + "path": "minimum_group_assignment_with_conflict_separation.md" + }, + { + "skill_id": "degree_constrained_minimum_spanning_tree_via_flow_based_connectivity", + "name": "Degree-Constrained Minimum Spanning Tree via Flow-Based Connectivity", + "description": "Build and solve a degree-constrained minimum spanning tree problem using binary edge selection, single-commodity flow for connectivity, and degree constraints, with two solver workflows: one using Pyomo with HiGHS and another using OR-Tools CP-SAT.", + "path": "degree_constrained_minimum_spanning_tree_via_flow_based_connectivity.md" + }, + { + "skill_id": "resource_constrained_assignment_problem_solver", + "name": "Resource-Constrained Assignment Problem Solver", + "description": "Models and solves assignment problems where tasks must be assigned to resources with capacity limits, minimizing total cost using either CP-SAT or MILP solvers.", + "path": "resource_constrained_assignment_problem_solver.md" + }, + { + "skill_id": "discrete_assignment_with_min_max_objective", + "name": "Discrete Assignment with Min-Max Objective", + "description": "Model and solve problems requiring assignment of discrete choices to entities with exactly-one and conflict-avoidance constraints, minimizing the maximum assigned value.", + "path": "discrete_assignment_with_min_max_objective.md" + }, + { + "skill_id": "maximum_weight_subgraph_with_degree_constraints", + "name": "Maximum Weight Subgraph with Degree Constraints", + "description": "Model and solve a maximum weight subgraph selection problem where each node has a capacity limit on the number of incident selected edges.", + "path": "maximum_weight_subgraph_with_degree_constraints.md" + }, + { + "skill_id": "quadratic_assignment_problem_solver", + "name": "Quadratic Assignment Problem Solver", + "description": "Models and solves one-to-one assignment problems with quadratic interaction costs using either MILP linearization or direct permutation enumeration.", + "path": "quadratic_assignment_problem_solver.md" + }, + { + "skill_id": "contiguous_interval_span_minimization", + "name": "contiguous_interval_span_minimization", + "description": "Models and solves problems requiring assignment of contiguous intervals to entities with non-overlap and adjacency interference constraints, minimizing the overall span using MILP or constraint programming.", + "path": "contiguous_interval_span_minimization.md" + }, + { + "skill_id": "time_constrained_routing_with_binary_decisions_and_arrival_time_variables", + "name": "Time-Constrained Routing with Binary Decisions and Arrival Time Variables", + "description": "Models and solves a single-vehicle routing problem with time windows using either a MILP formulation with MTZ constraints or a CP-SAT formulation with circuit and time propagation, minimizing total travel time.", + "path": "time_constrained_routing_with_binary_decisions_and_arrival_time_variables.md" + }, + { + "skill_id": "single_vehicle_routing_with_pickup_and_delivery_precedence", + "name": "Single Vehicle Routing with Pickup and Delivery Precedence", + "description": "Model and solve a single-vehicle routing problem that must start and end at a depot, visit all nodes exactly once, and respect precedence constraints for pickup-delivery pairs, minimizing total travel distance.", + "path": "single_vehicle_routing_with_pickup_and_delivery_precedence.md" + }, + { + "skill_id": "assignment_problem_solver", + "name": "Assignment Problem Solver", + "description": "Models and solves one-to-one assignment problems with binary decision variables, covering both MIP and CP-SAT solver backends.", + "path": "assignment_problem_solver.md" + }, + { + "skill_id": "maximum_leaf_spanning_tree", + "name": "Maximum Leaf Spanning Tree", + "description": "Models and solves the problem of finding a spanning tree that maximizes the number of leaf nodes, using either a CP-SAT or MILP solver.", + "path": "maximum_leaf_spanning_tree.md" + }, + { + "skill_id": "minimum_unique_label_assignment_via_graph_coloring", + "name": "Minimum Unique Label Assignment via Graph Coloring", + "description": "Models a conflict graph where each vertex must be assigned a unique label (e.g., frequency) such that adjacent vertices receive different labels, and minimizes the total number of distinct labels used.", + "path": "minimum_unique_label_assignment_via_graph_coloring.md" + }, + { + "skill_id": "minimum_cut_via_binary_partition", + "name": "Minimum Cut via Binary Partition", + "description": "Formulate and solve the minimum s-t cut problem using either a mixed-integer linear programming (MIP) approach with explicit binary partition variables or a max-flow reduction using the max-flow min-cut theorem.", + "path": "minimum_cut_via_binary_partition.md" + }, + { + "skill_id": "capacitated_arc_routing_problem_carp_solver", + "name": "Capacitated Arc Routing Problem (CARP) Solver", + "description": "Models and solves the Capacitated Arc Routing Problem using either a MILP-based task sequencing formulation or a state-based MILP formulation, with solver-specific extraction and validation.", + "path": "capacitated_arc_routing_problem_carp_solver.md" + }, + { + "skill_id": "permutation_flow_shop_scheduling", + "name": "Permutation Flow Shop Scheduling", + "description": "Model and solve permutation flow shop problems with makespan minimization using either CP-SAT with interval variables or MILP with position-based formulations.", + "path": "permutation_flow_shop_scheduling.md" + }, + { + "skill_id": "flexible_job_shop_scheduling_with_arbitrary_routing", + "name": "Flexible Job Shop Scheduling with Arbitrary Routing", + "description": "Model and solve flexible job shop problems where each job must visit all machines exactly once in any order, minimizing makespan using either CP-SAT with permutation variables or MILP with disjunctive constraints.", + "path": "flexible_job_shop_scheduling_with_arbitrary_routing.md" + }, + { + "skill_id": "maximum_flow_network_solver", + "name": "Maximum Flow Network Solver", + "description": "A skill for modeling and solving maximum flow problems on capacitated directed networks, providing workflows for specialized graph algorithms and general linear programming formulations.", + "path": "maximum_flow_network_solver.md" + }, + { + "skill_id": "rectangularassignmentskill", + "name": "RectangularAssignmentSkill", + "description": "Solves unbalanced assignment problems where one set must be fully matched to a subset of another set, minimizing total cost via binary linear programming or network flow.", + "path": "rectangularassignmentskill.md" + }, + { + "skill_id": "maximalcoveringlocationproblem", + "name": "MaximalCoveringLocationProblem", + "description": "A skill for solving maximal covering location problems by selecting a limited number of facilities to maximize weighted coverage of demand points within a specified distance threshold.", + "path": "maximalcoveringlocationproblem.md" + }, + { + "skill_id": "flowshopscheduling", + "name": "FlowShopScheduling", + "description": "Model and solve flow shop scheduling problems with disjunctive resources and precedence constraints to minimize makespan using either CP-SAT with interval variables or MIP with big-M disjunctive constraints.", + "path": "flowshopscheduling.md" + }, + { + "skill_id": "flexible_job_shop_scheduling_with_free_routing", + "name": "Flexible Job Shop Scheduling with Free Routing", + "description": "Model and solve flexible job shop problems where each job must visit all machines in any order, minimizing makespan via CP-SAT or MILP solvers.", + "path": "flexible_job_shop_scheduling_with_free_routing.md" + }, + { + "skill_id": "contiguous_interval_packing_with_non_overlap", + "name": "Contiguous Interval Packing with Non-Overlap", + "description": "Model and solve 1D contiguous interval packing problems with fixed lengths and pairwise non-overlap constraints to minimize the maximum used position.", + "path": "contiguous_interval_packing_with_non_overlap.md" + }, + { + "skill_id": "min_max_assignment_solver", + "name": "Min-Max Assignment Solver", + "description": "Solves one-to-one matching problems with a min-max (makespan) objective by formulating it as a Mixed-Integer Linear Program (MILP) and using a high-performance solver, with verification via feasibility checks.", + "path": "min_max_assignment_solver.md" + }, + { + "skill_id": "constrained_assignment_with_min_max_objective", + "name": "Constrained Assignment with Min-Max Objective", + "description": "Model and solve discrete assignment problems with domain restrictions, pairwise incompatibilities, and a min-max objective using either CP-SAT or MIP solvers.", + "path": "constrained_assignment_with_min_max_objective.md" + }, + { + "skill_id": "permutation_flow_shop_scheduling_2", + "name": "Permutation Flow Shop Scheduling", + "description": "Model and solve permutation flow shop problems with makespan minimization using either complete enumeration for small instances or MILP/CP-SAT formulations for larger ones, ensuring the same job order across all machines.", + "path": "permutation_flow_shop_scheduling_2.md" + }, + { + "skill_id": "minimumcostcirculation", + "name": "MinimumCostCirculation", + "description": "Model and solve minimum-cost circulation problems on directed networks with flow conservation, arc bounds, and linear costs using either a direct solver API or a modeling framework.", + "path": "minimumcostcirculation.md" + }, + { + "skill_id": "multi_machine_job_scheduling_with_precedence_and_no_overlap", + "name": "Multi-Machine Job Scheduling with Precedence and No-Overlap", + "description": "Model and solve scheduling problems where jobs must be processed on multiple machines in a fixed sequence, with each machine handling only one job at a time, to minimize the overall completion time (makespan).", + "path": "multi_machine_job_scheduling_with_precedence_and_no_overlap.md" + }, + { + "skill_id": "generalizedassignmentsolver", + "name": "GeneralizedAssignmentSolver", + "description": "Solves assignment problems with capacity constraints by formulating as a Mixed-Integer Linear Program (MILP) with binary decision variables, assignment and knapsack constraints, and a linear objective, then solving with a configured MILP solver.", + "path": "generalizedassignmentsolver.md" + }, + { + "skill_id": "cutting_stock_pattern_based_integer_optimization", + "name": "Cutting Stock - Pattern-Based Integer Optimization", + "description": "Model and solve one-dimensional cutting stock problems by enumerating feasible patterns, formulating a pattern-based integer program to minimize total stock items used, and solving with a MIP solver.", + "path": "cutting_stock_pattern_based_integer_optimization.md" + }, + { + "skill_id": "flow_shop_scheduling_with_makespan_minimization", + "name": "Flow Shop Scheduling with Makespan Minimization", + "description": "Model permutation flow shop problems with unary machine resources and precedence chains, then solve using either CP-SAT for exact solutions or MILP with symmetry breaking for heuristic/optimal results.", + "path": "flow_shop_scheduling_with_makespan_minimization.md" + }, + { + "skill_id": "assignment_problem_with_surplus_resources", + "name": "Assignment Problem with Surplus Resources", + "description": "Model and solve rectangular assignment problems (more agents than tasks) with binary assignment variables, cardinality limits, and a linear cost minimization objective using either a direct MIP formulation or a network flow transformation.", + "path": "assignment_problem_with_surplus_resources.md" + }, + { + "skill_id": "capacitatedassignmentsolver", + "name": "CapacitatedAssignmentSolver", + "description": "Solves binary assignment problems with exactly-one assignment and knapsack-style capacity constraints by formulating a Mixed-Integer Linear Program (MILP) and solving it with a configured MILP solver.", + "path": "capacitatedassignmentsolver.md" + }, + { + "skill_id": "multi_vehicle_routing_with_optional_fleet_usage", + "name": "Multi-Vehicle Routing with Optional Fleet Usage", + "description": "Formulate and solve vehicle routing problems with optional vehicle usage using binary arc variables and Miller-Tucker-Zemlin subtour elimination, producing executable routes from solver output.", + "path": "multi_vehicle_routing_with_optional_fleet_usage.md" + }, + { + "skill_id": "schedule_separation_with_makespan_minimization", + "name": "Schedule Separation with Makespan Minimization", + "description": "Model and solve scheduling problems with pairwise separation constraints and a makespan objective using integer programming techniques.", + "path": "schedule_separation_with_makespan_minimization.md" + }, + { + "skill_id": "rural_postman_problem_rpp_solver", + "name": "Rural Postman Problem (RPP) Solver", + "description": "Model and solve the Rural Postman Problem (RPP) to find a minimum-cost closed tour covering a set of required edges, using either a direct MIP formulation with connectivity cuts or a matching-based reduction to an Eulerian graph.", + "path": "rural_postman_problem_rpp_solver.md" + }, + { + "skill_id": "supply_allocation_profit_maximization", + "name": "Supply Allocation Profit Maximization", + "description": "Model and solve linear programming problems that allocate supply quantities across entities to maximize profit while satisfying demand equality and non-negativity constraints.", + "path": "supply_allocation_profit_maximization.md" + }, + { + "skill_id": "multi_product_multi_market_supply_allocation_with_integer_quantities", + "name": "Multi-Product Multi-Market Supply Allocation with Integer Quantities", + "description": "Models and solves a profit-maximizing supply allocation problem with integer quantities across multiple products and markets, using either OR-Tools or Pyomo with equality demand constraints.", + "path": "multi_product_multi_market_supply_allocation_with_integer_quantities.md" + }, + { + "skill_id": "employee_shift_assignment_with_unfulfilled_demand", + "name": "Employee Shift Assignment with Unfulfilled Demand", + "description": "Models and solves a resource-to-demand assignment problem where unfulfilled demand is penalized, using either CP-SAT or MIP solvers with binary assignment variables and integer slack variables.", + "path": "employee_shift_assignment_with_unfulfilled_demand.md" + }, + { + "skill_id": "binary_set_covering_with_cost_minimization", + "name": "Binary Set Covering with Cost Minimization", + "description": "Model and solve binary set covering problems where a subset of items must be selected to satisfy coverage requirements while minimizing total cost, using either OR-Tools or Pyomo.", + "path": "binary_set_covering_with_cost_minimization.md" + }, + { + "skill_id": "transportation_flow_optimizer", + "name": "Transportation Flow Optimizer", + "description": "Model and solve capacitated transportation problems with supply limits, demand satisfaction, and arc capacity constraints to minimize total flow cost.", + "path": "transportation_flow_optimizer.md" + }, + { + "skill_id": "weighted_set_covering_with_cardinality_constraints", + "name": "Weighted Set Covering with Cardinality Constraints", + "description": "Formulate and solve weighted set covering problems where each element must be covered by a minimum required number of selected sets, minimizing total selection cost.", + "path": "weighted_set_covering_with_cardinality_constraints.md" + }, + { + "skill_id": "multi_commodity_flow_allocation_2", + "name": "Multi-Commodity Flow Allocation", + "description": "Model and solve linear programs for allocating flows from multiple sources of multiple products to multiple destinations to maximize profit, with exact demand satisfaction and non-negative continuous variables.", + "path": "multi_commodity_flow_allocation_2.md" + }, + { + "skill_id": "bipartite_flow_allocation_with_capacity", + "name": "Bipartite Flow Allocation with Capacity", + "description": "Model and solve bipartite resource allocation problems with supply-demand balance, individual capacity limits, and linear costs using linear programming.", + "path": "bipartite_flow_allocation_with_capacity.md" + }, + { + "skill_id": "generalized_assignment_with_soft_demand", + "name": "Generalized Assignment with Soft Demand", + "description": "Model employee-to-shift assignments with skill and availability constraints, minimizing preference costs and penalties for unmet demand, then solve with deterministic configuration and robust verification.", + "path": "generalized_assignment_with_soft_demand.md" + }, + { + "skill_id": "binaryselectionmaximindistance", + "name": "BinarySelectionMaximinDistance", + "description": "Model and solve binary selection problems with pairwise activation constraints to maximize the minimum distance (or other pairwise metric) among selected items, using either a direct CP-SAT or a Pyomo-based MILP approach.", + "path": "binaryselectionmaximindistance.md" + } + ] +} diff --git a/skills/optskills/skill_library/integer_linear_programming_with_multiple_knapsack_constraints.md b/skills/optskills/skill_library/integer_linear_programming_with_multiple_knapsack_constraints.md new file mode 100644 index 0000000..91f9ff3 --- /dev/null +++ b/skills/optskills/skill_library/integer_linear_programming_with_multiple_knapsack_constraints.md @@ -0,0 +1,220 @@ +--- +name: Integer Linear Programming with Multiple Knapsack Constraints +description: | + Model and solve integer linear programs with bounded, nonnegative integer variables, a linear objective to maximize profit, and multiple linear inequality (knapsack) constraints using structured formulations and robust solver backends. +--- + +# Workflow 1 (OR-Tools / SCIP Backend) + +## Modeling stage + +### Strategy Overview +This workflow uses Google OR-Tools' linear solver wrapper (`pywraplp`) to construct a Mixed-Integer Linear Programming (MILP) model. It is ideal for direct, low-level model building with explicit variable and constraint creation, leveraging the robust SCIP or CBC solvers for integer problems. + +### Step 1 - Define Sets and Parameters +- Declare sets for items and constraints as Python lists or ranges for indexing. +- Store parameters (e.g., profit coefficients, demand upper bounds, constraint capacities, and coefficient matrices) in dictionaries or lists keyed by set indices. + +### Step 2 - Create Bounded Integer Variables +- For each item, create an integer decision variable using `solver.IntVar(lower_bound, upper_bound, name)`. +- Set the lower bound to 0 (nonnegative) and the upper bound to the item's demand limit, incorporating bounds directly to reduce constraint count. + +### Step 3 - Formulate Linear Objective +- Create an objective expression using `solver.Objective()`. +- For each variable, set its coefficient using `objective.SetCoefficient(variable, profit_coefficient)`. +- Call `objective.SetMaximization()` to define the optimization sense. + +### Step 4 - Add Multiple Knapsack Constraints +- For each constraint (knapsack), create a linear expression summing the relevant variables multiplied by their constraint-specific coefficients. +- Add the inequality to the solver using `solver.Add(sum_expr <= capacity)`. + +### Formulation Template +```json +{ + "sets": [ + "I: set of items", + "C: set of constraints (knapsacks)" + ], + "parameters": [ + "profit[i]: profit coefficient for item i in I", + "demand_limit[i]: upper bound for item i in I", + "capacity[c]: capacity of constraint c in C", + "coeff[c][i]: coefficient of item i in constraint c (often 0 or 1)" + ], + "decision_variables": [ + "x[i]: nonnegative integer quantity of item i to select, 0 <= x[i] <= demand_limit[i]" + ], + "objective": { + "sense": "max", + "expression": "sum(profit[i] * x[i] for i in I)" + }, + "constraints": [ + "sum(coeff[c][i] * x[i] for i in I) <= capacity[c] for each c in C" + ] +} +``` + +### Common Pitfalls +- Forgetting to set the upper bound on integer variables, leading to unbounded or unrealistic solutions. +- Inefficiently building constraint expressions inside nested loops for large problems; pre-structure coefficient data. +- Not using descriptive variable names, making debugging and solution interpretation difficult. + +## Solving stage + +### Strategy Overview +Solve the constructed model using the SCIP or CBC backend via OR-Tools. Configure solver parameters for performance, rigorously check the solution status, and extract and verify the integer solution. + +### Step 1 - Configure Solver and Solve +- Instantiate the solver: `solver = pywraplp.Solver.CreateSolver('SCIP')`. +- Set practical limits: `solver.SetTimeLimit(time_limit_in_milliseconds)` and `solver.SetNumThreads(number_of_threads)`. +- Call `solver.Solve()` to initiate the optimization. + +### Step 2 - Check Solver Status +- Check the result status: `status = solver.Objective().Value()` is only valid if status is `OPTIMAL` or `FEASIBLE`. +- Use `if status == pywraplp.Solver.OPTIMAL:` to handle optimal solutions; also check for `FEASIBLE`. + +### Step 3 - Extract and Verify Solution +- If the status is acceptable, extract variable values using `variable.solution_value()` and cast to integers. +- Programmatically verify that all variable bounds and constraints are satisfied within a small numerical tolerance. +- Compute the objective value from the extracted solution for cross-checking. + +### Step 4 - Structure Output +- Return a consistent output structure containing the solution status, objective value, and a dictionary of variable values. +- Log any warnings for non-optimal statuses or constraint violations. + +### Code Usage +```python +# build model from formulation +from ortools.linear_solver import pywraplp + +solver = pywraplp.Solver.CreateSolver('SCIP') +# ... (variable and constraint creation as per modeling stage) +solver.SetTimeLimit(30000) # 30 seconds +solver.SetNumThreads(4) + +# solve with status / termination checks +status = solver.Solve() +solution = {} +if status in [solver.OPTIMAL, solver.FEASIBLE]: + objective_value = solver.Objective().Value() + for i in items: + val = x[i].solution_value() + solution[i] = int(round(val)) # Ensure integer + # ... (verification logic) +else: + print("Solver did not find a feasible solution.") +``` + +### Common Pitfalls +- Assuming `OPTIMAL` status without checking, leading to errors when accessing solution values on failed solves. +- Not converting floating-point solution values to integers, which may cause issues in downstream integer-required applications. +- Omitting solution verification, potentially accepting solutions that violate constraints due to numerical tolerances. + +# Workflow 2 (Pyomo / HiGHS Backend) + +## Modeling stage + +### Strategy Overview +This workflow uses Pyomo for abstract, declarative model formulation, separating model structure from data. It leverages the HiGHS solver via Pyomo's `SolverFactory` for solving MILPs, benefiting from Pyomo's set-based indexing and rule-based constraint definitions. + +### Step 1 - Define Abstract Sets and Parameters +- Declare Pyomo Sets (`pyo.Set`) for items and constraints to enable indexed components. +- Define Pyomo Parameters (`pyo.Param`) for profit, demand limits, capacities, and constraint coefficients, initialized from data dictionaries. + +### Step 2 - Declare Bounded Integer Variables +- Create a Pyomo `Var` for items with domain `pyo.NonNegativeIntegers`. +- Set variable bounds using the `bounds` argument (e.g., `bounds=(0, model.demand_limit[i])`) to embed demand limits. + +### Step 3 - Formulate Objective Function +- Define a Pyomo `Objective` rule that maximizes the sum of profit coefficients multiplied by their corresponding variables. + +### Step 4 - Define Constraint Rules +- For each constraint, define a rule function that sums the coefficients of included variables. +- Use Pyomo's `Constraint` component, indexed by the constraint set, to create all inequalities efficiently. + +### Formulation Template +```json +{ + "sets": [ + "I: set of items", + "C: set of constraints (knapsacks)" + ], + "parameters": [ + "profit[i]: profit coefficient for item i in I", + "demand_limit[i]: upper bound for item i in I", + "capacity[c]: capacity of constraint c in C", + "coeff[c][i]: coefficient of item i in constraint c" + ], + "decision_variables": [ + "x[i]: nonnegative integer quantity of item i to select" + ], + "objective": { + "sense": "max", + "expression": "sum(profit[i] * x[i] for i in I)" + }, + "constraints": [ + "sum(coeff[c][i] * x[i] for i in I) <= capacity[c] for each c in C", + "x[i] <= demand_limit[i] for each i in I (optional, if not in variable bounds)" + ] +} +``` + +### Common Pitfalls +- Defining constraint rules that inefficiently iterate over full sets; instead, pre-compute relevant subsets for each constraint. +- Confusing Pyomo's 1-based indexing if data uses 0-based indexing; ensure consistent index mapping. +- Not using `initialize` correctly for Parameters, leading to uninitialized data errors. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the HiGHS MILP solver. Configure solver options for time limit and optimality gap, check termination conditions rigorously, and extract the solution while verifying its integer feasibility. + +### Step 1 - Configure and Execute Solver +- Create a solver object: `solver = pyo.SolverFactory('highs')`. +- Set options: `solver.options['time_limit'] = time_limit` and `solver.options['threads'] = thread_count`. +- Solve the model: `results = solver.solve(model, tee=False)`. + +### Step 2 - Inspect Solver Status and Termination +- Check `results.solver.status` is `pyo.SolverStatus.ok`. +- Check `results.solver.termination_condition` is `pyo.TerminationCondition.optimal` or `...feasible`. +- Proceed only if both checks indicate a valid solution. + +### Step 3 - Extract and Process Solution +- Access variable values using `pyo.value(model.x[i])` and convert to integers. +- Recompute constraint left-hand sides to verify satisfaction within tolerance. +- Calculate the objective value from the extracted solution as a sanity check. + +### Step 4 - Generate Structured Output +- Package the solution into a dictionary or dataclass, including status, objective value, variable values, and verification flags. +- Provide clear warnings or errors if the solution is not integer-feasible. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo + +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=items) +model.C = pyo.Set(initialize=constraints) +# ... (parameter and variable definition as per modeling stage) + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = 30 +solver.options['threads'] = 4 +results = solver.solve(model) + +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in [pyo.TerminationCondition.optimal, + pyo.TerminationCondition.feasible]): + objective_value = pyo.value(model.obj) + solution = {i: int(pyo.value(model.x[i])) for i in model.I} + # ... (verification logic) +else: + print("Solver did not return a valid solution.") +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition`, potentially misinterpreting infeasible or error states. +- Extracting variable values without converting `pyo.value` result to integer, risking floating-point values for integer variables. +- Ignoring the `tee` option during debugging; setting `tee=True` can provide valuable solver log output. diff --git a/skills/optskills/skill_library/integer_resource_allocation_with_demand_satisfaction.md b/skills/optskills/skill_library/integer_resource_allocation_with_demand_satisfaction.md new file mode 100644 index 0000000..6438238 --- /dev/null +++ b/skills/optskills/skill_library/integer_resource_allocation_with_demand_satisfaction.md @@ -0,0 +1,209 @@ +--- +name: Integer Resource Allocation with Demand Satisfaction +description: | + Model and solve integer linear programs for minimizing total resource usage while satisfying demand requirements and respecting individual usage limits. +--- + +# Workflow 1 (Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +This workflow uses the Pyomo modeling language for a declarative model definition, separating data from structure. It is well-suited for complex, sparse production matrices and integrates seamlessly with open-source solvers like HiGHS and CBC. + +### Step 1 - Define Data Structures +- Organize problem parameters into distinct dictionaries for demand, usage limits, and production yields. +- Represent the production matrix as a nested dictionary `yield_matrix[pattern][order]` to efficiently handle sparsity. +- Use Python sets to define the indices for patterns and orders. + +### Step 2 - Declare Variables and Objective +- Define non-negative integer decision variables `model.x[pattern]` using `pyo.Var(domain=pyo.NonNegativeIntegers)`. +- Set the objective to minimize the sum of all usage variables: `model.obj = pyo.Objective(expr=sum(model.x[p] for p in patterns), sense=pyo.minimize)`. + +### Step 3 - Formulate Demand Satisfaction Constraints +- For each order, create a constraint ensuring total production meets or exceeds demand. +- Use a generator expression with `.get(order, 0)` to safely sum contributions from all patterns: `sum(yield_matrix[p].get(o, 0) * model.x[p] for p in patterns) >= demand[o]`. + +### Step 4 - Apply Individual Usage Limits +- Add upper bound constraints for each pattern: `model.x[p] <= usage_limit[p]`. +- Alternatively, set variable upper bounds directly during variable declaration for a more compact model. + +### Formulation Template +```json +{ + "sets": ["patterns", "orders"], + "parameters": { + "demand": {"order": "quantity"}, + "usage_limit": {"pattern": "max_usage"}, + "yield_matrix": {"pattern": {"order": "yield_quantity"}} + }, + "decision_variables": ["x[pattern] (non-negative integer)"], + "objective": { + "sense": "min", + "expression": "sum(x[p] for p in patterns)" + }, + "constraints": [ + "demand_satisfaction[o]: sum(yield_matrix[p][o] * x[p] for p in patterns) >= demand[o] for all o in orders", + "usage_limit[p]: x[p] <= usage_limit[p] for all p in patterns" + ] +} +``` + +### Common Pitfalls +- Forgetting to handle missing keys in sparse yield matrices, leading to KeyErrors. Always use `.get(key, default_value)`. +- Defining variable bounds as constraints instead of using the variable's native `bounds` argument, which increases model size unnecessarily. +- Using list comprehensions inside Pyomo expressions without `pyo.quicksum`, which can cause performance issues. + +## Solving stage + +### Strategy Overview +The solving stage focuses on configuring the solver, executing the solve, and rigorously verifying the solution's feasibility and optimality. It emphasizes robust status checking and post-solution validation. + +### Step 1 - Configure and Execute Solver +- Instantiate the solver using `pyo.SolverFactory("highs")` or `pyo.SolverFactory("cbc")`. +- Configure key parameters: set a time limit (`time_limit`), optimality gap (`mip_rel_gap`), and number of threads. +- Execute the solve with `solver.solve(model, tee=False)`. + +### Step 2 - Check Solver Status and Termination +- Check `pyo.SolverStatus` and `pyo.TerminationCondition` from the results object. +- Accept solutions with status `ok` and termination condition `optimal` or `feasible`. Handle other conditions (e.g., `infeasible`, `maxTimeLimit`) with appropriate warnings. + +### Step 3 - Extract and Verify Solution +- Extract the objective value using `pyo.value(model.obj)`. +- Retrieve variable values with `pyo.value(model.x[p])` for non-zero usages. +- Implement a verification function that recalculates production per order from the solution and compares it against demand, reporting any violations. + +### Step 4 - Report Results +- Format output clearly, separating the objective value, pattern usage, and verification results. +- For automation, print the objective value in a parseable format like `RESULT: `. + +### Code Usage +```python +import pyomo.environ as pyo + +# Build model (refer to Modeling Stage steps) +model = pyo.ConcreteModel() +# ... model construction code ... + +# Solve with status / termination checks +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = 30 +solver.options['mip_rel_gap'] = 0.0 +results = solver.solve(model, tee=False) + +status = results.solver.status +termination = results.solver.termination_condition + +if status == pyo.SolverStatus.ok and termination in (pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible): + obj_val = pyo.value(model.obj) + print(f"RESULT: {obj_val}") + # Extract and print variable values... +else: + print(f"Solver failed. Status: {status}, Termination: {termination}") +``` + +### Common Pitfalls +- Assuming a `feasible` solution is optimal without checking the optimality gap. +- Not verifying constraints post-solve, which can miss subtle numerical issues or solver errors. +- Forgetting to set `tee=False` in production, causing unwanted console output. + +# Workflow 2 (OR-Tools with SCIP/CBC) + +## Modeling stage + +### Strategy Overview +This workflow uses Google's OR-Tools CP-SAT solver (or MPSolver with SCIP/CBC) for a procedural, API-driven modeling approach. It is efficient for large-scale problems and offers fine-grained control over the solving process. + +### Step 1 - Initialize Solver and Data +- Create a solver instance: `solver = pywraplp.Solver.CreateSolver("SCIP")`. +- Load problem data into dictionaries, similar to Workflow 1, ensuring keys are hashable. + +### Step 2 - Create Variables with Integrated Bounds +- Define integer variables using `solver.IntVar(lower_bound, upper_bound, name)`. +- Directly set the `upper_bound` argument to the pattern's usage limit, incorporating the limit into the variable definition instead of a separate constraint. + +### Step 3 - Build Demand Constraints +- For each order, create a constraint object: `ct = solver.Constraint(demand[o], solver.infinity(), f"demand_{o}")`. +- Iterate over all patterns. For each pattern, if `yield_matrix[p][o] > 0`, set the coefficient: `ct.SetCoefficient(x[p], yield_matrix[p][o])`. This implements sparse coefficient setting. + +### Step 4 - Define Linear Objective +- Create the objective expression: `objective = solver.Objective()`. +- Set all variable coefficients to 1.0: `objective.SetCoefficient(x[p], 1.0)`. +- Set the optimization sense to minimization: `objective.SetMinimization()`. + +### Formulation Template +```json +{ + "sets": ["patterns", "orders"], + "parameters": { + "demand": {"order": "quantity"}, + "usage_limit": {"pattern": "max_usage"}, + "yield_matrix": {"pattern": {"order": "yield_quantity"}} + }, + "decision_variables": ["x[pattern] (IntVar with lower=0, upper=limit)"], + "objective": { + "sense": "min", + "expression": "sum(x[p] for p in patterns)" + }, + "constraints": [ + "demand_satisfaction[o]: sum(yield_matrix[p][o] * x[p] for p in patterns) >= demand[o]" + ] +} +``` + +### Common Pitfalls +- Creating constraints with `solver.infinity()` as the lower bound by mistake; the lower bound should be the demand. +- Not leveraging variable upper bounds for usage limits, which adds unnecessary constraints and slows down the solver. +- Using `solver.NumVar` instead of `solver.IntVar`, resulting in a continuous relaxation. + +## Solving stage + +### Strategy Overview +The solving stage focuses on configuring solver parameters for performance, executing the solve, and implementing a comprehensive verification routine to ensure the solution's integrity. + +### Step 1 - Configure Solver Parameters +- Set a time limit: `solver.SetTimeLimit(30000)` (time in milliseconds). +- Configure parallelism: `solver.SetNumThreads(4)`. +- For CP-SAT, set additional parameters like `solver.parameters.max_time_in_seconds`. + +### Step 2 - Solve and Check Status +- Execute the solve: `status = solver.Solve()`. +- Check for `pywraplp.Solver.OPTIMAL` or `pywraplp.Solver.FEASIBLE` status. Provide clear messages for other statuses like `INFEASIBLE` or `ABNORMAL`. + +### Step 3 - Extract and Validate Solution +- If the status is acceptable, extract the objective value: `solver.Objective().Value()`. +- Extract variable values: `x[p].solution_value()`. +- Run a verification loop: for each order, sum `yield_matrix[p][o] * x[p].solution_value()` and assert it meets demand. Also verify variable values do not exceed their upper bounds. + +### Step 4 - Format and Output Results +- Print a summary including the objective value and a table of non-zero variable values. +- Optionally, output results in JSON format for easy integration with other systems. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# Build model from formulation +solver = pywraplp.Solver.CreateSolver('SCIP') +# ... variable and constraint creation ... + +# solve with status / termination checks +solver.SetTimeLimit(30000) +status = solver.Solve() + +if status in (solver.OPTIMAL, solver.FEASIBLE): + obj_val = solver.Objective().Value() + print(f"RESULT: {obj_val}") + # Verification loop + for o in orders: + total_prod = sum(yield_matrix[p].get(o, 0) * x[p].solution_value() for p in patterns) + if total_prod < demand[o] - 1e-6: # tolerance for numerical issues + print(f"WARNING: Demand for {o} not met.") +else: + print(f"No feasible solution found. Solver status: {status}") +``` + +### Common Pitfalls +- Not using a tolerance when checking demand satisfaction post-solve, leading to false failures due to floating-point arithmetic. +- Ignoring the `FEASIBLE` status and only accepting `OPTIMAL`, potentially discarding good solutions when time limits are hit. +- Forgetting to check variable bounds during verification, missing potential model definition errors. diff --git a/skills/optskills/skill_library/job_shop_scheduling_with_disjunctive_constraints.md b/skills/optskills/skill_library/job_shop_scheduling_with_disjunctive_constraints.md new file mode 100644 index 0000000..542f0a6 --- /dev/null +++ b/skills/optskills/skill_library/job_shop_scheduling_with_disjunctive_constraints.md @@ -0,0 +1,261 @@ +--- +name: Job Shop Scheduling with Disjunctive Constraints +description: | + Model job shop scheduling problems using binary precedence variables and big-M constraints to sequence operations on shared machines, then solve via MIP with careful numerical tolerance settings and solution verification. + +--- + +# Workflow 1 (Standard MIP with Gurobi) + +## Modeling stage + +### Strategy Overview +Formulate the job shop problem as a Mixed-Integer Program (MIP) using a disjunctive constraint formulation. Binary variables determine the precedence order between any two operations requiring the same machine. Continuous variables represent operation start times and the overall makespan. + +### Step 1 - Define Problem Data Structures +- Define sets for jobs, operations per job, and machines. +- Represent each operation as a tuple `(job_id, step_index)` for unique identification. +- Create a parameter mapping each operation to its required machine. +- Define a parameter for the processing time of each operation. + +### Step 2 - Create Decision Variables +- Create a continuous, non-negative variable for the start time of each operation. +- Create a continuous, non-negative variable for the makespan. +- For each unordered pair of operations assigned to the same machine, create a binary variable. A value of 1 indicates the first operation precedes the second. + +### Step 3 - Formulate Constraints +- **Job Precedence**: For each job, enforce `start_time[(job, step+1)] >= start_time[(job, step)] + processing_time[(job, step)]` for consecutive operations. +- **Machine Disjunction**: For each unordered pair `(op1, op2)` sharing a machine, use binary variable `precedes[op1, op2]` and big-M constraints: + - `start_time[op2] >= start_time[op1] + processing_time[op1] - M * (1 - precedes[op1, op2])` + - `start_time[op1] >= start_time[op2] + processing_time[op2] - M * precedes[op1, op2]` +- **Makespan Definition**: `makespan >= start_time[op] + processing_time[op]` for every operation. +- **Big-M Constant**: Set `M` to the sum of all processing times, a valid upper bound for start time differences. + +### Step 4 - Define Objective +- Set the objective to minimize the makespan variable. + +### Formulation Template +```json +{ + "sets": [ + {"name": "Jobs", "description": "Set of all jobs"}, + {"name": "Operations", "description": "Set of operations as tuples (job, step_index)"}, + {"name": "Machines", "description": "Set of all machines"}, + {"name": "MachinePairs", "description": "Set of unordered operation pairs ((job1, step1), (job2, step2)) that require the same machine", "dimen": 4} + ], + "parameters": [ + {"name": "machine_assignment", "domain": "Operations -> Machines", "description": "Machine required for each operation"}, + {"name": "processing_time", "domain": "Operations -> Real+", "description": "Processing duration for each operation"}, + {"name": "M", "domain": "Real+", "description": "Big-M constant, set to sum of all processing times"} + ], + "decision_variables": [ + {"name": "start_time", "domain": "Operations >= 0", "type": "Continuous", "description": "Start time of each operation"}, + {"name": "makespan", "domain": ">= 0", "type": "Continuous", "description": "Maximum completion time"}, + {"name": "precedes", "domain": "MachinePairs in {0,1}", "type": "Binary", "description": "1 if first operation precedes second in the pair"} + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + {"name": "job_precedence", "expression": "start_time[(job, step+1)] >= start_time[(job, step)] + processing_time[(job, step)] for all jobs and steps"}, + {"name": "disjunctive_order_1", "expression": "start_time[op2] >= start_time[op1] + processing_time[op1] - M * (1 - precedes[op1, op2]) for all (op1, op2) in MachinePairs"}, + {"name": "disjunctive_order_2", "expression": "start_time[op1] >= start_time[op2] + processing_time[op2] - M * precedes[op1, op2] for all (op1, op2) in MachinePairs"}, + {"name": "makespan_def", "expression": "makespan >= start_time[op] + processing_time[op] for all op in Operations"} + ] +} +``` + +### Common Pitfalls +- Hardcoding machine assignments instead of reading from a parameter. +- Creating duplicate binary variables for both orderings of an operation pair, leading to a redundant model. +- Using a big-M constant that is too small (infeasible) or excessively large (causing numerical instability). Use the sum of all processing times. +- Omitting validation that all required input parameters are provided before building the model. + +## Solving stage + +### Strategy Overview +Solve the MIP using the Gurobi solver with tightened numerical tolerances to ensure the big-M constraints are respected precisely. After solving, verify the solution status and check the resulting schedule for constraint violations. + +### Step 1 - Configure Solver and Solve +- Instantiate the Gurobi solver via Pyomo's `SolverFactory`. +- Set key options: `MIPGap=0.0` for optimality, `TimeLimit` to a reasonable value (e.g., `[TIME_LIMIT]`), `Threads` for parallelism, and `Seed` for reproducibility. +- Tighten feasibility and integer tolerances: `FeasibilityTol=1e-9`, `IntFeasTol=1e-9`. +- Call the solver with `tee=False` for clean output. + +### Step 2 - Check Solver Status and Load Solution +- Check if the solver status is `ok`. +- Check the termination condition for `optimal` or `feasible`. Proceed only if one is met. +- If the solve was successful, load the solution into the model instance. + +### Step 3 - Extract and Verify Schedule +- Extract the values of start time and precedence variables. +- **Verify job precedence constraints**: For each job, compare start times of consecutive operations to ensure the successor starts after the predecessor finishes. +- **Verify machine capacity constraints**: For each machine, sort assigned operations by start time and ensure no overlap (i.e., `start_time[next] >= start_time[prev] + processing_time[prev]`). +- Calculate the makespan from the extracted completion times and verify it matches the model's makespan variable. +- Print a human-readable schedule (operation, machine, start, finish) in a tabular format. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# ... (model building code based on formulation) ... + +# Step 1: Configure and solve +solver = pyo.SolverFactory('gurobi') +solver.options['MIPGap'] = 0.0 +solver.options['TimeLimit'] = [TIME_LIMIT] +solver.options['Threads'] = 4 +solver.options['Seed'] = 42 +solver.options['FeasibilityTol'] = 1e-9 +solver.options['IntFeasTol'] = 1e-9 + +results = solver.solve(model, tee=False) + +# Step 2: Check status and load +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in [TerminationCondition.optimal, + TerminationCondition.feasible]): + model.solutions.load_from(results) + # Step 3: Extract and verify + schedule = {} + for idx in model.Operations: + schedule[idx] = pyo.value(model.start_time[idx]) + makespan_val = pyo.value(model.makespan) + # ... Add verification logic for precedence and disjunctive constraints ... + # ... Calculate makespan from completion times and compare to makespan_val ... + print(f"Makespan: {makespan_val}") +else: + print("Solve failed or no feasible solution found.") +``` + +### Common Pitfalls +- Accepting solver results without checking termination condition. +- Not tightening numerical tolerances, allowing the solver to exploit slack in big-M constraints, which can produce infeasible schedules. +- Failing to verify the extracted schedule against the original problem constraints. +- Using conflicting solver options or incorrect option names for the chosen solver. + +# Workflow 2 (Time-Indexed MIP with HiGHS) + +## Modeling stage + +### Strategy Overview +Use a time-indexed formulation where binary variables indicate if an operation starts at a specific time period. This avoids big-M constraints but can lead to a larger model. The makespan is implicitly defined by the latest start time plus processing time. + +### Step 1 - Define Time Horizon and Sets +- Estimate a reasonable time horizon `T` (e.g., sum of all processing times). +- Define sets for jobs, operations, machines, and discrete time periods `{0,...,T-1}`. +- Define parameters for machine assignment and processing time for each operation. + +### Step 2 - Create Binary Decision Variables +- For each operation and each feasible start time `t` (where `t + processing_time <= T`), create a binary variable `x[op, t]`. It equals 1 if the operation starts at time `t`. +- Create a continuous makespan variable. + +### Step 3 - Formulate Assignment and Sequencing Constraints +- **Start Exactly Once**: `sum_{t in feasible_start[op]} x[op, t] == 1` for each operation. +- **Machine Capacity**: For each machine `m` and time period `t`, ensure the total number of operations active on `m` at time `t` does not exceed 1: `sum_{op in Ops_m} sum_{tau in active_periods(op, t)} x[op, tau] <= 1`. +- **Job Precedence**: For consecutive operations `(pred, succ)` in a job, enforce `sum_{t} t * x[succ, t] >= sum_{t} t * x[pred, t] + processing_time[pred]`. +- **Makespan Definition**: `makespan >= sum_{t} (t + processing_time[op]) * x[op, t]` for all operations. + +### Step 4 - Define Objective +- Minimize the makespan variable. + +### Formulation Template +```json +{ + "sets": [ + {"name": "Jobs", "description": "Set of all jobs"}, + {"name": "Operations", "description": "Set of operations (job, step_index)"}, + {"name": "Machines", "description": "Set of all machines"}, + {"name": "TimePeriods", "description": "Discrete time periods {0, 1, ..., T-1}"} + ], + "parameters": [ + {"name": "machine_assignment", "domain": "Operations -> Machines", "description": "Machine required for each operation"}, + {"name": "processing_time", "domain": "Operations -> Integer+", "description": "Processing duration for each operation"}, + {"name": "T", "domain": "Integer+", "description": "Time horizon upper bound"} + ], + "decision_variables": [ + {"name": "x", "domain": "Operations * TimePeriods in {0,1}", "type": "Binary", "description": "1 if operation starts at time t"}, + {"name": "makespan", "domain": ">= 0", "type": "Continuous", "description": "Maximum completion time"} + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + {"name": "start_exactly_once", "expression": "sum_{t in feasible_start[op]} x[op, t] == 1 for all op in Operations"}, + {"name": "machine_capacity", "expression": "sum_{op in Ops_m} sum_{tau in active_periods(op, t)} x[op, tau] <= 1 for all m in Machines, t in TimePeriods"}, + {"name": "job_precedence", "expression": "sum_{t} t * x[succ, t] >= sum_{t} t * x[pred, t] + processing_time[pred] for all (pred, succ) in job sequences"}, + {"name": "makespan_def", "expression": "makespan >= sum_{t} (t + processing_time[op]) * x[op, t] for all op in Operations"} + ] +} +``` + +### Common Pitfalls +- Choosing an overly large time horizon `T`, which unnecessarily blows up the model size and solve time. +- Incorrectly formulating the machine capacity constraint by not correctly summing over the periods where an operation started at `tau` would be active at time `t`. +- Linearizing the job precedence constraint incorrectly, leading to invalid sequencing. +- Forgetting to enforce that operations cannot start at a time that would cause them to finish after `T`. + +## Solving stage + +### Strategy Overview +Solve the time-indexed MIP using the HiGHS solver. Due to the potentially large number of binary variables, focus on setting appropriate limits for gap and runtime. Handle solution loading carefully as HiGHS may not load solutions automatically on non-optimal terminates. + +### Step 1 - Configure HiGHS Solver +- Instantiate the HiGHS solver via Pyomo's `SolverFactory`. +- Set options: `time_limit` for runtime control, `mip_rel_gap` for optimality tolerance. + +### Step 2 - Solve and Check Termination +- Call the solver with `load_solutions=False` to prevent automatic loading of potentially incomplete solutions. +- Check the solver results object for status `ok` and a termination condition of `optimal` or `feasible`. + +### Step 3 - Load Solution and Calculate Metrics +- If the termination is acceptable, manually load the solution into the model. +- Extract the start times by finding the time `t` where `x[op, t] == 1` for each operation (account for numerical tolerance, e.g., `> 0.5`). +- Compute the makespan from the loaded start times and processing times. + +### Step 4 - Validate Schedule +- Reconstruct the schedule from the start times. +- Verify machine capacity constraints for each time period and job precedence constraints. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# ... (model building code for time-indexed formulation) ... + +# Step 1 & 2: Configure, solve, and check +solver = pyo.SolverFactory('appsi_highs') # or 'highs' depending on Pyomo version +solver.options['time_limit'] = [TIME_LIMIT] +solver.options['mip_rel_gap'] = 0.01 + +results = solver.solve(model, load_solutions=False) + +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in [TerminationCondition.optimal, + TerminationCondition.feasible]): + # Step 3: Load solution + model.solutions.load_from(results) + # Extract start times + start_times = {} + for op in model.Operations: + for t in model.TimePeriods: + if pyo.value(model.x[op, t]) > 0.5: # Check binary variable + start_times[op] = t + break + # Calculate makespan + makespan_calc = max(start_times[op] + model.processing_time[op] for op in model.Operations) + # Step 4: Add validation logic + print(f"Calculated Makespan: {makespan_calc}") +else: + print("HiGHS solve did not return a usable solution.") +``` + +### Common Pitfalls +- Calling `solve()` without `load_solutions=False` and then trying to load results manually, causing conflicts. +- Not checking termination condition and assuming an optimal solution was found when the solver hit a time limit. +- Using an incorrect method to extract start times from the binary variables (e.g., not accounting for numerical tolerance when checking binary variable values). +- Setting solver options that are not applicable to HiGHS, causing warnings or errors. diff --git a/skills/optskills/skill_library/lexicographic_max_flow_min_cost_solver.md b/skills/optskills/skill_library/lexicographic_max_flow_min_cost_solver.md new file mode 100644 index 0000000..ff1fec5 --- /dev/null +++ b/skills/optskills/skill_library/lexicographic_max_flow_min_cost_solver.md @@ -0,0 +1,269 @@ +--- +name: Lexicographic Max-Flow Min-Cost Solver +description: | + Models and solves a two-stage lexicographic optimization problem that first maximizes total flow from source to sink, then minimizes total cost among all maximum-flow solutions. +--- + +# Workflow 1 (Sequential Pyomo with HiGHS) + +## Modeling stage + +### Strategy Overview +Formulate the lexicographic objective as two separate Pyomo models solved sequentially. Stage 1 maximizes total flow into the sink. Stage 2 fixes the optimal flow value from Stage 1 and minimizes total cost. Both models share the same variable definitions, flow conservation, and capacity constraints. + +### Step 1 - Define Sets and Parameters +- Define a set `NODES` containing all nodes, and a set `ARCS` as a subset of `NODES x NODES`. +- Specify source node `source` and sink node `sink` as scalar parameters. +- Define parameter `capacity[arc]` for each arc, and `cost[arc]` for each arc. + +### Step 2 - Declare Decision Variables +- Create a single continuous variable `flow_on_arc[arc]` for each arc in `ARCS`. +- Set domain to `pyo.NonNegativeReals` to enforce nonnegativity. + +### Step 3 - Enforce Flow Conservation +- For each intermediate node `n` (not source or sink), write a constraint: `sum(flow_on_arc[i,n] for i in NODES if (i,n) in ARCS) == sum(flow_on_arc[n,j] for j in NODES if (n,j) in ARCS)`. +- Use `pyo.Constraint.Skip` for source and sink nodes. + +### Step 4 - Enforce Arc Capacity +- For each arc `(i,j)` in `ARCS`, write: `flow_on_arc[i,j] <= capacity[i,j]`. + +### Step 5 - Stage 1 Objective (Max Flow) +- Define objective `obj1` as `sum(flow_on_arc[i,sink] for i in NODES if (i,sink) in ARCS)` with `sense=pyo.maximize`. + +### Step 6 - Stage 2 Objective (Min Cost) +- After Stage 1, create a new model with the same variables and constraints. +- Add a fixed-flow constraint: `sum(flow_on_arc[i,sink] for i in NODES if (i,sink) in ARCS) == max_flow_value`. +- Define objective `obj2` as `sum(cost[i,j] * flow_on_arc[i,j] for (i,j) in ARCS)` with `sense=pyo.minimize`. + +### Formulation Template +```json +{ + "sets": ["NODES", "ARCS"], + "parameters": ["source", "sink", "capacity[ARCS]", "cost[ARCS]"], + "decision_variables": ["flow_on_arc[ARCS]"], + "objective": { + "sense": "lexicographic (max flow, then min cost)", + "stage1": "maximize total flow into sink", + "stage2": "minimize total cost given max flow" + }, + "constraints": [ + "flow_conservation: sum(inflow) == sum(outflow) for each intermediate node", + "arc_capacity: flow_on_arc[i,j] <= capacity[i,j] for each arc", + "nonnegativity: flow_on_arc >= 0 (via domain)", + "stage2_fixed_flow: total flow into sink equals max_flow_value" + ] +} +``` + +### Common Pitfalls +- Forgetting to skip source and sink nodes in flow conservation constraints, causing infeasibility. +- Using the same model object for both stages without properly fixing the flow value, leading to incorrect cost minimization. +- Defining flow conservation with incorrect indexing (e.g., using `for each node` without filtering). + +## Solving stage + +### Strategy Overview +Use Pyomo with the HiGHS solver for efficient LP solving. Execute Stage 1 to obtain the maximum flow value, then build and solve Stage 2 with the fixed flow constraint. Always check solver status and termination conditions before proceeding. + +### Step 1 - Build and Solve Stage 1 Model +- Instantiate `pyo.SolverFactory("highs")` and set `solver.options["time_limit"] = [TIME_LIMIT]`. +- Solve the Stage 1 model: `results = solver.solve(m1, tee=False)`. +- Check `results.solver.status == SolverStatus.ok` and `results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}`. +- Extract `max_flow_value = float(pyo.value(m1.obj1))`. + +### Step 2 - Build and Solve Stage 2 Model +- Create a new Pyomo model `m2` with identical variables and constraints. +- Add the fixed-flow constraint using the `max_flow_value` from Stage 1. +- Solve `m2` with the same solver and status checks. +- Extract `min_cost_value = float(pyo.value(m2.obj2))`. + +### Step 3 - Extract and Verify Results +- For each arc, retrieve `flow_on_arc` values from `m2`. +- Manually verify flow conservation at all nodes and total cost calculation. +- Output results in a structured format (e.g., JSON with status, max flow, min cost, arc flows). + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# --- Stage 1: Max Flow --- +m1 = pyo.ConcreteModel() +m1.NODES = pyo.Set(initialize=node_list) +m1.ARCS = pyo.Set(initialize=arc_list, dimen=2) +m1.capacity = pyo.Param(m1.ARCS, initialize=capacity_dict) +m1.cost = pyo.Param(m1.ARCS, initialize=cost_dict) +m1.flow = pyo.Var(m1.ARCS, domain=pyo.NonNegativeReals) + +def flow_conservation_rule(m, n): + if n == source or n == sink: + return pyo.Constraint.Skip + inflow = sum(m.flow[i, n] for i in m.NODES if (i, n) in m.ARCS) + outflow = sum(m.flow[n, j] for j in m.NODES if (n, j) in m.ARCS) + return inflow == outflow +m1.flow_cons = pyo.Constraint(m1.NODES, rule=flow_conservation_rule) + +def cap_rule(m, i, j): + return m.flow[i, j] <= m.capacity[i, j] +m1.cap_cons = pyo.Constraint(m1.ARCS, rule=cap_rule) + +m1.obj1 = pyo.Objective( + expr=sum(m1.flow[i, sink] for i in m1.NODES if (i, sink) in m1.ARCS), + sense=pyo.maximize +) + +solver = pyo.SolverFactory("highs") +solver.options["time_limit"] = [TIME_LIMIT] +results = solver.solve(m1, tee=False) +assert results.solver.status == SolverStatus.ok +assert results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible} +max_flow_value = float(pyo.value(m1.obj1)) + +# --- Stage 2: Min Cost --- +m2 = pyo.ConcreteModel() +m2.NODES = pyo.Set(initialize=node_list) +m2.ARCS = pyo.Set(initialize=arc_list, dimen=2) +m2.capacity = pyo.Param(m2.ARCS, initialize=capacity_dict) +m2.cost = pyo.Param(m2.ARCS, initialize=cost_dict) +m2.flow = pyo.Var(m2.ARCS, domain=pyo.NonNegativeReals) + +m2.flow_cons = pyo.Constraint(m2.NODES, rule=flow_conservation_rule) +m2.cap_cons = pyo.Constraint(m2.ARCS, rule=cap_rule) + +def fixed_flow_rule(m): + return sum(m.flow[i, sink] for i in m.NODES if (i, sink) in m.ARCS) == max_flow_value +m2.fixed_flow = pyo.Constraint(rule=fixed_flow_rule) + +m2.obj2 = pyo.Objective( + expr=sum(m2.cost[i, j] * m2.flow[i, j] for (i, j) in m2.ARCS), + sense=pyo.minimize +) + +results2 = solver.solve(m2, tee=False) +assert results2.solver.status == SolverStatus.ok +assert results2.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible} +min_cost_value = float(pyo.value(m2.obj2)) + +# Output +print(f"Max flow: {max_flow_value}, Min cost: {min_cost_value}") +``` + +### Common Pitfalls +- Not checking solver termination condition, leading to use of suboptimal or infeasible solutions. +- Forgetting to create a new model for Stage 2 instead of reusing the Stage 1 model, causing variable conflicts. +- Using `tee=True` in production, which clutters output and slows execution. + +# Workflow 2 (OR-Tools Specialized Solvers) + +## Modeling stage + +### Strategy Overview +Leverage OR-Tools' dedicated network flow solvers (`SimpleMaxFlow` and `SimpleMinCostFlow`) for the lexicographic problem. These solvers are purpose-built for flow problems and handle the underlying algorithms efficiently. The modeling is done via solver API calls rather than algebraic constraints. + +### Step 1 - Define Network Topology +- Create a list of arcs as tuples `(from_node, to_node)` with integer node indices. +- Define parallel arrays for arc capacities and unit costs. +- Identify source and sink node indices. + +### Step 2 - Stage 1: Max Flow Modeling +- Instantiate `pywrapgraph.SimpleMaxFlow`. +- For each arc, call `AddArcWithCapacity(from_node, to_node, capacity)`. +- Call `Solve(source, sink)` to compute maximum flow. + +### Step 3 - Stage 2: Min-Cost Flow Modeling +- Instantiate `pywrapgraph.SimpleMinCostFlow`. +- Set supply at source to `max_flow_value` and demand at sink to `-max_flow_value` using `SetNodeSupply(node, supply)`. +- For each arc, call `AddArcWithCapacityAndUnitCost(from_node, to_node, capacity, unit_cost)`. +- Call `Solve()` to compute min-cost flow. + +### Formulation Template +```json +{ + "sets": ["nodes (implicit via indices)", "arcs (list of tuples)"], + "parameters": ["source_index", "sink_index", "capacity_per_arc", "unit_cost_per_arc"], + "decision_variables": ["flow_on_arc (managed internally by solver)"], + "objective": { + "sense": "lexicographic", + "stage1": "maximize total flow", + "stage2": "minimize total cost given max flow" + }, + "constraints": [ + "flow_conservation (enforced by solver)", + "arc_capacity (enforced by solver)", + "nonnegativity (enforced by solver)", + "stage2_supply_demand: source supply == max_flow_value, sink demand == -max_flow_value" + ] +} +``` + +### Common Pitfalls +- Using non-integer node indices or capacities, as OR-Tools solvers require integer inputs. +- Forgetting to set node supplies for all nodes (non-source/sink nodes must have supply 0). +- Mixing up the order of arguments in `AddArcWithCapacityAndUnitCost`. + +## Solving stage + +### Strategy Overview +Execute two sequential OR-Tools solver calls. First, use `SimpleMaxFlow` to get the maximum flow value. Then, use `SimpleMinCostFlow` with the source supply set to that value to find the minimum cost flow. Extract results using solver API methods. + +### Step 1 - Solve Max Flow +- Create `SimpleMaxFlow` object and add all arcs with capacities. +- Call `Solve(source, sink)` and check return status is `SimpleMaxFlow.OPTIMAL`. +- Retrieve `max_flow_value = max_flow.OptimalFlow()`. + +### Step 2 - Solve Min-Cost Flow +- Create `SimpleMinCostFlow` object. +- Set source supply to `max_flow_value` and sink supply to `-max_flow_value`. +- Set all other node supplies to 0. +- Add all arcs with capacities and unit costs. +- Call `Solve()` and check return status is `SimpleMinCostFlow.OPTIMAL`. +- Retrieve `min_cost_value = min_cost_flow.OptimalCost()`. + +### Step 3 - Extract Arc Flows +- For each arc index `i`, call `min_cost_flow.Flow(i)` to get the flow on that arc. +- Verify that total flow into sink equals `max_flow_value`. +- Output results in a structured format. + +### Code Usage +```python +from ortools.graph import pywrapgraph + +# --- Stage 1: Max Flow --- +max_flow = pywrapgraph.SimpleMaxFlow() +for (start, end), cap in zip(arcs, capacities): + max_flow.AddArcWithCapacity(start, end, int(cap)) + +status = max_flow.Solve(source, sink) +if status != max_flow.OPTIMAL: + raise RuntimeError(f"Max flow failed with status {status}") + +max_flow_value = max_flow.OptimalFlow() +print(f"Maximum flow: {max_flow_value}") + +# --- Stage 2: Min-Cost Flow --- +min_cost_flow = pywrapgraph.SimpleMinCostFlow() +min_cost_flow.SetNodeSupply(source, int(max_flow_value)) +min_cost_flow.SetNodeSupply(sink, -int(max_flow_value)) +# Set all other node supplies to 0 +for node in all_nodes: + if node != source and node != sink: + min_cost_flow.SetNodeSupply(node, 0) + +for (start, end), cap, cost in zip(arcs, capacities, unit_costs): + min_cost_flow.AddArcWithCapacityAndUnitCost(start, end, int(cap), int(cost)) + +status = min_cost_flow.Solve() +if status != min_cost_flow.OPTIMAL: + raise RuntimeError(f"Min-cost flow failed with status {status}") + +min_cost_value = min_cost_flow.OptimalCost() +print(f"Minimum cost: {min_cost_value}") + +# Extract arc flows +flow_on_arcs = [min_cost_flow.Flow(i) for i in range(min_cost_flow.NumArcs())] +``` + +### Common Pitfalls +- Not converting float capacities/costs to integers, causing silent truncation or errors. +- Forgetting to set zero supply for intermediate nodes, leading to infeasibility. +- Assuming arc indices in `SimpleMinCostFlow` correspond to the order they were added (they do, but verify with `Tail(i)` and `Head(i)` if needed). diff --git a/skills/optskills/skill_library/maximalcoveringlocationproblem.md b/skills/optskills/skill_library/maximalcoveringlocationproblem.md new file mode 100644 index 0000000..5bc2e64 --- /dev/null +++ b/skills/optskills/skill_library/maximalcoveringlocationproblem.md @@ -0,0 +1,257 @@ +--- +name: MaximalCoveringLocationProblem +description: | + A skill for solving maximal covering location problems by selecting a limited number of facilities to maximize weighted coverage of demand points within a specified distance threshold. + +--- +# Workflow 1 (Explicit Coverage Sets) + +## Modeling stage + +### Strategy Overview +Explicitly precompute the coverage relationship between demand points and candidate facilities before model construction. This separation of data preparation and modeling leads to cleaner, more readable constraints and can improve solver performance by reducing constraint complexity. + +### Step 1 - Define Sets and Parameters +- Define `I` as the set of demand points and `J` as the set of candidate facility sites. +- Define `w_i` as the weight (value) for covering demand point `i`. +- Define `d_{ij}` as the distance between demand point `i` and candidate site `j`. +- Define `R` as the coverage radius (distance threshold). +- Define `k` as the cardinality limit for the number of facilities to select. + +### Step 2 - Precompute Coverage Sets +- For each demand point `i`, compute `N(i) = {j in J | d_{ij} <= R}`. This is the set of candidate sites that can cover demand point `i`. +- Identify any demand points where `N(i)` is empty. These points are uncoverable and must be handled separately to avoid infeasible constraints. + +### Step 3 - Create Decision Variables +- Create binary variable `x_j` for each candidate site `j` in `J`. `x_j = 1` if site `j` is selected. +- Create binary variable `y_i` for each demand point `i` in `I`. `y_i = 1` if demand point `i` is covered. + +### Step 4 - Formulate Constraints +- Add cardinality constraint: `sum_{j in J} x_j = k`. +- For each demand point `i` where `N(i)` is not empty, add coverage activation constraint: `y_i <= sum_{j in N(i)} x_j`. +- For each demand point `i` where `N(i)` is empty, fix `y_i = 0`. + +### Step 5 - Define Objective +- Maximize total weighted coverage: `maximize sum_{i in I} w_i * y_i`. + +### Formulation Template +```json +{ + "sets": [ + "I: demand_points", + "J: candidate_sites" + ], + "parameters": [ + "w[i in I]: weight of demand point i", + "d[i in I][j in J]: distance from i to j", + "R: coverage radius", + "k: number of facilities to select" + ], + "decision_variables": [ + "x[j in J]: binary, 1 if site j selected", + "y[i in I]: binary, 1 if demand point i covered" + ], + "objective": { + "sense": "max", + "expression": "sum(i in I) w[i] * y[i]" + }, + "constraints": [ + "cardinality: sum(j in J) x[j] == k", + "coverage[i in I where N(i) nonempty]: y[i] <= sum(j in N(i)) x[j]", + "uncoverable[i in I where N(i) empty]: y[i] == 0" + ] +} +``` + +### Common Pitfalls +- Forgetting to handle demand points with empty coverage sets (`N(i) = {}`), which leads to infeasible constraints if `y_i` is forced to be `<= 0`. +- Incorrectly defining the coverage set `N(i)` (e.g., using strict inequality `<` instead of `<=` for the distance threshold). +- Not verifying that the precomputed coverage sets align with the distance matrix and threshold after data loading. + +## Solving stage + +### Strategy Overview +Use a high-performance MIP solver (e.g., Gurobi, HiGHS) via a modeling framework (e.g., Pyomo, OR-Tools). Focus on configuring the solver for optimality, implementing solution verification, and providing structured output. + +### Step 1 - Solver Initialization and Configuration +- Instantiate a solver object suitable for MIP problems. +- Set a time limit (e.g., `SetTimeLimit([TIME_LIMIT])`). +- Set the number of threads for parallel solving (e.g., `SetNumThreads(4)`). +- Set optimality tolerance (e.g., `SetRelativeGapTolerance(0.0)` for an optimality certificate). + +### Step 2 - Build Model from Formulation +- Translate the mathematical formulation into solver API calls. +- Add all variables, constraints, and the objective function as defined in the modeling stage. + +### Step 3 - Solve and Check Status +- Execute the `Solve()` method. +- Check the solver status (`OPTIMAL`, `FEASIBLE`, `INFEASIBLE`, etc.) and termination condition. +- Proceed only if a feasible or optimal solution is found. + +### Step 4 - Extract and Verify Solution +- Extract the values of `x_j` and `y_i` variables. Use a tolerance (e.g., `> 0.5`) to interpret binary values. +- Verify the solution: ensure selected facilities respect the cardinality `k`, and that every demand point with `y_i = 1` is within distance `R` of at least one selected facility. +- Compute the achieved objective value from the extracted solution for cross-checking. + +### Step 5 - Output Structured Results +- Output key results: selected facility indices, covered demand point indices, and total covered weight. +- Format output (e.g., JSON) for downstream processing, including solver status and verification flag. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('SCIP') +solver.SetTimeLimit([TIME_LIMIT]) +solver.SetNumThreads(4) + +# Create variables +x = [solver.IntVar(0, 1, f'x_{j}') for j in range(n_sites)] +y = [solver.IntVar(0, 1, f'y_{i}') for i in range(n_demands)] + +# Add cardinality constraint +solver.Add(sum(x) == k) + +# Add coverage constraints using precomputed N_i +for i in range(n_demands): + covering_sites = [x[j] for j in N[i]] + if covering_sites: + solver.Add(y[i] <= sum(covering_sites)) + else: + solver.Add(y[i] == 0) + +# Set objective +objective = solver.Objective() +for i in range(n_demands): + objective.SetCoefficient(y[i], weights[i]) +objective.SetMaximization() + +# solve with status / termination checks +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + # Extract solution + selected = [j for j in range(n_sites) if x[j].solution_value() > 0.5] + covered = [i for i in range(n_demands) if y[i].solution_value() > 0.5] + # ... verification and output +``` + +### Common Pitfalls +- Not checking solver status before extracting solution values, which can cause errors. +- Using a loose optimality gap, which may return suboptimal solutions for the weighted coverage objective. +- Failing to verify the solution logic, potentially accepting results that violate the coverage distance constraint due to solver tolerances. + +# Workflow 2 (Implicit Coverage Constraints) + +## Modeling stage + +### Strategy Overview +Embed the coverage condition directly within the model constraints using a binary parameter, avoiding the need for explicit precomputation of sets in the modeling language. This can be more natural in some algebraic modeling systems and keeps the distance logic inside the model. + +### Step 1 - Define Sets and Parameters +- Define `I` as the set of demand points and `J` as the set of candidate sites. +- Define `w_i` as the weight for covering demand point `i`. +- Define a binary parameter `a_{ij}` where `a_{ij} = 1` if distance `d_{ij} <= R`, else `0`. +- Define `k` as the cardinality limit for facility selection. + +### Step 2 - Create Decision Variables +- Create binary variable `x_j` for each candidate site `j` in `J`. `x_j = 1` if site `j` is selected. +- Create binary variable `y_i` for each demand point `i` in `I`. `y_i = 1` if demand point `i` is covered. + +### Step 3 - Formulate Constraints +- Add cardinality constraint: `sum_{j in J} x_j = k`. +- For each demand point `i`, add coverage activation constraint: `y_i <= sum_{j in J} a_{ij} * x_j`. This sums over all sites, but `a_{ij}` filters to only those within range. + +### Step 4 - Define Objective +- Maximize total weighted coverage: `maximize sum_{i in I} w_i * y_i`. + +### Formulation Template +```json +{ + "sets": [ + "I: demand_points", + "J: candidate_sites" + ], + "parameters": [ + "w[i in I]: weight of demand point i", + "a[i in I][j in J]: binary, 1 if site j covers point i", + "k: number of facilities to select" + ], + "decision_variables": [ + "x[j in J]: binary, 1 if site j selected", + "y[i in I]: binary, 1 if demand point i covered" + ], + "objective": { + "sense": "max", + "expression": "sum(i in I) w[i] * y[i]" + }, + "constraints": [ + "cardinality: sum(j in J) x[j] == k", + "coverage[i in I]: y[i] <= sum(j in J) a[i][j] * x[j]" + ] +} +``` + +### Common Pitfalls +- Creating a dense parameter `a_{ij}` for large problems, which can consume significant memory. Consider using sparse representation if possible. +- The constraint `y_i <= sum_{j in J} a_{ij} * x_j` is correct, but mistakenly writing `y_i == ...` would be incorrect, as coverage can be *activated* by the sum but not *forced*. +- Not recognizing that the parameter `a_{ij}` must be precomputed from distances `d_{ij}` and radius `R` before model instantiation. + +## Solving stage + +### Strategy Overview +Use a modeling library (e.g., PuLP, OR-Tools) with a focus on clear model definition using implicit summation. Include post-solution validation through brute-force enumeration for small instances to verify optimality and model correctness. + +### Step 1 - Model Construction with Implicit Constraints +- Use a modeling library to declare variables, objective, and constraints directly as per the algebraic formulation. +- The coverage constraint `y_i <= sum(a_ij * x_j)` is added in a loop over all demand points `i`. + +### Step 2 - Solver Configuration for Exact Solution +- Select a MIP solver (e.g., CBC via PuLP). +- Set emphasis on optimality (e.g., `gapRel=0.0`). +- Set a reasonable time limit. + +### Step 3 - Solve and Process Results +- Invoke the solver. +- Check the solution status. If optimal, extract the list of selected facilities and covered demand points. + +### Step 4 - Validation and Sanity Checking +- For small-scale problems (e.g., where the number of combinations `choose(|J|, k)` is manageable, e.g., ≤ 1000), implement brute-force enumeration to verify the solver found the true optimum. +- Programmatically verify that each covered demand point has at least one selected facility `j` where `a_{ij} = 1`. +- Report any discrepancies between the solver solution and the validation checks. + +### Step 5 - Generate Analysis Output +- Output the optimal facility selection and the corresponding covered demand points with their weights. +- Include validation results and, for small instances, the objective value from exhaustive search. + +### Code Usage +```python +# build model from formulation +import pulp +prob = pulp.LpProblem('MCLP', pulp.LpMaximize) + +# Variables +x = {j: pulp.LpVariable(f'x_{j}', cat='Binary') for j in J} +y = {i: pulp.LpVariable(f'y_{i}', cat='Binary') for i in I} + +# Objective +prob += pulp.lpSum(w[i] * y[i] for i in I) + +# Constraints +prob += pulp.lpSum(x[j] for j in J) == k # Cardinality +for i in I: + prob += y[i] <= pulp.lpSum(a[i][j] * x[j] for j in J) # Coverage + +# solve with status / termination checks +solver = pulp.PULP_CBC_CMD(timeLimit=[TIME_LIMIT], gapRel=0.0, msg=True) +prob.solve(solver) + +status = pulp.LpStatus[prob.status] +if status == 'Optimal': + selected = [j for j in J if pulp.value(x[j]) > 0.5] + covered = [i for i in I if pulp.value(y[i]) > 0.5] + # ... validation and output +``` + +### Common Pitfalls +- Using `pulp.lpSum` incorrectly inside constraints, leading to silent model building errors. +- Not setting `gapRel=0.0`, which may cause the solver to stop early with a suboptimal solution. +- Assuming the model is correct without validation, especially for the first run with new data. The brute-force check is crucial for debugging. diff --git a/skills/optskills/skill_library/maximum_cardinality_bipartite_matching.md b/skills/optskills/skill_library/maximum_cardinality_bipartite_matching.md new file mode 100644 index 0000000..d862ea3 --- /dev/null +++ b/skills/optskills/skill_library/maximum_cardinality_bipartite_matching.md @@ -0,0 +1,240 @@ +--- +name: Maximum Cardinality Bipartite Matching +description: | + Model and solve one-to-one assignment problems between two disjoint sets with preference compatibility, aiming to maximize the total number of matches. +--- + +# Workflow 1 (Network Flow via Max-Flow) + +## Modeling stage + +### Strategy Overview +Transform the bipartite matching problem into a maximum flow problem on a directed network. The flow value directly represents the number of matches, and unit capacities enforce one-to-one constraints. + +### Step 1 - Identify Sets and Compatibility +- Define two disjoint sets: `LEFT_SET` (size `m`) and `RIGHT_SET` (size `n`). +- Define a binary compatibility parameter `ALLOWED_PAIRS` as a list of tuples `(i, j)` where `i ∈ LEFT_SET` and `j ∈ RIGHT_SET`. Use zero-based indices consistent with the problem description. +- Elements with no feasible edges are automatically excluded from the matching. + +### Step 2 - Construct Flow Network +- Create a source node (index `0`) and a sink node (index `m + n + 1`). +- Add arcs from source to each left node `i` (index `1 + i`) with capacity `1`. +- For each allowed pair `(i, j)`, add an arc from left node `i` to right node `j` (index `1 + m + j`) with capacity `1`. +- Add arcs from each right node `j` to the sink with capacity `1`. + +### Step 3 - Define Objective Implicitly +- The objective to maximize the number of matches is equivalent to maximizing the total flow from source to sink. No explicit objective function is needed in the flow model. + +### Formulation Template +```json +{ + "sets": ["LEFT_SET", "RIGHT_SET"], + "parameters": ["ALLOWED_PAIRS"], + "decision_variables": ["flow_arcs (implicit via solver)"], + "objective": { + "sense": "max", + "expression": "maximize total flow from source to sink" + }, + "constraints": [ + "source → LEFT_SET arcs: capacity = 1", + "LEFT_SET → RIGHT_SET arcs: capacity = 1 for ALLOWED_PAIRS", + "RIGHT_SET → sink arcs: capacity = 1", + "flow conservation at all nodes except source and sink" + ] +} +``` + +### Common Pitfalls +- Incorrect node indexing leading to arc misplacement. Use a systematic offset scheme (e.g., source=0, left nodes=1..m, right nodes=m+1..m+n, sink=m+n+1). +- Adding arcs for non-allowed pairs, which incorrectly expands the feasible region. +- Forgetting to set arc capacities to 1, which violates the one-to-one matching constraint. + +## Solving stage + +### Strategy Overview +Use a dedicated max-flow algorithm (e.g., OR-Tools SimpleMaxFlow) to find the maximum flow, then extract assignments by identifying arcs with positive flow between the two sets. + +### Step 1 - Initialize Max-Flow Solver +- Instantiate the max-flow solver (e.g., `SimpleMaxFlow()`). +- Define node indices according to the predetermined offset scheme. + +### Step 2 - Build Network Programmatically +- Add source-to-left arcs with `add_arc_with_capacity(source, left_node, 1)`. +- Add left-to-right arcs for each `(i, j)` in `ALLOWED_PAIRS`. +- Add right-to-sink arcs with `add_arc_with_capacity(right_node, sink, 1)`. + +### Step 3 - Solve and Check Status +- Call the solver's `solve(source, sink)` method. +- Verify the status is `OPTIMAL` before proceeding. If not `OPTIMAL`, return `None` and an empty list. + +### Step 4 - Extract and Map Assignments +- Iterate over all arcs. +- Filter arcs where `flow(arc) > 0` and the tail is a left node and the head is a right node (excluding the sink). +- Map the internal node indices back to the original set indices using the offset scheme. + +### Step 5 - Validate Solution Correctness +- Verify all assignments are in `ALLOWED_PAIRS`. +- Confirm each left node appears at most once and each right node appears at most once. +- The maximum number of matches is bounded by `min(m, n)`; achieving full utilization of the smaller set indicates optimality. + +### Reusable Code Template +```python +from ortools.graph.python import max_flow + +def max_bipartite_matching(left_size, right_size, allowed_pairs): + smf = max_flow.SimpleMaxFlow() + source = 0 + sink = left_size + right_size + 1 + left_offset = 1 + right_offset = left_size + 1 + + # source to left nodes + for i in range(left_size): + smf.add_arc_with_capacity(source, left_offset + i, 1) + + # left to right (allowed pairs) + for i, j in allowed_pairs: + smf.add_arc_with_capacity(left_offset + i, right_offset + j, 1) + + # right nodes to sink + for j in range(right_size): + smf.add_arc_with_capacity(right_offset + j, sink, 1) + + status = smf.solve(source, sink) + if status != smf.OPTIMAL: + return None, [] + + max_matches = smf.optimal_flow() + assignments = [] + for arc in range(smf.num_arcs()): + if smf.flow(arc) > 0: + tail = smf.tail(arc) + head = smf.head(arc) + if left_offset <= tail < right_offset and head >= right_offset and head != sink: + i = tail - left_offset + j = head - right_offset + assignments.append((i, j)) + return max_matches, assignments +``` + +### Common Pitfalls +- Not checking solver status, leading to extraction errors on infeasible or non-optimal solves. +- Misinterpreting arc indices when extracting matches; always verify tail/head belong to the correct node partitions. +- Assuming a perfect matching is always possible; the optimal flow may be less than `min(left_size, right_size)`. + +# Workflow 2 (Binary Integer Programming via CP-SAT/MILP) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Binary Integer Program (BIP) with explicit binary assignment variables, linear constraints for one-to-one matching and preference compatibility, and a linear objective to maximize the sum of assignments. + +### Step 1 - Define Sets and Parameters +- Define sets `SET_A` (size `m`) and `SET_B` (size `n`). +- Define a binary parameter `COMPATIBLE[i][j]` (1 if assignment is allowed, 0 otherwise). + +### Step 2 - Create Binary Assignment Variables +- Create a binary decision variable `x[i][j]` for each `i ∈ SET_A`, `j ∈ SET_B` where `COMPATIBLE[i][j] == 1`. +- The variable equals 1 if element `i` is matched to element `j`. +- For sparse compatibility, only create variables for allowed pairs to avoid model bloat. + +### Step 3 - Add One-to-One Matching Constraints +- For each `i ∈ SET_A`: `sum(x[i][j] for j in SET_B) <= 1`. +- For each `j ∈ SET_B`: `sum(x[i][j] for i ∈ SET_A) <= 1`. + +### Step 4 - Enforce Preference Compatibility +- For each pair `(i, j)` where `COMPATIBLE[i][j] == 0`, either omit the variable or fix `x[i][j] = 0`. +- If variables are created for all pairs, add constraints: `x[i][j] <= COMPATIBLE[i][j]`. + +### Step 5 - Set Maximization Objective +- Define objective: `maximize sum(x[i][j] for i ∈ SET_A for j ∈ SET_B)`. + +### Formulation Template +```json +{ + "sets": ["SET_A", "SET_B"], + "parameters": ["COMPATIBLE (binary matrix)"], + "decision_variables": ["x[i][j] ∈ {0,1}"], + "objective": { + "sense": "max", + "expression": "sum_{i ∈ SET_A, j ∈ SET_B} x[i][j]" + }, + "constraints": [ + "sum_{j ∈ SET_B} x[i][j] <= 1, ∀ i ∈ SET_A", + "sum_{i ∈ SET_A} x[i][j] <= 1, ∀ j ∈ SET_B", + "x[i][j] <= COMPATIBLE[i][j], ∀ i ∈ SET_A, j ∈ SET_B" + ] +} +``` + +### Common Pitfalls +- Creating variables for all possible pairs when compatibility is sparse, leading to unnecessary model bloat. +- Using equality (`=`) instead of inequality (`<=`) in one-to-one constraints, which forces perfect matching and may cause infeasibility. +- Forgetting to enforce compatibility constraints, allowing invalid assignments. + +## Solving stage + +### Strategy Overview +Use a BIP-capable solver (e.g., OR-Tools CP-SAT or a MILP solver) to find an optimal assignment. Configure for exact solution and extract results by checking variable values. + +### Step 1 - Instantiate Solver and Model +- Create a model instance (e.g., `CpModel()` or `ConcreteModel()`). +- Define sets and parameters within the modeling framework. + +### Step 2 - Add Variables and Constraints +- Create binary variables only for allowed pairs. +- Add the one-to-one and compatibility constraints using the model's API. + +### Step 3 - Set Objective and Configure Solver +- Set the maximization objective. +- Configure solver parameters: set a time limit `[TIME_LIMIT]`, enable parallel search, set optimality gap to zero, and optionally set a random seed for reproducibility. + +### Step 4 - Solve and Validate Status +- Call the solver. +- Check termination status (`OPTIMAL` or `FEASIBLE` for CP-SAT; `ok` and `optimal`/`feasible` for Pyomo). + +### Step 5 - Extract Solution +- Iterate over all `(i, j)` pairs for which a variable was created. +- If the variable value is 1 (or > 0.5 for continuous solvers), record `(i, j)` as an assignment. + +### Code Usage +```python +# Example using OR-Tools CP-SAT +from ortools.sat.python import cp_model + +def solve_bipartite_matching_cp_sat(m, n, compatible): + model = cp_model.CpModel() + x = {} + for i in range(m): + for j in range(n): + if compatible[i][j]: + x[(i, j)] = model.NewBoolVar(f"x_{i}_{j}") + + # One-to-one constraints + for i in range(m): + model.Add(sum(x.get((i, j), 0) for j in range(n)) <= 1) + for j in range(n): + model.Add(sum(x.get((i, j), 0) for i in range(m)) <= 1) + + # Objective + model.Maximize(sum(x.values())) + + # Solve + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = 30.0 + solver.parameters.num_search_workers = 8 + solver.parameters.random_seed = 42 + status = solver.Solve(model) + + if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + assignments = [(i, j) for (i, j), var in x.items() if solver.Value(var) == 1] + objective_value = solver.ObjectiveValue() + return objective_value, assignments + else: + return None, [] +``` + +### Common Pitfalls +- Not handling sparse variable creation correctly, leading to KeyError during constraint building. +- Misinterpreting solver status codes between different solver APIs. +- Extracting assignments without verifying the solution status first, potentially reading invalid values. diff --git a/skills/optskills/skill_library/maximum_cardinality_matching_via_binary_edge_assignment.md b/skills/optskills/skill_library/maximum_cardinality_matching_via_binary_edge_assignment.md new file mode 100644 index 0000000..0c6d509 --- /dev/null +++ b/skills/optskills/skill_library/maximum_cardinality_matching_via_binary_edge_assignment.md @@ -0,0 +1,234 @@ +--- +name: Maximum Cardinality Matching via Binary Edge Assignment +description: | + Models and solves the maximum cardinality matching problem on a general graph using binary edge assignment variables, degree constraints, and edge set restrictions, with two distinct solver backends (CP-SAT and MILP). +--- + +# Workflow 1 (CP-SAT with OR-Tools) + +## Modeling stage + +### Strategy Overview +Formulate the maximum cardinality matching as a constraint satisfaction problem using binary variables for each allowed edge, node-degree constraints, and a maximization objective. This approach is suitable for graphs up to moderate size (thousands of edges) and guarantees exact optimality. + +### Step 1 - Define Edge Variables +- For each edge `(u, v)` in the allowed edge set `E`, create a binary variable `x[(u,v)]` using `model.NewBoolVar()`. +- Use a dictionary keyed by the edge tuple to store variables for easy access during constraint building. + +### Step 2 - Enforce Node Degree Constraints +- For each node in the graph, collect all variables corresponding to edges incident to that node. +- Add a constraint `sum(incident_vars) <= 1` for each node using `model.Add()`. + +### Step 3 - Set Objective +- Maximize the total number of selected edges by calling `model.Maximize(sum(x.values()))`. + +### Formulation Template +```json +{ + "sets": ["E: allowed edges (list of tuples)"], + "parameters": ["N: set of nodes derived from edges"], + "decision_variables": [ + "x_e ∈ {0,1} for each e ∈ E" + ], + "objective": { + "sense": "max", + "expression": "sum_{e ∈ E} x_e" + }, + "constraints": [ + "For each v ∈ N: sum_{e incident to v} x_e ≤ 1" + ] +} +``` + +### Common Pitfalls +- Do not create variables for edges not in the allowed set; this would violate the edge set restriction. +- Do not forget to collect all incident edges for each node, including both endpoints of each edge. + +## Solving stage + +### Strategy Overview +Use OR-Tools' CP-SAT solver with appropriate parameters for performance. Parse the solver status and extract selected edges only on successful termination. + +### Step 1 - Configure Solver +- Instantiate `cp_model.CpSolver()`. +- Set parameters: `max_time_in_seconds` (e.g., `[TIME_LIMIT]`), `num_search_workers` (e.g., 8), `random_seed` (e.g., 42), and `relative_gap_limit` (e.g., 0.0 for exact optimality). + +### Step 2 - Solve and Check Status +- Call `solver.Solve(model)`. +- Check if status is `cp_model.OPTIMAL` or `cp_model.FEASIBLE`. If not, report failure with the solver status code. + +### Step 3 - Extract and Verify Solution +- For each variable, use `solver.Value(x_e)` to check if it is 1. +- Build a list of selected edges. +- Verify that no node appears in more than one selected edge to confirm the matching property. + +### Step 4 - Verify Optimality (Optional) +- To confirm optimality, add a constraint `sum(x.values()) >= k+1` where `k` is the objective value found, and check for infeasibility. This proves no larger matching exists. + +### Code Usage +```python +from ortools.sat.python import cp_model +import json + +# edges: list of tuples, e.g., [(0,1), (0,4), ...] +edges = [...] + +model = cp_model.CpModel() +x = {} +for (u, v) in edges: + x[(u, v)] = model.NewBoolVar(f'x_{u}_{v}') + +# Node degree constraints +nodes = set() +for u, v in edges: + nodes.add(u) + nodes.add(v) +for v in nodes: + incident = [x[e] for e in edges if v in e] + if incident: + model.Add(sum(incident) <= 1) + +# Objective +model.Maximize(sum(x.values())) + +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 +solver.parameters.relative_gap_limit = 0.0 + +status = solver.Solve(model) + +if status == cp_model.OPTIMAL or status == cp_model.FEASIBLE: + selected = [e for e in edges if solver.Value(x[e]) > 0.5] + payload = { + "status": "optimal" if status == cp_model.OPTIMAL else "feasible", + "objective": int(solver.ObjectiveValue()), + "selected_edges": selected, + } + print(f"RESULT_JSON:{json.dumps(payload)}") +else: + payload = { + "status": "failed", + "reason": "solver_error", + "solver_status": str(status), + } + print(f"RESULT_JSON:{json.dumps(payload)}") +``` + +### Common Pitfalls +- Do not trust non-zero return codes or infeasible/unknown statuses; always check the status enum. +- Do not output pseudo-numeric answers when execution fails; always use the structured JSON output. + +# Workflow 2 (MILP with Pyomo and CBC) + +## Modeling stage + +### Strategy Overview +Formulate the maximum cardinality matching as a mixed-integer linear program using Pyomo's `ConcreteModel`. This approach is suitable for larger graphs and provides access to advanced MILP solver features like warm starts and cut generation. + +### Step 1 - Define Sets and Variables +- Create a Pyomo `Set` `m.E` indexed by edge indices (0 to len(edges)-1). +- Define binary variables `m.x[e]` for each edge index using `domain=pyo.Binary`. + +### Step 2 - Build Node Degree Constraints +- Extract all unique nodes from the edge list. +- For each node, find incident edge indices and add a constraint `sum(m.x[e] for e in incident) <= 1` using `m.degree_con.add()`. + +### Step 3 - Set Objective +- Maximize the sum of all edge variables: `m.obj = pyo.Objective(expr=sum(m.x[e] for e in m.E), sense=pyo.maximize)`. + +### Formulation Template +```json +{ + "sets": ["E: set of edge indices"], + "parameters": ["edges: list of tuples mapping index to (u,v)"], + "decision_variables": [ + "x_e ∈ {0,1} for each e ∈ E" + ], + "objective": { + "sense": "max", + "expression": "sum_{e ∈ E} x_e" + }, + "constraints": [ + "For each v ∈ N: sum_{e incident to v} x_e ≤ 1" + ] +} +``` + +### Common Pitfalls +- Do not use Pyomo's `Set` with tuples directly for indexing; use integer indices and maintain a separate mapping list. +- Do not forget to handle nodes with no incident edges (isolated nodes) gracefully. + +## Solving stage + +### Strategy Overview +Use the CBC solver via Pyomo's `SolverFactory`. Configure solver options for time limits and optimality gap. Parse results with explicit status and termination condition checks. + +### Step 1 - Configure Solver +- Instantiate `pyo.SolverFactory("cbc")`. +- Set options: `seconds` for time limit (e.g., `[TIME_LIMIT]`) and `ratio` for MIP gap (e.g., 0.0 for optimality). + +### Step 2 - Solve and Check Status +- Call `solver.solve(m, tee=False)`. +- Check `results.solver.status` equals `SolverStatus.ok` and `results.solver.termination_condition` is either `TerminationCondition.optimal` or `TerminationCondition.feasible`. + +### Step 3 - Extract and Output Solution +- Iterate over edge indices, check `pyo.value(m.x[e]) > 0.5` to identify selected edges. +- Build a JSON payload with status, objective (cast to int), and selected edges list. +- On failure, output a JSON payload with failure reason and solver details. + +### Code Usage +```python +import json +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# edges: list of tuples, e.g., [(0,1), (0,4), ...] +edges = [...] + +m = pyo.ConcreteModel() +m.E = pyo.Set(initialize=range(len(edges))) +m.x = pyo.Var(m.E, domain=pyo.Binary) + +m.obj = pyo.Objective(expr=sum(m.x[e] for e in m.E), sense=pyo.maximize) + +nodes = list(set([v for e in edges for v in e])) +m.degree_con = pyo.ConstraintList() +for v in nodes: + incident = [e for e, (u, w) in enumerate(edges) if u == v or w == v] + if incident: + m.degree_con.add(expr=sum(m.x[e] for e in incident) <= 1) + +solver = pyo.SolverFactory("cbc") +solver.options["seconds"] = [TIME_LIMIT] +solver.options["ratio"] = 0.0 +results = solver.solve(m, tee=False) + +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + selected = [] + for e in m.E: + if pyo.value(m.x[e]) > 0.5: + selected.append(edges[e]) + payload = { + "status": "optimal" if term == TerminationCondition.optimal else "feasible", + "objective": int(pyo.value(m.obj)), + "selected_edges": selected, + } + print(f"RESULT_JSON:{json.dumps(payload)}") +else: + payload = { + "status": "failed", + "reason": "infeasible_or_error", + "solver_status": str(status), + "termination_condition": str(term), + } + print(f"RESULT_JSON:{json.dumps(payload)}") +``` + +### Common Pitfalls +- Do not skip validation of the matching solution; always verify that no node appears in more than one selected edge. +- Do not attempt to formulate a general graph matching as a max-flow problem without node-splitting constraints that enforce each original node appears at most once. This only works for bipartite graphs. diff --git a/skills/optskills/skill_library/maximum_flow_network_solver.md b/skills/optskills/skill_library/maximum_flow_network_solver.md new file mode 100644 index 0000000..c87017b --- /dev/null +++ b/skills/optskills/skill_library/maximum_flow_network_solver.md @@ -0,0 +1,291 @@ +--- +name: Maximum Flow Network Solver +description: | + A skill for modeling and solving maximum flow problems on capacitated directed networks, providing workflows for specialized graph algorithms and general linear programming formulations. + +--- +# Workflow 1 (Specialized Graph Algorithm) + +## Modeling stage + +### Strategy Overview +Model the problem as a directed graph where arcs have capacities. Use a specialized max-flow algorithm (e.g., Edmonds-Karp, Push-Relabel) which implicitly handles variables and constraints, focusing on graph construction and source-sink definition. + +### Step 1 - Define Network Topology +- Identify the set of nodes and the set of directed arcs. +- For each arc, define its tail (start node), head (end node), and capacity. + +### Step 2 - Map Data to Solver Input +- Prepare three parallel lists: `start_nodes`, `end_nodes`, and `capacities`. +- Ensure node indices are unique consecutive integers, typically starting from 0. + +### Step 3 - Configure Source and Sink +- Designate a single source node (where flow originates) and a single sink node (where flow terminates). + +### Formulation Template +```json +{ + "sets": [ + "NODES", + "ARCS" + ], + "parameters": [ + {"name": "capacity", "index": "ARCS"} + ], + "decision_variables": [], + "objective": { + "sense": "max", + "expression": "total_flow_from_source_to_sink" + }, + "constraints": [ + "capacity_constraint (implicit in algorithm)", + "flow_conservation (implicit in algorithm)" + ] +} +``` + +### Common Pitfalls +- Using non-integer or non-consecutive node identifiers, causing solver errors. +- Incorrectly ordering the `start_nodes` and `end_nodes` lists, reversing arc direction. +- Forgetting to verify the solver's status is `OPTIMAL` before extracting results. + +## Solving stage + +### Strategy Overview +Utilize a dedicated max-flow solver API. The solving process involves adding arcs, invoking the algorithm, and extracting the flow value and distribution. + +### Step 1 - Initialize Solver +- Instantiate the specialized max-flow solver object (e.g., `SimpleMaxFlow`). + +### Step 2 - Build the Graph +- Iterate through all arcs, adding each to the solver with its capacity using `add_arc_with_capacity`. + +### Step 3 - Solve and Check Status +- Call the solver's `solve(source, sink)` method. +- Check the returned status equals `OPTIMAL`; handle `INFEASIBLE` or other statuses appropriately. + +### Step 4 - Extract and Verify Solution +- Retrieve the optimal flow value via `optimal_flow()`. +- Optionally, iterate through arcs to get the flow on each arc for verification or reporting. + +### Code Usage +```python +# build model from formulation +solver = max_flow.SimpleMaxFlow() +for i in range(num_arcs): + solver.add_arc_with_capacity(start_nodes[i], end_nodes[i], capacities[i]) + +# solve with status / termination checks +status = solver.solve(source, sink) +if status == solver.OPTIMAL: + max_flow_value = solver.optimal_flow() + # Optional: Retrieve per-arc flows + for i in range(solver.num_arcs()): + flow = solver.flow(i) +else: + # Handle failure +``` + +### Common Pitfalls +- Assuming the solver found a feasible solution without checking the status. +- Misinterpreting arc indices when retrieving per-arc flows after the solve. +- Not accounting for the possibility of multiple optimal flow distributions. + +# Workflow 2 (Linear Programming Formulation) + +## Modeling stage + +### Strategy Overview +Formulate the maximum flow problem as a Linear Program (LP). Explicitly define continuous flow variables for each arc, subject to capacity bounds and flow conservation constraints at all nodes except the source and sink. + +### Step 1 - Define Variables and Parameters +- Create one non-negative continuous variable for each directed arc, representing the flow on that arc. Set its upper bound equal to the arc's capacity. +- Use descriptive naming (e.g., `flow[(i,j)]`) for clarity and retrieval. + +### Step 2 - Enforce Capacity Constraints +- For each arc, the flow variable is automatically bounded by `[0, capacity]` when created with these bounds. + +### Step 3 - Enforce Flow Conservation +- For each node that is neither the source nor the sink, create a constraint: the sum of flows on incoming arcs equals the sum of flows on outgoing arcs. +- Programmatically build inflow and outflow lists by scanning all arcs. + +### Step 4 - Formulate the Objective +- Define the objective to maximize the total flow into the sink node (sum of flows on all arcs terminating at the sink). + +### Formulation Template +```json +{ + "sets": [ + "NODES", + "ARCS" + ], + "parameters": [ + {"name": "capacity", "index": "ARCS"} + ], + "decision_variables": [ + {"name": "flow", "index": "ARCS", "type": "continuous", "lb": 0} + ], + "objective": { + "sense": "max", + "expression": "sum(flow[(i, sink)] for all arcs (i, sink) into sink)" + }, + "constraints": [ + "flow[a] <= capacity[a] for all a in ARCS (enforced by variable bounds)", + "sum(flow[(i, n)]) == sum(flow[(n, j)]) for all n in NODES where n != source, sink" + ] +} +``` + +### Common Pitfalls +- Incorrectly indexing nodes when constructing flow conservation constraints, leading to unbalanced equations. +- Adding unnecessary constraints for the source or sink nodes, which are handled implicitly by the objective. +- Formulating the objective on the wrong set of arcs (e.g., maximizing flow out of the sink). + +## Solving stage + +### Strategy Overview +Use a general-purpose LP solver. The process involves building the model with explicit variables and constraints, solving it, and then verifying the solution satisfies the network flow properties. + +### Step 1 - Initialize Solver and Model +- Create a solver instance capable of handling linear programming (e.g., `GLOP`, `HiGHS`). + +### Step 2 - Build the LP Model +- Create flow variables with bounds `(0, capacity)` in one pass, storing them in a dictionary keyed by arc. +- For each intermediate node, dynamically build inflow and outflow lists from the arc dictionary and add the conservation constraint. +- Set the objective to maximize the sum of flow variables on arcs ending at the sink. + +### Step 3 - Solve and Check Termination +- Execute the solver. +- Verify the termination condition is `OPTIMAL`. + +### Step 4 - Extract and Validate Solution +- Retrieve the objective value. +- Optionally, extract the flow values for each arc. +- Perform a post-solve validation: check that flow conservation holds at intermediate nodes and that no arc exceeds its capacity within a small tolerance (e.g., `1e-6`). + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +solver = pywraplp.Solver.CreateSolver('GLOP') +flow_vars = {} +for (i, j), cap in capacities.items(): + flow_vars[(i, j)] = solver.NumVar(0, cap, f'flow_{i}_{j}') + +for node in intermediate_nodes: + inflow = [flow_vars[(i, j)] for (i, j) in arcs if j == node] + outflow = [flow_vars[(i, j)] for (i, j) in arcs if i == node] + if inflow or outflow: + solver.Add(sum(inflow) == sum(outflow)) + +sink_inflow = [flow_vars[(i, j)] for (i, j) in arcs if j == sink] +solver.Maximize(sum(sink_inflow)) + +status = solver.Solve() +if status == solver.OPTIMAL: + max_flow_value = solver.Objective().Value() + # Optional validation + for (i, j), var in flow_vars.items(): + flow_val = var.solution_value() + # Check capacity + assert flow_val <= capacities[(i, j)] + 1e-6 +else: + # Handle failure +``` + +### Common Pitfalls +- Using an LP solver without enabling presolve or appropriate scaling for large networks. +- Not verifying that the extracted solution satisfies all constraints within a small tolerance due to numerical precision. +- Confusing the solver's status codes (e.g., `OPTIMAL` vs. `FEASIBLE`). + +# Workflow 3 (Edmonds-Karp Algorithm Implementation) + +## Modeling stage + +### Strategy Overview +Recognize maximum flow problems by identifying source-sink networks with capacity constraints on arcs and flow conservation at intermediate nodes. The objective is to maximize total flow from source to sink. Map network elements to a graph structure: nodes as vertices and directed links as arcs with capacity parameters. + +### Step 1 - Define Graph Representation +- Store the graph as adjacency lists of edge indices for efficient BFS traversal. +- For each directed edge, store its destination, capacity, and a pointer to its reverse edge in the residual graph. + +### Step 2 - Implement Residual Network +- When adding an edge, simultaneously add a reverse edge with zero initial capacity to enable flow reduction. + +## Solving stage + +### Strategy Overview +Implement the Edmonds-Karp algorithm (BFS-based Ford-Fulkerson) for reliability when external libraries are unavailable. Structure the algorithm to find augmenting paths in the residual network and saturate them until no path exists. + +### Step 1 - Initialize Algorithm Structures +- Create a `MaxFlow` class with methods for adding edges, BFS for finding augmenting paths, and the main Edmonds-Karp loop. +- Use efficient data structures: store edges in a flat list with indices, maintain adjacency lists for quick neighbor access, and use a parent array to reconstruct augmenting paths. + +### Step 2 - Execute Edmonds-Karp Loop +- While BFS finds a path from source to sink in the residual graph: + - Determine the bottleneck capacity along the path. + - Augment flow along the path and its reverse edges. +- The loop terminates when BFS finds no augmenting path. + +### Step 3 - Validate with LP Formulation +- Cross-check algorithm results using the LP formulation from Workflow 2 to ensure correctness. +- After solving, examine the graph structure to identify limiting arcs (e.g., minimum cut edges) and verify the solution makes intuitive sense. + +### Step 4 - Handle Edge Cases +- Ensure the algorithm works for networks with multiple paths, cycles, and disconnected components by properly implementing BFS termination conditions. +- Analyze network bottlenecks by identifying the minimum cut after the algorithm completes. + +### Code Usage +```python +class MaxFlow: + def __init__(self, n): + self.n = n + self.adj = [[] for _ in range(n)] + self.edges = [] # [to, cap, flow] + + def add_edge(self, u, v, cap): + self.edges.append([v, cap, 0]) + self.adj[u].append(len(self.edges) - 1) + self.edges.append([u, 0, 0]) + self.adj[v].append(len(self.edges) - 1) + + def bfs(self, s, t, parent): + visited = [False] * self.n + queue = [s] + visited[s] = True + while queue: + u = queue.pop(0) + for idx in self.adj[u]: + v, cap, flow = self.edges[idx] + if not visited[v] and cap - flow > 0: + visited[v] = True + parent[v] = (u, idx) + if v == t: + return True + queue.append(v) + return False + + def edmonds_karp(self, s, t): + max_flow = 0 + parent = [-1] * self.n + while self.bfs(s, t, parent): + path_flow = float('inf') + v = t + while v != s: + u, idx = parent[v] + path_flow = min(path_flow, self.edges[idx][1] - self.edges[idx][2]) + v = u + v = t + while v != s: + u, idx = parent[v] + self.edges[idx][2] += path_flow + self.edges[idx ^ 1][2] -= path_flow + v = u + max_flow += path_flow + return max_flow + +# Usage +mf = MaxFlow(num_nodes) +for u, v, cap in arcs: + mf.add_edge(u, v, cap) +max_flow_value = mf.edmonds_karp(source, sink) diff --git a/skills/optskills/skill_library/maximum_leaf_spanning_tree.md b/skills/optskills/skill_library/maximum_leaf_spanning_tree.md new file mode 100644 index 0000000..5a3128b --- /dev/null +++ b/skills/optskills/skill_library/maximum_leaf_spanning_tree.md @@ -0,0 +1,346 @@ +--- +name: Maximum Leaf Spanning Tree +description: | + Models and solves the problem of finding a spanning tree that maximizes the number of leaf nodes, using either a CP-SAT or MILP solver. + +--- +# Workflow 1 (CP-SAT with Flow Connectivity) + +## Modeling stage + +### Strategy Overview +Model the problem using binary edge selection variables, integer degree variables, and binary leaf indicators. Enforce connectivity and acyclicity via a single-commodity flow formulation. Use CP-SAT's implication constraints for exact leaf detection. + +### Step 1 - Define Edge Selection Variables +- Define a binary variable `x[e]` for each eligible edge `e` in the set `E`. +- `x[e]` is 1 if the edge is selected in the spanning tree, 0 otherwise. + +### Step 2 - Enforce Spanning Tree Cardinality +- Add a constraint that exactly `n-1` edges are selected: `sum(x[e] for e in E) == n-1`, where `n` is the number of nodes. + +### Step 3 - Compute Node Degrees +- For each node `i`, create an integer variable `deg[i]` with domain `[0, n-1]`. +- Precompute the list of incident edges for each node. +- Add constraints: `deg[i] == sum(x[e] for e incident to i)`. + +### Step 4 - Model Leaf Node Indicators +- For each node `i`, create a binary variable `leaf[i]`. +- Use CP-SAT implication constraints to enforce logical equivalence `(deg[i] == 1) ⇔ (leaf[i] == 1)`: + - `model.Add(deg[i] == 1).OnlyEnforceIf(leaf[i])` + - `model.Add(deg[i] != 1).OnlyEnforceIf(leaf[i].Not())` + +### Step 5 - Enforce Connectivity and Acyclicity via Flow +- Choose a root node `r` (e.g., node 0). +- For each undirected edge `(u,v)`, create two directed flow variables `f[(u,v)]` and `f[(v,u)]` with domain `[0, n-1]`. +- Link flow to edge selection: `f[(u,v)] <= (n-1) * x[e]` and `f[(v,u)] <= (n-1) * x[e]`. +- Flow conservation constraints: + - For root `r`: `sum(f[(r,j)] for j in neighbors) - sum(f[(j,r)] for j in neighbors) == n-1` + - For each other node `i`: `sum(f[(j,i)] for j in neighbors) - sum(f[(i,j)] for j in neighbors) == 1` + +### Step 6 - Set Objective +- Maximize the total number of leaf nodes: `model.Maximize(sum(leaf[i] for i in nodes))` + +### Formulation Template +```json +{ + "sets": { + "N": "set of nodes, indexed 0..n-1", + "E": "set of eligible undirected edges (u,v)" + }, + "parameters": { + "n": "number of nodes", + "root": "chosen root node for flow (e.g., 0)" + }, + "decision_variables": { + "x[e]": "binary, 1 if edge e is selected", + "deg[i]": "integer [0, n-1], degree of node i", + "leaf[i]": "binary, 1 if node i is a leaf", + "f[(u,v)]": "integer [0, n-1], flow on directed arc (u,v)" + }, + "objective": { + "sense": "max", + "expression": "sum(leaf[i] for i in N)" + }, + "constraints": [ + "sum(x[e] for e in E) == n-1", + "deg[i] == sum(x[e] for e incident to i) for all i in N", + "deg[i] == 1 => leaf[i] == 1 for all i in N", + "deg[i] != 1 => leaf[i] == 0 for all i in N", + "f[(u,v)] <= (n-1) * x[(u,v)] for all (u,v) in directed arcs", + "f[(v,u)] <= (n-1) * x[(u,v)] for all (u,v) in directed arcs", + "flow conservation at root: outflow - inflow == n-1", + "flow conservation at non-root: inflow - outflow == 1" + ] +} +``` + +### Common Pitfalls +- Forgetting to create both directed flow variables for each undirected edge. +- Using incorrect flow conservation signs (root sends out, others receive). +- Not linking flow variables to edge selection, allowing flow on unselected edges. + +## Solving stage + +### Strategy Overview +Use OR-Tools CP-SAT solver to handle the binary and integer variables efficiently. Set a time limit and verify solution validity post-solve. + +### Step 1 - Initialize Solver and Set Parameters +- Create a `cp_model.CpSolver` instance. +- Set a time limit: `solver.parameters.max_time_in_seconds = [TIME_LIMIT]`. +- Optionally enable logging: `solver.parameters.log_search_progress = True`. + +### Step 2 - Solve the Model +- Call `status = solver.Solve(model)`. +- Check the status: `status == cp_model.OPTIMAL` or `status == cp_model.FEASIBLE`. + +### Step 3 - Extract and Verify Solution +- Extract objective value: `solver.ObjectiveValue()`. +- Extract selected edges: `[e for e in E if solver.Value(x[e]) > 0.5]`. +- **Verification checks**: + - Verify exactly `n-1` edges are selected. + - Verify connectivity using BFS/DFS from the root on selected edges. + - Verify acyclicity (automatically satisfied if connected with `n-1` edges). + - Compute degrees and confirm leaf nodes match `leaf` variables. +- If any check fails, treat the solution as invalid. + +### Step 4 - Output Results +- Print a structured result (e.g., JSON) with status, objective value, leaf list, selected edges, and degrees. + +### Code Usage +```python +from ortools.sat.python import cp_model + +def build_and_solve_max_leaf_spanning_tree(nodes, eligible_edges, root=0, time_limit=60): + model = cp_model.CpModel() + n = len(nodes) + edge_list = list(eligible_edges) + + # Decision variables + x = {e: model.NewBoolVar(f'x_{e[0]}_{e[1]}') for e in edge_list} + deg = {i: model.NewIntVar(0, n-1, f'deg_{i}') for i in nodes} + leaf = {i: model.NewBoolVar(f'leaf_{i}') for i in nodes} + + # Flow variables + f = {} + for (u,v) in edge_list: + f[(u,v)] = model.NewIntVar(0, n-1, f'f_{u}_{v}') + f[(v,u)] = model.NewIntVar(0, n-1, f'f_{v}_{u}') + + # Spanning tree cardinality + model.Add(sum(x[e] for e in edge_list) == n-1) + + # Degree constraints + incident_edges = {i: [] for i in nodes} + for e in edge_list: + u, v = e + incident_edges[u].append(x[e]) + incident_edges[v].append(x[e]) + for i in nodes: + model.Add(deg[i] == sum(incident_edges[i])) + + # Leaf detection + for i in nodes: + model.Add(deg[i] == 1).OnlyEnforceIf(leaf[i]) + model.Add(deg[i] != 1).OnlyEnforceIf(leaf[i].Not()) + + # Flow constraints + for (u,v) in edge_list: + model.Add(f[(u,v)] <= (n-1) * x[(u,v)]) + model.Add(f[(v,u)] <= (n-1) * x[(u,v)]) + + # Flow conservation + for i in nodes: + outflow = sum(f[(i,j)] for (a,b) in edge_list for (i2,j) in [(a,b),(b,a)] if i2 == i) + inflow = sum(f[(j,i)] for (a,b) in edge_list for (j,i2) in [(a,b),(b,a)] if i2 == i) + if i == root: + model.Add(outflow - inflow == n-1) + else: + model.Add(inflow - outflow == 1) + + # Objective + model.Maximize(sum(leaf[i] for i in nodes)) + + # Solve + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = time_limit + status = solver.Solve(model) + + # Result parsing + result = {"status": solver.StatusName(status), "objective": None, "edges": [], "leaves": []} + if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + result["objective"] = solver.ObjectiveValue() + result["edges"] = [e for e in edge_list if solver.Value(x[e]) > 0.5] + result["leaves"] = [i for i in nodes if solver.Value(leaf[i]) > 0.5] + return result +``` + +### Common Pitfalls +- Not checking solver status before reading variable values, leading to runtime errors. +- Using too large a flow domain (`n-1` is sufficient; larger values slow the solver). +- Forgetting that CP-SAT requires explicit `OnlyEnforceIf` for implications. + +# Workflow 2 (MILP with Linear Leaf Detection) + +## Modeling stage + +### Strategy Overview +Formulate as a Mixed Integer Linear Program (MILP) using binary edge and leaf variables. Enforce connectivity and acyclicity via single-commodity flow. Use linear constraints to link degree to leaf status, relying on the objective to enforce correctness. + +### Step 1 - Define Edge Selection Variables +- Define a binary variable `x[e]` for each eligible edge `e` in the set `E`. +- `x[e]` is 1 if the edge is selected, 0 otherwise. + +### Step 2 - Enforce Spanning Tree Cardinality +- Add constraint: `sum(x[e] for e in E) == n-1`. + +### Step 3 - Compute Node Degrees +- For each node `i`, define `deg[i]` as the sum of incident selected edges: `deg[i] = sum(x[e] for e incident to i)`. + +### Step 4 - Model Leaf Node Indicators with Linear Constraints +- For each node `i`, define a binary variable `y[i]` indicating if it is a leaf. +- Add constraints: + - `y[i] <= deg[i]` (leaf implies degree >= 1). + - `deg[i] <= 1 + (n-1) * (1 - y[i])` (if leaf, degree <= 1; otherwise, no restriction). +- Since the objective maximizes leaf count, the solver will set `y[i]=1` exactly when `deg[i]==1`. + +### Step 5 - Enforce Connectivity and Acyclicity via Flow +- Choose a root node `r`. +- For each undirected edge `(u,v)`, create two directed flow variables `f[(u,v)]` and `f[(v,u)]` with domain `[0, n-1]`. +- Link flow to edge selection: `f[(u,v)] <= (n-1) * x[e]` and `f[(v,u)] <= (n-1) * x[e]`. +- Flow conservation: + - Root: `outflow - inflow == n-1`. + - Non-root: `inflow - outflow == 1`. + +### Step 6 - Set Objective +- Maximize the sum of leaf indicators: `maximize sum(y[i] for i in nodes)`. + +### Formulation Template +```json +{ + "sets": { + "N": "set of nodes, indexed 0..n-1", + "E": "set of eligible undirected edges (u,v)" + }, + "parameters": { + "n": "number of nodes", + "root": "chosen root node for flow (e.g., 0)" + }, + "decision_variables": { + "x[e]": "binary, 1 if edge e is selected", + "y[i]": "binary, 1 if node i is a leaf", + "f[(u,v)]": "continuous [0, n-1], flow on directed arc (u,v)" + }, + "objective": { + "sense": "max", + "expression": "sum(y[i] for i in N)" + }, + "constraints": [ + "sum(x[e] for e in E) == n-1", + "deg[i] = sum(x[e] for e incident to i) for all i in N", + "y[i] <= deg[i] for all i in N", + "deg[i] <= 1 + (n-1) * (1 - y[i]) for all i in N", + "f[(u,v)] <= (n-1) * x[(u,v)] for all (u,v) in directed arcs", + "f[(v,u)] <= (n-1) * x[(u,v)] for all (u,v) in directed arcs", + "flow conservation at root: outflow - inflow == n-1", + "flow conservation at non-root: inflow - outflow == 1" + ] +} +``` + +### Common Pitfalls +- Using `deg[i] <= 1 + M * (1 - y[i])` with too large `M` (use `n-1` for tightness). +- Confusing flow conservation signs between root and non-root nodes. +- Not ensuring flow variables are bounded by edge selection, allowing cycles. + +## Solving stage + +### Strategy Overview +Use a MILP solver (e.g., CBC via `mip` library or `pulp`). Set a time limit and optimality gap. Verify solution validity post-solve. + +### Step 1 - Initialize Solver and Set Parameters +- Create a solver instance (e.g., `mip.CBC` or `pulp.PULP_CBC_CMD`). +- Set a time limit: `solver.seconds = [TIME_LIMIT]`. +- Set optimality gap: `solver.emphasis = 0` (optimal) or `solver.max_mip_gap = 0.0`. + +### Step 2 - Solve the Model +- Call `status = model.optimize()`. +- Check status: `status == mip.OptimizationStatus.OPTIMAL` or `status == mip.OptimizationStatus.FEASIBLE`. + +### Step 3 - Extract and Verify Solution +- Extract objective value: `model.objective_value`. +- Extract selected edges: `[e for e in E if x[e].x > 0.5]`. +- **Verification checks**: + - Verify exactly `n-1` edges are selected. + - Verify connectivity using BFS/DFS from the root. + - Verify acyclicity (automatically satisfied if connected with `n-1` edges). + - Compute degrees and confirm leaf nodes match `y` variables. +- If any check fails, treat the solution as invalid. + +### Step 4 - Output Results +- Print a structured result (e.g., JSON) with status, objective value, leaf list, selected edges, and degrees. + +### Code Usage +```python +import mip + +def build_and_solve_max_leaf_spanning_tree_milp(nodes, eligible_edges, root=0, time_limit=60): + n = len(nodes) + edge_list = list(eligible_edges) + + model = mip.Model("MaxLeafSpanningTree") + + # Decision variables + x = {e: model.add_var(var_type=mip.BINARY, name=f'x_{e[0]}_{e[1]}') for e in edge_list} + y = {i: model.add_var(var_type=mip.BINARY, name=f'y_{i}') for i in nodes} + f = {} + for (u,v) in edge_list: + f[(u,v)] = model.add_var(lb=0, ub=n-1, name=f'f_{u}_{v}') + f[(v,u)] = model.add_var(lb=0, ub=n-1, name=f'f_{v}_{u}') + + # Spanning tree cardinality + model += mip.xsum(x[e] for e in edge_list) == n-1 + + # Degree and leaf constraints + incident_edges = {i: [] for i in nodes} + for e in edge_list: + u, v = e + incident_edges[u].append(x[e]) + incident_edges[v].append(x[e]) + for i in nodes: + deg = mip.xsum(incident_edges[i]) + model += y[i] <= deg + model += deg <= 1 + (n-1) * (1 - y[i]) + + # Flow constraints + for (u,v) in edge_list: + model += f[(u,v)] <= (n-1) * x[(u,v)] + model += f[(v,u)] <= (n-1) * x[(u,v)] + + # Flow conservation + for i in nodes: + outflow = mip.xsum(f[(i,j)] for (a,b) in edge_list for (i2,j) in [(a,b),(b,a)] if i2 == i) + inflow = mip.xsum(f[(j,i)] for (a,b) in edge_list for (j,i2) in [(a,b),(b,a)] if i2 == i) + if i == root: + model += outflow - inflow == n-1 + else: + model += inflow - outflow == 1 + + # Objective + model.objective = mip.maximize(mip.xsum(y[i] for i in nodes)) + + # Solve + model.optimize(max_seconds=time_limit) + + # Result parsing + result = {"status": str(model.status), "objective": None, "edges": [], "leaves": []} + if model.status in (mip.OptimizationStatus.OPTIMAL, mip.OptimizationStatus.FEASIBLE): + result["objective"] = model.objective_value + result["edges"] = [e for e in edge_list if x[e].x > 0.5] + result["leaves"] = [i for i in nodes if y[i].x > 0.5] + return result +``` + +### Common Pitfalls +- Not checking solver status before accessing `.x` values, causing attribute errors. +- Using `mip.xsum` incorrectly with generator expressions (wrap in list if needed). +- Forgetting to set a time limit, causing indefinite solve on large instances. diff --git a/skills/optskills/skill_library/maximum_weight_assignment_with_capacity_constraints.md b/skills/optskills/skill_library/maximum_weight_assignment_with_capacity_constraints.md new file mode 100644 index 0000000..d836a72 --- /dev/null +++ b/skills/optskills/skill_library/maximum_weight_assignment_with_capacity_constraints.md @@ -0,0 +1,249 @@ +--- +name: Maximum Weight Assignment with Capacity Constraints +description: | + Models and solves a maximum weight assignment problem where each resource has a capacity limit on the number of incident assignments it can participate in, using either CP-SAT or MILP solvers. +--- + +# Workflow 1 (CP-SAT Solver) + +## Modeling stage + +### Strategy Overview +Model the problem as a binary assignment problem with per-node degree constraints. Use Google OR-Tools CP-SAT solver, which is well-suited for combinatorial optimization with binary variables and linear constraints. + +### Step 1 - Define Decision Variables +- For each potential assignment (edge/link), create a binary variable `x_e` indicating whether that assignment is selected. +- Use `model.NewBoolVar(name)` for each variable, with a descriptive name like `"x_{i}_{j}"`. + +### Step 2 - Define Capacity Constraints +- For each resource (node/cell) with a finite capacity, sum the binary variables of all incident assignments. +- Enforce `sum(x_e for e incident to i) <= capacity[i]` using `model.Add()`. +- This models the constraint that each resource can be involved in at most a given number of assignments. + +### Step 3 - Define Objective Function +- Maximize the total weighted sum of selected assignments. +- Use `model.Maximize(sum(weight[e] * x_e for e in edges))` to directly encode the goal of maximizing total value. + +### Formulation Template +```json +{ + "sets": ["E: set of edges/assignments", "N: set of nodes/resources"], + "parameters": ["weight[e]: weight of edge e", "capacity[n]: capacity of node n", "incident_edges[n]: list of edges incident to node n"], + "decision_variables": ["x[e]: binary variable indicating if edge e is selected"], + "objective": { + "sense": "max", + "expression": "sum(weight[e] * x[e] for e in E)" + }, + "constraints": ["sum(x[e] for e in incident_edges[n]) <= capacity[n] for all n in N"] +} +``` + +### Common Pitfalls +- Forgetting to convert node indices to integers when using them in variable names or constraint indexing. +- Using `model.Add(sum(...))` without wrapping in parentheses for multi-line expressions. +- Not storing incident edges per node in a precomputed list, leading to O(n*m) constraint generation. + +## Solving stage + +### Strategy Overview +Use CP-SAT solver with parallel search and time limits. Extract solution by checking variable values and validate capacity constraints manually. + +### Step 1 - Configure Solver +- Create `CpSolver()` instance. +- Set `max_time_in_seconds` to a reasonable limit (e.g., 30 seconds) to avoid indefinite runtime. +- Enable `num_search_workers = 8` for parallel search. +- Set `random_seed = 42` for reproducibility. +- Use `relative_gap_limit = 0.0` to require optimality proof. + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve(model)`. +- Check if `status` is `cp_model.OPTIMAL` or `cp_model.FEASIBLE`. +- For infeasible/unbounded cases, print a JSON with `status: "failed"` and a descriptive reason. + +### Step 3 - Extract Solution +- Iterate over all variables and collect those with `solver.Value(x_e) == 1` to get the selected assignments. +- Compute the objective value via `solver.ObjectiveValue()`. +- Build structured output (e.g., JSON) containing status, objective value, and list of selected edges. + +### Code Usage +```python +from ortools.sat.python import cp_model +import json + +# Data preparation +edges = [(i, j, weight) for ...] # list of tuples +capacities = {node: cap for node, cap in ...} +incident_edges = {node: [] for node in capacities} +for idx, (i, j, w) in enumerate(edges): + incident_edges[i].append(idx) + incident_edges[j].append(idx) + +# Build model +model = cp_model.CpModel() +x = {} +for idx, (i, j, w) in enumerate(edges): + x[idx] = model.NewBoolVar(f"x_{i}_{j}") + +# Capacity constraints +for node, cap in capacities.items(): + model.Add(sum(x[idx] for idx in incident_edges[node]) <= cap) + +# Objective +model.Maximize(sum(w * x[idx] for idx, (_, _, w) in enumerate(edges))) + +# Solve +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30 +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 +solver.parameters.relative_gap_limit = 0.0 +status = solver.Solve(model) + +# Extract results +if status == cp_model.OPTIMAL or status == cp_model.FEASIBLE: + selected = [] + for idx, (i, j, w) in enumerate(edges): + if solver.Value(x[idx]) == 1: + selected.append({"edge": idx, "nodes": (i, j), "weight": w}) + payload = { + "status": "optimal" if status == cp_model.OPTIMAL else "feasible", + "objective": solver.ObjectiveValue(), + "selected_edges": selected + } + print(f"RESULT_JSON:{json.dumps(payload)}") +else: + payload = {"status": "failed", "reason": "infeasible_or_error"} + print(f"RESULT_JSON:{json.dumps(payload)}") +``` + +### Common Pitfalls +- Not checking for `cp_model.FEASIBLE` in addition to `cp_model.OPTIMAL`, missing valid solutions when optimality is not proven. +- Forgetting to convert `solver.ObjectiveValue()` to float for JSON serialization. +- Not precomputing incident edges per node, causing O(n*m) runtime in solution extraction. + +# Workflow 2 (MILP Solver) + +## Modeling stage + +### Strategy Overview +Model the problem as a mixed-integer linear program (MILP) using Pyomo. Use binary decision variables for each assignment and linear capacity constraints. This approach allows using high-performance MILP solvers like HiGHS or CBC. + +### Step 1 - Define Sets and Parameters +- Create a Pyomo Set for edges/links: `m.E = pyo.Set(initialize=range(len(edges_data)))`. +- Create a Pyomo Set for nodes/resources: `m.N = pyo.Set(initialize=nodes)`. +- Store weights in a dictionary keyed by edge index. +- Store edge-node relationships in a dictionary keyed by edge index. + +### Step 2 - Define Decision Variables +- Define binary variables for each edge: `m.x = pyo.Var(m.E, domain=pyo.Binary)`. +- Use descriptive variable names for debugging. + +### Step 3 - Define Objective Function +- Maximize total weighted sum: `m.obj = pyo.Objective(expr=sum(weights[e] * m.x[e] for e in m.E), sense=pyo.maximize)`. + +### Step 4 - Define Capacity Constraints +- For each node, create a constraint rule that sums binary variables of incident edges. +- Use `pyo.Constraint(m.N, rule=capacity_rule)` where `capacity_rule` checks if node is in edge's node tuple. + +### Formulation Template +```json +{ + "sets": ["E: set of edges/assignments", "N: set of nodes/resources"], + "parameters": ["weight[e]: weight of edge e", "capacity[n]: capacity of node n", "edge_nodes[e]: tuple of (node1, node2) for edge e"], + "decision_variables": ["x[e]: binary variable indicating if edge e is selected"], + "objective": { + "sense": "max", + "expression": "sum(weight[e] * x[e] for e in E)" + }, + "constraints": ["sum(x[e] for e in E if n in edge_nodes[e]) <= capacity[n] for all n in N"] +} +``` + +### Common Pitfalls +- Using `pyo.Set(initialize=edges_data)` directly instead of initializing with indices, causing indexing issues. +- Not converting node indices to integers when checking membership in edge tuples. +- Forgetting to use `pyo.value()` when extracting variable values after solving. + +## Solving stage + +### Strategy Overview +Use a high-performance MILP solver (HiGHS or CBC) with time limits and MIP gap tolerance. Extract solution by checking variable values and validate capacity constraints manually. + +### Step 1 - Configure Solver +- Use `pyo.SolverFactory("highs")` for high-performance MILP solving, or `pyo.SolverFactory("cbc")` for open-source alternative. +- Set key options: `time_limit=30`, `mip_rel_gap=0.0`, `threads=4`. +- For CBC, use `solver.options["seconds"] = 30` and `solver.options["ratio"] = 0.0`. + +### Step 2 - Solve and Check Status +- Call `results = solver.solve(model, tee=False)`. +- Check `results.solver.status == SolverStatus.ok` and `term in {TerminationCondition.optimal, TerminationCondition.feasible}`. +- For infeasible/unbounded cases, print a JSON with `status: "failed"` and descriptive reason. + +### Step 3 - Extract Solution +- Retrieve selected variables with `[e for e in m.E if pyo.value(m.x[e]) > 0.5]`. +- Convert to native Python types (e.g., `int(a), int(b)`) for JSON serialization. +- Build structured output (e.g., JSON) containing status, objective value, and list of selected edges. + +### Code Usage +```python +import json +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Data preparation +nodes = list(range(num_nodes)) +capacities = {node: cap_value for node in nodes} +edges_data = [(i, j, weight) for ...] # list of tuples + +# Build model +m = pyo.ConcreteModel() +m.E = pyo.Set(initialize=range(len(edges_data))) +m.N = pyo.Set(initialize=nodes) + +weights = {e: edges_data[e][2] for e in m.E} +edge_nodes = {e: (edges_data[e][0], edges_data[e][1]) for e in m.E} + +m.x = pyo.Var(m.E, domain=pyo.Binary) +m.obj = pyo.Objective(expr=sum(weights[e] * m.x[e] for e in m.E), sense=pyo.maximize) + +def capacity_rule(model, n): + return sum(model.x[e] for e in model.E if n in edge_nodes[e]) <= capacities[n] + +m.capacity = pyo.Constraint(m.N, rule=capacity_rule) + +# Solve +solver = pyo.SolverFactory("highs") +solver.options["time_limit"] = 30 +solver.options["mip_rel_gap"] = 0.0 +solver.options["threads"] = 4 +results = solver.solve(m, tee=False) + +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + selected = [] + for e in m.E: + if pyo.value(m.x[e]) > 0.5: + selected.append({"edge": e, "nodes": edge_nodes[e], "weight": weights[e]}) + payload = { + "status": "optimal" if term == TerminationCondition.optimal else "feasible", + "objective": float(pyo.value(m.obj)), + "selected_edges": selected, + } + print(f"RESULT_JSON:{json.dumps(payload)}") +else: + payload = { + "status": "failed", + "reason": "infeasible_or_error", + "solver_status": str(status), + "termination_condition": str(term), + } + print(f"RESULT_JSON:{json.dumps(payload)}") +``` + +### Common Pitfalls +- Not checking both `SolverStatus.ok` and `TerminationCondition.feasible`, missing valid solutions when optimality is not proven. +- Using `pyo.value()` on the entire variable object instead of individual variable values. +- Forgetting to convert objective value to float for JSON serialization. +- Not handling the case where solver returns `TerminationCondition.unbounded` separately from infeasible. diff --git a/skills/optskills/skill_library/maximum_weight_subgraph_with_degree_constraints.md b/skills/optskills/skill_library/maximum_weight_subgraph_with_degree_constraints.md new file mode 100644 index 0000000..cf40b46 --- /dev/null +++ b/skills/optskills/skill_library/maximum_weight_subgraph_with_degree_constraints.md @@ -0,0 +1,267 @@ +--- +name: Maximum Weight Subgraph with Degree Constraints +description: | + Model and solve a maximum weight subgraph selection problem where each node has a capacity limit on the number of incident selected edges. + +--- +# Workflow 1 (OR-Tools CP-SAT) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a maximum-weight b-matching integer program using OR-Tools CP-SAT. Use binary variables for edge selection, enforce degree constraints per node via precomputed incident lists, and maximize total selected edge weight. + +### Step 1 - Define Decision Variables +- Create a `BoolVar` for each edge in the edge list using `model.NewBoolVar(f'x_{i}')`. +- Store variables in a list indexed by edge index for easy access. + +### Step 2 - Precompute Incident Edge Lists +- For each node, initialize an empty list. +- For each edge index `i` and its endpoints `(u, v)`, append `i` to the incident lists for `u` and `v`. +- This ensures O(|E|) constraint building. + +### Step 3 - Enforce Degree Constraints +- For each node `v` with capacity `c[v]`, add a linear constraint: + `model.Add(sum(x[idx] for idx in incident[v]) <= c[v])`. +- Isolated nodes (no incident edges) require no constraint. + +### Step 4 - Define Objective +- Compute total weight as `sum(edge_weight[i] * x[i] for i in range(num_edges))`. +- Set objective with `model.Maximize(total_weight)`. + +### Formulation Template +```json +{ + "sets": ["E: edges", "V: nodes"], + "parameters": ["w[e]: weight of edge e", "c[v]: capacity of node v", "incident[v]: list of edge indices incident to node v"], + "decision_variables": ["x[e] ∈ {0,1}: 1 if edge e selected"], + "objective": { + "sense": "max", + "expression": "sum(w[e] * x[e] for e in E)" + }, + "constraints": ["sum(x[e] for e in incident[v]) <= c[v] for all v in V"] +} +``` + +### Common Pitfalls +- Forgetting to precompute incident edge lists, leading to O(|V|*|E|) constraint building. +- Using `model.AddBoolOr` or `model.AddBoolAnd` instead of linear sum constraints for degree limits. +- Not handling isolated nodes gracefully; they require no constraint. + +## Solving stage + +### Strategy Overview +Configure the CP-SAT solver with parameters for exact optimality, solve, and parse results with explicit status checking and feasibility verification. + +### Step 1 - Configure Solver +- Create `cp_model.CpSolver()` instance. +- Set parameters: + - `max_time_in_seconds` (e.g., `[TIME_LIMIT]`), + - `num_search_workers` (e.g., 8), + - `random_seed` (e.g., 42), + - `relative_gap_limit = 0.0` to ensure exact optimum. + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve(model)`. +- Check status against `cp_model.OPTIMAL` or `cp_model.FEASIBLE`. +- If status is `cp_model.INFEASIBLE`, output failure JSON. + +### Step 3 - Extract and Verify Results +- For each edge, check `solver.Value(x[i]) == 1` to identify selected edges. +- Compute objective value as `float(solver.ObjectiveValue())`. +- **Verification:** Recalculate node usage from selected edges to ensure all degree constraints are satisfied. + +### Code Usage +```python +from ortools.sat.python import cp_model + +def solve_max_weight_subgraph(edges, node_capacities): + """ + edges: list of (u, v, weight) + node_capacities: dict {node: capacity} + """ + model = cp_model.CpModel() + num_edges = len(edges) + x = [model.NewBoolVar(f'x_{i}') for i in range(num_edges)] + + # Build incident edge lists + incident = {node: [] for node in node_capacities} + for idx, (u, v, w) in enumerate(edges): + incident[u].append(idx) + incident[v].append(idx) + + # Degree constraints + for node, cap in node_capacities.items(): + if incident[node]: + model.Add(sum(x[idx] for idx in incident[node]) <= cap) + + # Objective + model.Maximize(sum(w * x[i] for i, (_, _, w) in enumerate(edges))) + + # Solve + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = 30.0 + solver.parameters.num_search_workers = 8 + solver.parameters.random_seed = 42 + solver.parameters.relative_gap_limit = 0.0 + + status = solver.Solve(model) + + if status == cp_model.OPTIMAL or status == cp_model.FEASIBLE: + selected = [(u, v, w) for i, (u, v, w) in enumerate(edges) if solver.Value(x[i]) == 1] + objective = float(solver.ObjectiveValue()) + # Verification + node_usage = {node: 0 for node in node_capacities} + for u, v, _ in selected: + node_usage[u] += 1 + node_usage[v] += 1 + feasible = all(node_usage[node] <= node_capacities[node] for node in node_capacities) + return {"status": "success", "objective": objective, "selected_edges": selected, "feasible": feasible} + else: + return {"status": "failed", "reason": "no_feasible_solution", "solver_status": status} +``` + +### Common Pitfalls +- Not setting `relative_gap_limit` to 0.0 when exact optimality is required. +- Forgetting to convert `solver.ObjectiveValue()` to float for JSON serialization. +- Assuming `solver.Value()` returns integer for BoolVar (it returns 0 or 1 as int). + +# Workflow 2 (Pyomo with HiGHS) + +## Modeling stage + +### Strategy Overview +Use Pyomo's algebraic modeling language with HiGHS solver. Define sets for edges and nodes, binary variables for edge selection, and linear constraints for degree limits using precomputed incident lists. + +### Step 1 - Define Sets and Parameters +- Create `pyomo.Set` for nodes and edges. +- Store edge weights in a dictionary keyed by edge index. +- Store node capacities in a dictionary. + +### Step 2 - Precompute Incident Edge Lists +- For each node, initialize an empty list. +- For each edge index and its endpoints `(u, v)`, append the edge index to the incident lists for `u` and `v`. + +### Step 3 - Create Decision Variables +- Declare `pyomo.Var(model.E, domain=pyomo.Binary)` for edge selection. + +### Step 4 - Write Degree Constraints +- For each node, create a constraint rule that sums binary variables of incident edges. +- Use `pyomo.Constraint(model.V, rule=degree_rule)` where `degree_rule` accesses the precomputed incident list. + +### Step 5 - Define Objective +- Use `pyomo.Objective(expr=sum(weight[e] * x[e] for e in model.E), sense=pyomo.maximize)`. + +### Formulation Template +```json +{ + "sets": ["E: edges", "V: nodes"], + "parameters": ["w[e]: weight of edge e", "c[v]: capacity of node v", "incident[v]: list of edges incident to node v"], + "decision_variables": ["x[e] ∈ {0,1}: 1 if edge e selected"], + "objective": { + "sense": "max", + "expression": "sum(w[e] * x[e] for e in E)" + }, + "constraints": ["sum(x[e] for e in incident[v]) <= c[v] for all v in V"] +} +``` + +### Common Pitfalls +- Using `pyomo.Set(initialize=...)` with mutable objects that cause indexing errors. +- Not precomputing incident edge lists inside the constraint rule, causing repeated computation. +- Forgetting to handle isolated nodes (they require no constraint). + +## Solving stage + +### Strategy Overview +Configure HiGHS solver with time limit and zero MIP gap tolerance, solve, and parse results with proper status checking and feasibility verification. + +### Step 1 - Configure Solver +- Use `pyomo.SolverFactory("highs")` to create solver instance. +- Set options: + - `"time_limit"` (e.g., `[TIME_LIMIT]`), + - `"mip_rel_gap" = 0.0` for exact optimum, + - `"threads"` (e.g., 4). + +### Step 2 - Solve and Check Status +- Call `result = solver.solve(model, tee=False)`. +- Check `result.solver.status == SolverStatus.ok` and `termination_condition` in `{TerminationCondition.optimal, TerminationCondition.feasible}`. + +### Step 3 - Extract and Verify Results +- For each edge, check `pyo.value(x[e]) > 0.5` to identify selected edges. +- Compute objective value as `float(pyo.value(model.obj))`. +- **Verification:** Recalculate node usage from selected edges to ensure all degree constraints are satisfied. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +def solve_max_weight_subgraph_pyomo(edges, node_capacities): + """ + edges: list of (u, v, weight) + node_capacities: dict {node: capacity} + """ + model = pyo.ConcreteModel() + + # Sets + edge_indices = list(range(len(edges))) + node_set = list(node_capacities.keys()) + model.E = pyo.Set(initialize=edge_indices) + model.V = pyo.Set(initialize=node_set) + + # Parameters + weight = {i: w for i, (_, _, w) in enumerate(edges)} + model.w = pyo.Param(model.E, initialize=weight) + + # Incident edges per node + incident = {node: [] for node in node_set} + for idx, (u, v, _) in enumerate(edges): + incident[u].append(idx) + incident[v].append(idx) + + # Variables + model.x = pyo.Var(model.E, domain=pyo.Binary) + + # Constraints + def degree_rule(model, node): + if not incident[node]: + return pyo.Constraint.Skip + return sum(model.x[e] for e in incident[node]) <= node_capacities[node] + model.degree_con = pyo.Constraint(model.V, rule=degree_rule) + + # Objective + model.obj = pyo.Objective(expr=sum(model.w[e] * model.x[e] for e in model.E), sense=pyo.maximize) + + # Solve + solver = pyo.SolverFactory("highs") + solver.options["time_limit"] = 30 + solver.options["mip_rel_gap"] = 0.0 + solver.options["threads"] = 4 + + result = solver.solve(model, tee=False) + + if (result.solver.status == SolverStatus.ok and + result.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}): + selected = [(u, v, w) for i, (u, v, w) in enumerate(edges) if pyo.value(model.x[i]) > 0.5] + objective = float(pyo.value(model.obj)) + # Verification + node_usage = {node: 0 for node in node_capacities} + for u, v, _ in selected: + node_usage[u] += 1 + node_usage[v] += 1 + feasible = all(node_usage[node] <= node_capacities[node] for node in node_capacities) + return {"status": "success", "objective": objective, "selected_edges": selected, "feasible": feasible} + else: + return { + "status": "failed", + "reason": "solver_error", + "solver_status": str(result.solver.status), + "termination_condition": str(result.solver.termination_condition) + } +``` + +### Common Pitfalls +- Not checking `termination_condition` in addition to `solver.status` for feasible solutions. +- Using `tee=True` in production code, which floods output with solver logs. +- Forgetting to convert `pyo.value()` results to native Python types for JSON serialization. diff --git a/skills/optskills/skill_library/min_max_assignment_solver.md b/skills/optskills/skill_library/min_max_assignment_solver.md new file mode 100644 index 0000000..73763a9 --- /dev/null +++ b/skills/optskills/skill_library/min_max_assignment_solver.md @@ -0,0 +1,232 @@ +--- +name: Min-Max Assignment Solver +description: | + Solves one-to-one matching problems with a min-max (makespan) objective by formulating it as a Mixed-Integer Linear Program (MILP) and using a high-performance solver, with verification via feasibility checks. + +--- +# Workflow 1 (CP-SAT with Big-M Linearization) + +## Modeling stage + +### Strategy Overview +Formulate the min-max assignment problem for a CP-SAT solver by linearizing the maximum cost constraint using a big-M formulation. This approach is effective for solvers that handle linear constraints well but do not natively support variable multiplication. + +### Step 1 - Define Core Variables +- Declare binary assignment variables `x[i][j]` for each agent i and task j. +- Declare an integer variable `max_cost` to represent the objective value to minimize. Set its lower bound to 0 and upper bound to a sufficiently large constant `M`. + +### Step 2 - Enforce Assignment Constraints +- Add constraints `sum(x[i][j] for j in tasks) == 1` for each agent i. +- Add constraints `sum(x[i][j] for i in agents) == 1` for each task j. + +### Step 3 - Link Max Cost to Assignments via Big-M +- For each pair (i, j), add a linear constraint: `max_cost >= cost[i][j] - M * (1 - x[i][j])`. +- Choose `M` as a sufficiently large number (e.g., `max(cost_matrix) + 1`) to deactivate the constraint when `x[i][j] = 0`. Setting `M` too small may cut off valid solutions; setting it excessively large can cause numerical instability. + +### Formulation Template +```json +{ + "sets": ["agents", "tasks"], + "parameters": ["cost[agents][tasks]"], + "decision_variables": [ + {"name": "x", "indices": ["agents", "tasks"], "type": "binary"}, + {"name": "max_cost", "type": "integer", "bounds": [0, "M"]} + ], + "objective": { + "sense": "min", + "expression": "max_cost" + }, + "constraints": [ + "assignment_agent[i in agents]: sum(x[i][j] for j in tasks) == 1", + "assignment_task[j in tasks]: sum(x[i][j] for i in agents) == 1", + "max_cost_link[i in agents][j in tasks]: max_cost >= cost[i][j] - M * (1 - x[i][j])" + ] +} +``` + +### Common Pitfalls +- Setting `M` too small, which may cut off valid solutions. +- Setting `M` excessively large, which can cause numerical instability in the solver. +- Forgetting to define `max_cost` as an integer variable when costs are integral, leading to unnecessary continuous relaxation. + +## Solving stage + +### Strategy Overview +Use the OR-Tools CP-SAT solver to find the optimal min-max assignment. Configure it for optimality and use a secondary feasibility model to verify the optimality gap. + +### Step 1 - Configure Solver for Optimality +- Set `solver.parameters.max_time_in_seconds` to control runtime. +- Set `solver.parameters.num_search_workers` for parallelism. +- Set `solver.parameters.random_seed` for reproducibility. +- Set `solver.parameters.relative_gap_limit = 0.0` to enforce optimality. + +### Step 2 - Solve and Check Status +- Call `solver.Solve(model)`. +- Check the status is `OPTIMAL` or `FEASIBLE`. Handle `INFEASIBLE` or `UNKNOWN` statuses with appropriate logging. + +### Step 3 - Verify Optimality via Feasibility Check +- Let `V` be the objective value from the solution. +- Create a new feasibility model with the same assignment constraints. +- Add constraints `cost[i][j] * x[i][j] <= V - 1` for all i, j (or use the big-M form: `max_cost >= cost[i][j] - M * (1 - x[i][j])` and set `max_cost <= V - 1`). +- Solve the feasibility model. If it is infeasible, `V` is proven optimal. + +### Step 4 - Extract and Validate Solution +- Retrieve the assignment by iterating over `x[i][j]` variables and checking if `solver.Value(x[i][j]) == 1`. +- Compute the actual maximum cost from the assignment and verify it matches the solver's `max_cost` value. + +### Code Usage +```python +# Build model from formulation +model = cp_model.CpModel() +x = {} +for i in agents: + for j in tasks: + x[i, j] = model.NewBoolVar(f"x_{i}_{j}") +max_cost = model.NewIntVar(0, M, "max_cost") + +# Assignment constraints +for i in agents: + model.Add(sum(x[i, j] for j in tasks) == 1) +for j in tasks: + model.Add(sum(x[i, j] for i in agents) == 1) + +# Big-M linking constraints +for i in agents: + for j in tasks: + model.Add(max_cost >= cost[i][j] - M * (1 - x[i, j])) + +# Objective +model.Minimize(max_cost) + +# Solve with status / termination checks +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = 8 +solver.parameters.relative_gap_limit = 0.0 +status = solver.Solve(model) + +if status in [cp_model.OPTIMAL, cp_model.FEASIBLE]: + # Extract solution + assignment = {i: j for i in agents for j in tasks if solver.Value(x[i, j]) == 1} + # Verification and output +else: + # Handle infeasible or unknown status +``` + +### Common Pitfalls +- Not checking solver status before extracting values, leading to runtime errors. +- Using an incorrect threshold (e.g., `V - epsilon` with inappropriate `epsilon`) in the feasibility check, causing false infeasibility. +- Assuming the solver's objective value is integral; use integer variable for `max_cost` when costs are integers. + +# Workflow 2 (Pyomo with Direct Multiplication) + +## Modeling stage + +### Strategy Overview +Formulate the problem in Pyomo using direct multiplication of binary variables and cost parameters within constraints. This declarative approach leverages Pyomo's ability to handle bilinear terms (binary * parameter) and is suitable for MILP solvers. + +### Step 1 - Define Model Structure +- Create a `ConcreteModel()`. +- Define sets for `agents` and `tasks`. +- Define a `Param` for the `cost` matrix. + +### Step 2 - Declare Decision Variables +- Define binary variables `x[i, j]` over the cross product of agents and tasks. +- Define a continuous, non-negative variable `max_cost`. + +### Step 3 - Enforce Assignment and Max Cost Constraints +- Add constraints ensuring each agent is assigned to exactly one task and each task gets exactly one agent. +- Add constraints `max_cost >= cost[i, j] * x[i, j]` for all i, j. Pyomo will linearize this product for MILP solvers. + +### Formulation Template +```json +{ + "sets": ["agents", "tasks"], + "parameters": ["cost[agents][tasks]"], + "decision_variables": [ + {"name": "x", "indices": ["agents", "tasks"], "type": "binary"}, + {"name": "max_cost", "type": "continuous", "domain": "NonNegativeReals"} + ], + "objective": { + "sense": "min", + "expression": "max_cost" + }, + "constraints": [ + "assign_agent[i in agents]: sum(x[i, j] for j in tasks) == 1", + "assign_task[j in tasks]: sum(x[i, j] for i in agents) == 1", + "max_cost_def[i in agents][j in tasks]: max_cost >= cost[i, j] * x[i, j]" + ] +} +``` + +### Common Pitfalls +- Forgetting to initialize the `cost` parameter, leading to an incomplete model. +- Defining `max_cost` without a proper domain, which may allow negative values. +- Assuming the solver automatically linearizes the product; some solvers may require explicit linearization hints. + +## Solving stage + +### Strategy Overview +Use a high-performance MILP solver (e.g., Gurobi, HiGHS) via Pyomo's `SolverFactory`. Configure for deterministic optimization and use a binary search on cost thresholds for optimality verification. + +### Step 1 - Select and Configure Solver +- Instantiate the solver: `solver = SolverFactory('solver_name')`. +- Set options: `time_limit` for runtime, `mip_gap=0.0` for optimality, `threads` for parallelism, and `seed` for reproducibility. + +### Step 2 - Solve and Inspect Termination Condition +- Call `results = solver.solve(model, tee=False)`. +- Check `results.solver.status` and `results.solver.termination_condition`. Proceed only if status is `ok` and termination is `optimal` or `feasible`. + +### Step 3 - Verify Optimality via Binary Search +- Let `candidate_value` be the objective value from the solution. +- Perform a binary search on the discrete set of possible cost values. +- For a test threshold `T`, create a feasibility model with assignment constraints and additional constraints: `x[i, j] == 0` for all pairs where `cost[i][j] > T`. +- Solve the feasibility model. The smallest `T` for which a feasible assignment exists is the optimal makespan. + +### Step 4 - Extract and Present Solution +- Retrieve the assignment by iterating over `x[i, j]` and checking `pyo.value(x[i, j]) > 0.5`. +- Output the assignment mapping and the confirmed optimal makespan. + +### Code Usage +```python +# Build model from formulation +import pyomo.environ as pyo +model = pyo.ConcreteModel() +model.agents = pyo.Set(initialize=agents) +model.tasks = pyo.Set(initialize=tasks) +model.cost = pyo.Param(model.agents, model.tasks, initialize=cost_data) +model.x = pyo.Var(model.agents, model.tasks, domain=pyo.Binary) +model.max_cost = pyo.Var(domain=pyo.NonNegativeReals) + +def assign_agent_rule(m, i): + return sum(m.x[i, j] for j in m.tasks) == 1 +model.assign_agent = pyo.Constraint(model.agents, rule=assign_agent_rule) + +def assign_task_rule(m, j): + return sum(m.x[i, j] for i in m.agents) == 1 +model.assign_task = pyo.Constraint(model.tasks, rule=assign_task_rule) + +def max_cost_rule(m, i, j): + return m.max_cost >= m.cost[i, j] * m.x[i, j] +model.max_cost_def = pyo.Constraint(model.agents, model.tasks, rule=max_cost_rule) + +model.obj = pyo.Objective(expr=model.max_cost, sense=pyo.minimize) + +# Solve with status / termination checks +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = [TIME_LIMIT] +solver.options['mip_rel_gap'] = 0.0 +results = solver.solve(model, tee=False) + +if results.solver.status == pyo.SolverStatus.ok and results.solver.termination_condition == pyo.TerminationCondition.optimal: + # Extract solution + assignment = {i: j for i in model.agents for j in model.tasks if pyo.value(model.x[i, j]) > 0.5} + # Verification and output +else: + # Handle other statuses +``` + +### Common Pitfalls +- Not setting `load_solutions=False` when performing feasibility checks, which can cause errors if the model is infeasible. +- Using `pyo.value()` on variables before ensuring a solution is loaded. +- Overlooking the need to convert the cost matrix into a Pyomo `Param` dictionary correctly. diff --git a/skills/optskills/skill_library/minimax_assignment_with_absolute_difference_constraints.md b/skills/optskills/skill_library/minimax_assignment_with_absolute_difference_constraints.md new file mode 100644 index 0000000..92c40af --- /dev/null +++ b/skills/optskills/skill_library/minimax_assignment_with_absolute_difference_constraints.md @@ -0,0 +1,242 @@ +--- +name: Minimax Assignment with Absolute Difference Constraints +description: | + Build and solve optimization models that assign integer values to entities while enforcing minimum absolute differences between pairs and minimizing the maximum assigned value. + +--- +# Workflow 1 (CP-SAT with Boolean Encoding) + +## Modeling stage + +### Strategy Overview +Model the problem as a graph labeling problem where vertices represent entities and edges represent separation constraints. Use integer assignment variables with explicit bounds and encode absolute difference constraints via Boolean indicator variables. This approach leverages CP-SAT's native support for logical constraints and avoids big-M formulations. + +### Step 1 - Define Assignment Variables +- Create an integer variable `x_i` for each entity `i` with domain `[0, U]`. Set `U` conservatively (e.g., `max_constraint_distance * number_of_entities` or a large constant like `[UPPER_BOUND]`). +- Use `model.NewIntVar(0, U, f"x_{i}")` for each entity index `i`. + +### Step 2 - Encode Absolute Difference Constraints +- For each pair `(u, v, d)` requiring `|x_u - x_v| >= d`, introduce a Boolean variable `b` using `model.NewBoolVar(f"b_{u}_{v}")`. +- Add two conditional constraints: + - `x_u - x_v >= d` enforced only when `b` is true: `model.Add(x_u - x_v >= d).OnlyEnforceIf(b)` + - `x_v - x_u >= d` enforced only when `b` is false: `model.Add(x_v - x_u >= d).OnlyEnforceIf(b.Not())` + +### Step 3 - Minimax Objective +- Create an auxiliary variable `max_x = model.NewIntVar(0, U, "max_x")`. +- Add constraints `model.Add(max_x >= x_i)` for all entities `i`. +- Set objective: `model.Minimize(max_x)`. + +### Formulation Template +```json +{ + "sets": ["I: entities"], + "parameters": ["E: set of (u, v, d) pairs requiring |x_u - x_v| >= d", "U: upper bound for assignment values"], + "decision_variables": [ + "x_i: integer assignment for entity i, domain [0, U]", + "b_e: Boolean variable for each edge e in E", + "max_x: auxiliary variable for maximum assignment" + ], + "objective": { + "sense": "min", + "expression": "max_x" + }, + "constraints": [ + "max_x >= x_i for all i in I", + "For each (u, v, d) in E: x_u - x_v >= d if b_e is true", + "For each (u, v, d) in E: x_v - x_u >= d if b_e is false" + ] +} +``` + +### Common Pitfalls +- Choosing too small an upper bound `U` may make the problem infeasible; set it generously based on problem size. +- Forgetting to enforce both directions of the absolute difference constraint (only one direction is enforced per Boolean state). +- Not using `.OnlyEnforceIf(b.Not())` correctly — ensure the negation is applied to the same Boolean variable. + +## Solving stage + +### Strategy Overview +Use OR-Tools CP-SAT solver with parallel search and time limits. Extract results with status checking and output in structured JSON format. + +### Step 1 - Configure Solver +- Create solver instance: `solver = cp_model.CpSolver()` +- Set parameters: `solver.parameters.max_time_in_seconds = [TIME_LIMIT]`, `solver.parameters.num_search_workers = [N_WORKERS]`, `solver.parameters.random_seed = [SEED]`. + +### Step 2 - Solve and Extract Results +- Call `status = solver.Solve(model)`. +- Check status: `status == cp_model.OPTIMAL` or `status == cp_model.FEASIBLE`. +- Retrieve values: `solver.Value(x_i)` for each entity, `solver.Value(max_x)` for objective. +- Compute objective as `float(solver.ObjectiveValue())`. + +### Step 3 - Verify Solution and Prove Optimality +- **Verify Feasibility**: Explicitly check all absolute difference constraints `|x_u - x_v| >= d` are satisfied by the retrieved assignments. +- **Prove Optimality**: After finding a solution with objective `K`, attempt to find a feasible solution with `max_x <= K-1`. Infeasibility at the lower bound confirms optimality. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# Build model +model = cp_model.CpModel() +x = {i: model.NewIntVar(0, U, f"x_{i}") for i in entities} +max_x = model.NewIntVar(0, U, "max_x") +for i in entities: + model.Add(max_x >= x[i]) + +for (u, v, d) in edges: + b = model.NewBoolVar(f"b_{u}_{v}") + model.Add(x[u] - x[v] >= d).OnlyEnforceIf(b) + model.Add(x[v] - x[u] >= d).OnlyEnforceIf(b.Not()) + +model.Minimize(max_x) + +# Solve +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = [N_WORKERS] +solver.parameters.random_seed = [SEED] +status = solver.Solve(model) + +# Extract results +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + assignments = {i: solver.Value(x[i]) for i in entities} + result = { + "status": "optimal" if status == cp_model.OPTIMAL else "feasible", + "objective": float(solver.ObjectiveValue()), + "assignments": assignments, + "max_assigned": solver.Value(max_x) + } +else: + result = {"status": "failed", "solver_status_code": status} +print(result) +``` + +### Common Pitfalls +- Not checking for `FEASIBLE` status in addition to `OPTIMAL` — feasible solutions may be acceptable. +- Forgetting to convert `solver.ObjectiveValue()` to float for JSON serialization. +- Using too few search workers for large instances; increase to match available cores. + +# Workflow 2 (MILP with Big-M Disjunction) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program using big-M disjunctions for absolute difference constraints. This approach works with any MILP solver and provides a compact linear formulation. + +### Step 1 - Define Assignment Variables +- Create integer variables `x_i` for each entity `i` with explicit bounds: `bounds=(0, U)` where `U` is a generous upper bound. +- Use `pyo.Var(domain=pyo.NonNegativeIntegers, bounds=(0, U))` for each entity. + +### Step 2 - Encode Absolute Difference Constraints with Big-M +- For each pair `(u, v, d)`, introduce a binary variable `y_e` (0 or 1). +- Add two constraints using big-M: + - `x_u - x_v >= d - M * (1 - y_e)` + - `x_v - x_u >= d - M * y_e` +- Set `M = U + max(d)` (or larger) to ensure the constraints are redundant when the binary variable is in the wrong state. + +### Step 3 - Minimax Objective +- Add a variable `M_max` with domain `NonNegativeIntegers` and bounds `(0, U)`. +- Add constraints `M_max >= x_i` for all entities `i`. +- Set objective: `pyo.Objective(expr=M_max, sense=pyo.minimize)`. + +### Formulation Template +```json +{ + "sets": ["I: entities"], + "parameters": ["E: set of (u, v, d) pairs", "U: upper bound for assignment values", "M: big-M value, typically U + max(d)"], + "decision_variables": [ + "x_i: integer assignment for entity i, bounds [0, U]", + "y_e: binary variable for each edge e in E", + "M_max: integer variable for maximum assignment, bounds [0, U]" + ], + "objective": { + "sense": "min", + "expression": "M_max" + }, + "constraints": [ + "M_max >= x_i for all i in I", + "For each (u, v, d) in E: x_u - x_v >= d - M * (1 - y_e)", + "For each (u, v, d) in E: x_v - x_u >= d - M * y_e" + ] +} +``` + +### Common Pitfalls +- Choosing too small a big-M value can cut off feasible solutions; ensure `M >= U + max(d)`. +- Not setting explicit bounds on integer variables, leading to unbounded search space. +- Forgetting that both big-M constraints must be added for each pair to enforce the absolute difference. + +## Solving stage + +### Strategy Overview +Use a MILP solver (e.g., HiGHS via Pyomo) with optimality gap set to zero and time limit. Extract results with proper status checking and output in structured format. + +### Step 1 - Configure Solver +- Create solver: `solver = pyo.SolverFactory("highs")` +- Set options: `solver.options["time_limit"] = [TIME_LIMIT]`, `solver.options["mip_rel_gap"] = 0.0`, `solver.options["threads"] = [N_THREADS]`. + +### Step 2 - Solve and Extract Results +- Call `result = solver.solve(model, tee=False)`. +- Check status: `result.solver.status == pyo.SolverStatus.ok` and `result.solver.termination_condition in {pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible}`. +- Extract values: `int(pyo.value(model.x[i]))` for each entity, `int(pyo.value(model.M_max))` for objective. + +### Step 3 - Verify Solution and Prove Optimality +- **Verify Feasibility**: Explicitly check all absolute difference constraints `|x_u - x_v| >= d` are satisfied by the extracted assignments. +- **Prove Optimality**: After finding a solution with objective `K`, attempt to find a feasible solution with `M_max <= K-1`. Infeasibility at the lower bound confirms optimality. + +### Code Usage +```python +import pyomo.environ as pyo + +# Build model +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=entities) +model.E = pyo.Set(initialize=edges, dimen=3) + +U = [UPPER_BOUND] # generous upper bound +M = U + max(d for _, _, d in edges) + +model.x = pyo.Var(model.I, domain=pyo.NonNegativeIntegers, bounds=(0, U)) +model.y = pyo.Var(model.E, domain=pyo.Binary) +model.M_max = pyo.Var(domain=pyo.NonNegativeIntegers, bounds=(0, U)) + +# Objective +model.obj = pyo.Objective(expr=model.M_max, sense=pyo.minimize) + +# Max constraints +def max_rule(m, i): + return m.M_max >= m.x[i] +model.max_con = pyo.Constraint(model.I, rule=max_rule) + +# Absolute difference constraints +model.abs_con = pyo.ConstraintList() +for u, v, d in edges: + model.abs_con.add(model.x[u] - model.x[v] >= d - M * (1 - model.y[u, v, d])) + model.abs_con.add(model.x[v] - model.x[u] >= d - M * model.y[u, v, d]) + +# Solve +solver = pyo.SolverFactory("highs") +solver.options["time_limit"] = [TIME_LIMIT] +solver.options["mip_rel_gap"] = 0.0 +solver.options["threads"] = [N_THREADS] +result = solver.solve(model, tee=False) + +# Extract results +if (result.solver.status == pyo.SolverStatus.ok and + result.solver.termination_condition in {pyo.TerminationCondition.optimal, + pyo.TerminationCondition.feasible}): + assignments = {i: int(pyo.value(model.x[i])) for i in entities} + output = { + "status": "optimal" if result.solver.termination_condition == pyo.TerminationCondition.optimal else "feasible", + "objective": float(pyo.value(model.M_max)), + "assignments": assignments + } +else: + output = {"status": "failed", "solver_status": str(result.solver.status)} +print(output) +``` + +### Common Pitfalls +- Not converting Pyomo variable values to native Python types (int/float) before JSON serialization. +- Forgetting to set `mip_rel_gap = 0.0` for exact optimality when required. +- Using `tee=True` in production code — set to `False` to avoid excessive console output. diff --git a/skills/optskills/skill_library/minimum_cost_flow_with_route_constraints.md b/skills/optskills/skill_library/minimum_cost_flow_with_route_constraints.md new file mode 100644 index 0000000..9304e93 --- /dev/null +++ b/skills/optskills/skill_library/minimum_cost_flow_with_route_constraints.md @@ -0,0 +1,237 @@ +--- +name: Minimum Cost Flow with Route Constraints +description: | + Model and solve a minimum cost flow problem with flow balance, minimum flow, and capacity constraints on routes using either Pyomo or OR-Tools. +--- + +# Workflow 1 (Pyomo LP Formulation) + +## Modeling stage + +### Strategy Overview +Define a linear programming model using Pyomo's ConcreteModel with continuous decision variables for flow on each route. Enforce flow balance at nodes, minimum flow, and capacity constraints using indexed parameters and constraint rules. + +### Step 1 - Define Sets and Parameters +- Create a set `ROUTES` indexed by route identifiers. +- Create a set `NODES` for all warehouse locations. +- For each route, define parameters: `origin`, `destination`, `cost_per_unit`, `min_flow`, `max_flow` stored in dictionaries keyed by route ID. + +### Step 2 - Create Decision Variables +- Declare a continuous non-negative variable `flow[r]` for each route `r` in `ROUTES` using `pyo.Var(ROUTES, domain=pyo.NonNegativeReals)`. + +### Step 3 - Define Objective +- Minimize total cost: `sum(flow[r] * cost_per_unit[r] for r in ROUTES)`. + +### Step 4 - Add Constraints +- **Flow balance**: For each node `n` in `NODES`, enforce `sum(flow[r] for r in ROUTES if destination[r] == n) == sum(flow[r] for r in ROUTES if origin[r] == n)`. +- **Minimum flow**: For each route `r`, `flow[r] >= min_flow[r]`. +- **Capacity**: For each route `r`, `flow[r] <= max_flow[r]`. + +### Formulation Template +```json +{ + "sets": ["ROUTES", "NODES"], + "parameters": ["origin[ROUTES]", "destination[ROUTES]", "cost_per_unit[ROUTES]", "min_flow[ROUTES]", "max_flow[ROUTES]"], + "decision_variables": ["flow[ROUTES] >= 0"], + "objective": { + "sense": "min", + "expression": "sum(flow[r] * cost_per_unit[r] for r in ROUTES)" + }, + "constraints": [ + "flow_balance[n]: sum(flow[r] for r in ROUTES if destination[r]==n) == sum(flow[r] for r in ROUTES if origin[r]==n) for n in NODES", + "min_flow[r]: flow[r] >= min_flow[r] for r in ROUTES", + "capacity[r]: flow[r] <= max_flow[r] for r in ROUTES" + ] +} +``` + +### Common Pitfalls +- Using integer variables unnecessarily when continuous flows are acceptable, increasing solve time. +- Forgetting to include all nodes in the flow balance set, leading to infeasibility. +- Mixing up origin and destination in flow balance constraints. + +## Solving stage + +### Strategy Overview +Use Pyomo's SolverFactory with an LP-capable solver (e.g., GLPK, HiGHS). Set time limits and check solver status before extracting results. Output structured JSON for downstream parsing. + +### Step 1 - Instantiate Solver +- Create solver with `pyo.SolverFactory("glpk")` or `pyo.SolverFactory("highs")`. + +### Step 2 - Configure Solver Options +- Set time limit: `solver.options["tmlim"] = 30` (seconds). +- Set MIP gap if using MIP: `solver.options["mipgap"] = 0.0`. +- Set threads: `solver.options["threads"] = 4`. + +### Step 3 - Solve and Check Status +- Call `result = solver.solve(model, tee=False)`. +- Check `result.solver.status == pyo.SolverStatus.ok` and `result.solver.termination_condition in {pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible}`. + +### Step 4 - Extract Results +- Get objective: `float(pyo.value(model.obj))`. +- Get variable values: `{r: float(pyo.value(model.flow[r])) for r in model.ROUTES}`. + +### Code Usage +```python +import pyomo.environ as pyo + +# Build model +model = pyo.ConcreteModel() +model.ROUTES = pyo.Set(initialize=route_ids) +model.NODES = pyo.Set(initialize=node_ids) + +# Parameters (example dictionaries) +origin = {...}; dest = {...}; cost = {...}; min_f = {...}; max_f = {...} + +model.flow = pyo.Var(model.ROUTES, domain=pyo.NonNegativeReals) + +def obj_rule(m): + return sum(m.flow[r] * cost[r] for r in m.ROUTES) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + +def flow_balance_rule(m, n): + inflow = sum(m.flow[r] for r in m.ROUTES if dest[r] == n) + outflow = sum(m.flow[r] for r in m.ROUTES if origin[r] == n) + return inflow == outflow +model.flow_balance = pyo.Constraint(model.NODES, rule=flow_balance_rule) + +def min_flow_rule(m, r): + return m.flow[r] >= min_f[r] +model.min_flow = pyo.Constraint(model.ROUTES, rule=min_flow_rule) + +def cap_rule(m, r): + return m.flow[r] <= max_f[r] +model.capacity = pyo.Constraint(model.ROUTES, rule=cap_rule) + +# Solve +solver = pyo.SolverFactory("glpk") +solver.options["tmlim"] = 30 +result = solver.solve(model, tee=False) + +# Check and output +if result.solver.status == pyo.SolverStatus.ok and result.solver.termination_condition in {pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible}: + obj_val = float(pyo.value(model.obj)) + flows = {r: float(pyo.value(model.flow[r])) for r in model.ROUTES} + print(f"RESULT_JSON:{{\"status\":\"success\",\"objective\":{obj_val},\"flows\":{flows}}}") +else: + print(f"RESULT_JSON:{{\"status\":\"failed\",\"solver_status\":\"{result.solver.status}\",\"termination\":\"{result.solver.termination_condition}\"}}") +``` + +### Common Pitfalls +- Not checking termination condition, leading to reading results from infeasible models. +- Using `tee=True` in production, cluttering logs. +- Forgetting to convert Pyomo values to Python floats/ints for JSON serialization. + +# Workflow 2 (OR-Tools Integer Flow) + +## Modeling stage + +### Strategy Overview +Use OR-Tools pywraplp with SCIP backend to model flows as integer variables. Encode minimum flow and capacity directly as variable bounds. Use additive constraints for flow balance and objective coefficients. + +### Step 1 - Define Data Structures +- Create lists for route origins, destinations, costs, minimum flows, and maximum flows, indexed by route index. +- Create a set of unique node identifiers. + +### Step 2 - Create Solver and Variables +- Instantiate solver: `pywraplp.Solver.CreateSolver("SCIP")`. +- For each route `i`, create an integer variable: `solver.IntVar(min_flow[i], max_flow[i], f"flow_{i}")`. This directly enforces both minimum flow and capacity constraints. + +### Step 3 - Add Flow Balance Constraints +- For each node `n`, compute `outflow = sum(var[i] for i where origin[i]==n)` and `inflow = sum(var[i] for i where dest[i]==n)`. +- Add constraint: `solver.Add(outflow == inflow)`. + +### Step 4 - Set Objective +- Create objective: `objective = solver.Objective()`. +- For each route `i`, set coefficient: `objective.SetCoefficient(var[i], cost[i])`. +- Set minimization: `objective.SetMinimization()`. + +### Formulation Template +```json +{ + "sets": ["routes indexed by i", "nodes"], + "parameters": ["origin[i]", "destination[i]", "cost[i]", "min_flow[i]", "max_flow[i]"], + "decision_variables": ["flow[i] integer in [min_flow[i], max_flow[i]]"], + "objective": { + "sense": "min", + "expression": "sum(flow[i] * cost[i] for i in routes)" + }, + "constraints": [ + "flow_balance[n]: sum(flow[i] for i where origin[i]==n) == sum(flow[i] for i where dest[i]==n) for n in nodes" + ] +} +``` + +### Common Pitfalls +- Using `IntVar` when continuous flows are acceptable, unnecessarily restricting the solution space. +- Forgetting to include all nodes in flow balance, especially nodes that only appear as origin or destination. +- Not setting variable bounds correctly, leading to infeasibility. + +## Solving stage + +### Strategy Overview +Solve using OR-Tools SCIP solver with time limits and thread settings. Check solver status against OPTIMAL or FEASIBLE before extracting results. Output structured JSON. + +### Step 1 - Configure Solver +- Set time limit: `solver.SetTimeLimit(30000)` (milliseconds). +- Set threads: `solver.SetNumThreads(4)`. + +### Step 2 - Solve +- Call `status = solver.Solve()`. + +### Step 3 - Check Status and Extract +- Check `if status in (pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE):`. +- Get objective: `objective.Value()`. +- Get variable values: `var.solution_value()` for each route. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# Data +origins = [...] # list of origin nodes per route +dests = [...] # list of destination nodes per route +costs = [...] # list of costs per unit +min_flows = [...] # list of minimum flows +max_flows = [...] # list of maximum capacities +nodes = set(origins + dests) + +# Create solver +solver = pywraplp.Solver.CreateSolver("SCIP") +solver.SetTimeLimit(30000) +solver.SetNumThreads(4) + +# Variables with bounds +vars = [] +for i in range(len(origins)): + var = solver.IntVar(min_flows[i], max_flows[i], f"flow_{i}") + vars.append(var) + +# Flow balance constraints +for n in nodes: + outflow = sum(vars[i] for i in range(len(origins)) if origins[i] == n) + inflow = sum(vars[i] for i in range(len(origins)) if dests[i] == n) + solver.Add(outflow == inflow) + +# Objective +objective = solver.Objective() +for i in range(len(origins)): + objective.SetCoefficient(vars[i], costs[i]) +objective.SetMinimization() + +# Solve +status = solver.Solve() + +# Check and output +if status in (pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE): + obj_val = objective.Value() + flows = {f"route_{i}": int(var.solution_value()) for i, var in enumerate(vars)} + print(f"RESULT_JSON:{{\"status\":\"success\",\"objective\":{obj_val},\"flows\":{flows}}}") +else: + print(f"RESULT_JSON:{{\"status\":\"failed\",\"solver_status_code\":{status}}}") +``` + +### Common Pitfalls +- Not converting `solution_value()` to int for integer variables, causing JSON serialization issues. +- Using `solver.Solve()` without checking status, leading to crashes on infeasible models. +- Setting time limit too low for large instances, causing premature termination without feasible solution. diff --git a/skills/optskills/skill_library/minimum_cost_network_flow.md b/skills/optskills/skill_library/minimum_cost_network_flow.md new file mode 100644 index 0000000..548dd53 --- /dev/null +++ b/skills/optskills/skill_library/minimum_cost_network_flow.md @@ -0,0 +1,271 @@ +--- +name: Minimum Cost Network Flow +description: | + Model and solve capacitated network flow problems with supply/demand nodes and linear transportation costs using structured data and solver-aware implementations. + +--- +# Workflow 1 (Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +Use Pyomo's abstract modeling language for a declarative formulation that cleanly separates data and model. This approach is easy to modify, scale, and is well-suited for problems where the network structure is defined by explicit sets and parameters. + +### Step 1 - Define Problem Sets and Parameters +- Define a `Set` for nodes (`model.N`). +- Define a `Set` for arcs (`model.A`) as a subset of `model.N × model.N`, populated from available cost or capacity data to ensure sparsity. +- Create `Param` dictionaries for `cost`, `capacity`, and `demand`. +- **Adopt a consistent sign convention**: For each node `i`, assign `demand[i] = net inflow - net outflow`. Therefore, supply nodes (net outflow) have negative demand values; demand nodes (net inflow) have positive demand values. +- **Prerequisite check**: Before solving, verify total supply equals total demand (`sum(d for d in demand.values() if d < 0) == sum(d for d in demand.values() if d > 0)`). If not, the problem is infeasible. + +### Step 2 - Create Decision Variables +- Define a `Var` for flow on each arc (`model.x`). For standard capacitated flow, use `domain=pyo.NonNegativeReals` and set capacity via a separate constraint or via variable bounds. +- **For circulation problems or problems with lower bounds**: Define variable bounds directly using the `bounds` argument referencing parameters `lower_bound` and `upper_bound`. This integrates capacity constraints efficiently and reduces explicit constraint count. + +### Step 3 - Formulate Flow Conservation Constraints +- For each node `i` in `model.N`, create a constraint: `sum(inflow) - sum(outflow) == demand[i]`. +- In code, implement as: `sum(model.x[j,i] for j if (j,i) in model.A) - sum(model.x[i,j] for j if (i,j) in model.A) == model.demand[i]`. +- For circulation problems (net demand zero at all nodes), set `demand[i] = 0` for all `i`. +- Pre-compute incoming/outgoing arc lists for efficiency, or use Pyomo's `Arc` component if available. + +### Step 4 - Add Capacity Constraints (if not integrated via bounds) +- For each arc `(i,j)` in the arc set, add a constraint: `model.x[i,j] <= capacity[i,j]`. +- Use `Constraint.Skip` for arcs without an explicit capacity to maintain a sparse model. + +### Step 5 - Define Linear Cost Objective +- Define the objective to minimize: `sum(cost[i,j] * model.x[i,j] for (i,j) in model.A)`. + +### Formulation Template +```json +{ + "sets": [ + "N: set of nodes", + "A: subset of N × N, directed arcs" + ], + "parameters": [ + "demand[i ∈ N]: net demand (positive for sink, negative for source; zero for circulation)", + "cost[(i,j) ∈ A]: unit flow cost", + "capacity[(i,j) ∈ A]: maximum flow", + "lower_bound[(i,j) ∈ A]: minimum flow (optional, default 0)" + ], + "decision_variables": [ + "x[(i,j) ∈ A]: flow variable with bounds [lower_bound[i,j], capacity[i,j]]" + ], + "objective": { + "sense": "min", + "expression": "∑_{(i,j) ∈ A} cost[i,j] * x[i,j]" + }, + "constraints": [ + "flow_conservation[i ∈ N]: ∑_{j:(j,i) ∈ A} x[j,i] - ∑_{j:(i,j) ∈ A} x[i,j] = demand[i]" + ] +} +``` + +### Common Pitfalls +- Using an inconsistent sign convention for demand/supply without clear documentation. +- Creating variables and constraints for all possible node pairs instead of only existing arcs, resulting in a dense, inefficient model. +- Implementing flow conservation with nested generator expressions that cause `KeyError` by accessing invalid variable indices. +- Redundantly setting `bounds=(0, None)` on variables already declared with `domain=pyo.NonNegativeReals`. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using an open-source LP/MILP solver (HiGHS or CBC) via `SolverFactory`. Focus on robust solution handling, including status checks, graceful failure reporting, and optional solution verification. + +### Step 1 - Configure and Execute Solver +- Instantiate the solver: `solver = pyo.SolverFactory("highs")` (or `"cbc"`). +- Set appropriate options: `"time_limit": [TIME_LIMIT]`, `"presolve": "on"`. +- Solve with `load_solutions=False`: `results = solver.solve(model, load_solutions=False, tee=False)`. + +### Step 2 - Check Solver Status and Termination +- Check `results.solver.status` is `SolverStatus.ok`. +- Check `results.solver.termination_condition` is `TerminationCondition.optimal` or `TerminationCondition.feasible`. +- If not acceptable, analyze and report failure (e.g., `infeasible`, `unbounded`). + +### Step 3 - Load and Extract Solution +- If status is good, load the solution: `model.solutions.load_from(results)`. +- Extract the objective value: `obj_val = pyo.value(model.obj)`. +- Retrieve non-zero flows by iterating over `model.x` and filtering with a tolerance (e.g., `if pyo.value(model.x[i,j]) > 1e-6`). + +### Step 4 - (Optional) Verify Solution Correctness +- Programmatically verify flow conservation at each node by recomputing net flow from the solution. +- Check that all flows respect capacity and lower bounds within a numerical tolerance. +- **Be aware of multiple optimal solutions**: Network flow problems can have multiple optimal flow distributions with the same objective value. Different solvers may return different flows; verify the objective value matches. +- **Cross-Check with Alternative Solver**: For critical validation, solve the same model with a different solver (e.g., CBC after HiGHS) to confirm optimal objective value and solution integrity. +- This step is for debugging and should be optional in production. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Data +nodes = [...] # list of node identifiers +demand = {node: value, ...} # positive for sink, negative for source; zero for circulation +arcs_data = { + (i, j): {'cost': c, 'cap': cap, 'lb': lb} for all arcs # lb optional, default 0 +} + +# Prerequisite check: verify supply-demand balance +total_supply = sum(v for v in demand.values() if v < 0) # negative values are supply +total_demand = sum(v for v in demand.values() if v > 0) # positive values are demand +assert abs(total_supply + total_demand) < 1e-6, f"Supply-demand imbalance detected; problem is infeasible. Supply: {-total_supply}, Demand: {total_demand}." + +model = pyo.ConcreteModel() +model.N = pyo.Set(initialize=nodes) +model.A = pyo.Set(initialize=arcs_data.keys(), dimen=2) +model.demand = pyo.Param(model.N, initialize=demand) +model.cost = pyo.Param(model.A, initialize={(i,j): d['cost'] for (i,j), d in arcs_data.items()}) +model.capacity = pyo.Param(model.A, initialize={(i,j): d['cap'] for (i,j), d in arcs_data.items()}) +model.lower_bound = pyo.Param(model.A, initialize={(i,j): d.get('lb', 0) for (i,j), d in arcs_data.items()}) + +# Define variable with integrated bounds +model.x = pyo.Var( + model.A, + bounds=lambda m, i, j: (m.lower_bound[i, j], m.capacity[i, j]) +) + +def flow_balance_rule(m, i): + outflow = sum(m.x[i, j] for j in m.N if (i, j) in m.A) + inflow = sum(m.x[j, i] for j in m.N if (j, i) in m.A) + return inflow - outflow == m.demand[i] # inflow - outflow = demand +model.flow_balance = pyo.Constraint(model.N, rule=flow_balance_rule) + +def obj_rule(m): + return sum(m.cost[i, j] * m.x[i, j] for (i, j) in m.A) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + +solver = pyo.SolverFactory("highs") +solver.options = {"time_limit": [TIME_LIMIT], "presolve": "on"} +results = solver.solve(model, load_solutions=False, tee=False) + +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in [TerminationCondition.optimal, TerminationCondition.feasible]): + model.solutions.load_from(results) + obj_val = pyo.value(model.obj) + print(f"RESULT:{obj_val}") + for (i, j) in model.A: + val = pyo.value(model.x[i, j]) + if val > 1e-6: + print(f" x[{i},{j}] = {val:.4f}") +else: + raise RuntimeError(f"Solver failed: {results.solver.status}, {results.solver.termination_condition}") +``` + +### Common Pitfalls +- Using `tee=True` in production, generating excessive solver logs. +- Setting solver options not supported by the backend (e.g., `"threads"` for some CBC versions). +- Not checking termination condition before loading solutions, risking `NoFeasibleSolutionError`. +- Implementing redundant verification by re-solving the model instead of checking the existing solution. + +# Workflow 2 (OR-Tools Linear Solver) + +## Modeling stage + +### Strategy Overview +Use Google OR-Tools' linear solver API (`pywraplp`) for an imperative, programmatic model build. This is well-suited for rapid prototyping and problems where the model structure is generated algorithmically. Variable bounds are set directly upon creation, integrating capacity constraints. + +### Step 1 - Initialize Solver and Data Structures +- Choose a solver backend: `solver = pywraplp.Solver.CreateSolver('GLOP')` for LP or `'CBC'` for MILP. +- Store problem parameters in nested dictionaries or lists keyed by node indices: `cost`, `capacity`, `demand`, `lower_bound` (optional, default 0). +- **Adopt a consistent sign convention**: For each node `i`, assign `demand[i] = net inflow - net outflow`. Therefore, supply nodes (net outflow) have negative demand values; demand nodes (net inflow) have positive demand values. +- **Prerequisite check**: Verify total supply equals total demand (`sum(v for v in demand.values() if v < 0) == sum(v for v in demand.values() if v > 0)`) before solving; if not, the problem is infeasible. + +### Step 2 - Create Flow Variables with Integrated Bounds +- For each directed arc `(i, j)` where capacity data exists, create a variable: `x[i][j] = solver.NumVar(lower_bound[i][j], capacity[i][j], f'x_{i}_{j}')`. +- This directly enforces lower and upper bounds. + +### Step 3 - Build Flow Conservation Constraints +- For each node `i`, create a constraint object: `constraint = solver.Constraint(demand[i], demand[i])`. +- For each outgoing arc `(i, j)`, add `-1 * x[i][j]` to the constraint (representing outflow). +- For each incoming arc `(j, i)`, add `+1 * x[j][i]` to the constraint (representing inflow). +- This yields: `inflow - outflow = demand[i]`. + +### Step 4 - Define Linear Cost Objective +- Create the objective: `objective = solver.Objective()`. +- For each arc `(i, j)`, add term `cost[i][j] * x[i][j]` to the objective. +- Set the objective sense to minimization: `objective.SetMinimization()`. + +### Formulation Template +```json +{ + "sets": [ + "N: list of node indices", + "A: list of (i,j) tuples for existing arcs" + ], + "parameters": [ + "demand[i ∈ N]: net demand (positive for sink, negative for source; zero for circulation)", + "cost[(i,j) ∈ A]: unit flow cost", + "capacity[(i,j) ∈ A]: maximum flow", + "lower_bound[(i,j) ∈ A]: minimum flow (optional, default 0)" + ], + "decision_variables": [ + "x[(i,j) ∈ A]: flow variable with bounds [lower_bound[i,j], capacity[i,j]]" + ], + "objective": { + "sense": "min", + "expression": "∑_{(i,j) ∈ A} cost[i,j] * x[i,j]" + }, + "constraints": [ + "flow_conservation[i ∈ N]: ∑_{j:(j,i) ∈ A} x[j,i] - ∑_{j:(i,j) ∈ A} x[i,j] = demand[i]" + ] +} +``` + +### Common Pitfalls +- Manually managing constraint coefficients with incorrect signs for inflow/outflow. +- Creating variables for non-existent arcs, leading to a dense model and potential key errors. +- Not verifying total supply/demand balance before solving, which can cause infeasibility. +- Mixing parameter types in a single data structure, reducing clarity. + +## Solving stage + +### Strategy Overview +Solve using OR-Tools' solver object directly. Focus on efficient model construction, robust solution status checking, and extracting results into a structured format for downstream use. + +### Step 1 - Execute Solve and Check Status +- Call `solver.Solve()`. +- Check the result status: `status = solver.Solve()`. +- Interpret status: `status == pywraplp.Solver.OPTIMAL` or `FEASIBLE` indicates success. + +### Step 2 - Extract and Validate Solution +- If optimal/feasible, get objective value: `obj_val = objective.Value()`. +- Retrieve flow values by iterating over arcs: `flow_val = x[i][j].solution_value()`. +- Filter near-zero flows using a tolerance (e.g., `1e-6`) to report only significant flows. +- **Optionally verify solution correctness**: Recompute net flow at each node and check bound adherence. +- **Be aware of multiple optimal solutions**: Different solvers may return different flow distributions; verify the objective value matches. + +### Step 3 - Handle Failures and Provide Diagnostics +- For non-optimal statuses (e.g., `INFEASIBLE`, `UNBOUNDED`), output a structured error message. +- Include diagnostic information such as total supply/demand imbalance or capacity tightness to aid debugging. + +### Step 4 - Structure Output +- Return a consistent output format, e.g., a JSON payload containing solver status, objective value, and a list of non-zero flows with details (from, to, amount, cost, capacity). + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +solver = pywraplp.Solver.CreateSolver('GLOP') +# ... build model as described ... + +status = solver.Solve() +if status in [pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE]: + obj_val = solver.Objective().Value() + solution_flows = {} + for i, j in arcs: + val = x[i][j].solution_value() + if val > 1e-6: + solution_flows[(i, j)] = val + # Package results... +else: + # Handle failure + status_map = {pywraplp.Solver.INFEASIBLE: 'INFEASIBLE', ...} + raise RuntimeError(f"Solver status: {status_map.get(status, 'UNKNOWN')}") +``` + +### Common Pitfalls +- Not using a tolerance when checking for non-zero flows, potentially including numerical noise. +- Running redundant verification solves instead of analyzing the existing solution. +- Printing extensive debug information (like all flows) in production, cluttering output. +- Hardcoding tolerance values without documentation or consideration for problem scale. diff --git a/skills/optskills/skill_library/minimum_cut_via_binary_partition.md b/skills/optskills/skill_library/minimum_cut_via_binary_partition.md new file mode 100644 index 0000000..40dbb91 --- /dev/null +++ b/skills/optskills/skill_library/minimum_cut_via_binary_partition.md @@ -0,0 +1,240 @@ +--- +name: Minimum Cut via Binary Partition +description: | + Formulate and solve the minimum s-t cut problem using either a mixed-integer linear programming (MIP) approach with explicit binary partition variables or a max-flow reduction using the max-flow min-cut theorem. + +--- +# Workflow 1 (MIP with Binary Partition Variables) + +## Modeling stage + +### Strategy Overview +Model the minimum cut problem directly by assigning each node to one of two partitions (S or T) using binary variables. The cut capacity is linearized via auxiliary continuous variables, resulting in a mixed-integer linear program that can be solved with any standard MIP solver. + +### Step 1 - Define Node Partition Variables +- Introduce a binary variable `x[i]` for each node `i` in the set of nodes `N`. +- Set `x[i] = 1` if node `i` belongs to the source side (S), and `x[i] = 0` if it belongs to the sink side (T). + +### Step 2 - Fix Source and Sink Membership +- Add a constraint `x[source] == 1` to enforce that the source node is in S. +- Add a constraint `x[sink] == 0` to enforce that the sink node is in T. + +### Step 3 - Linearize Cut Indicator +- For each directed arc `(i, j)` in the set of arcs `A`, introduce a continuous auxiliary variable `y[i,j] >= 0`. +- Add constraints `y[i,j] >= x[i] - x[j]` for all arcs `(i,j)`. This ensures `y[i,j] = 1` when the arc crosses from S to T (i.e., `x[i]=1` and `x[j]=0`), and `y[i,j] = 0` otherwise due to the minimization objective. + +### Step 4 - Define Objective +- Minimize the total cut capacity: `minimize sum(capacity[i,j] * y[i,j] for all arcs (i,j) in A)`. + +### Formulation Template +```json +{ + "sets": ["N: set of nodes", "A: set of directed arcs (i,j)"], + "parameters": ["source: source node index", "sink: sink node index", "capacity[i,j]: capacity of arc (i,j) for (i,j) in A"], + "decision_variables": [ + "x[i] in {0,1} for i in N: 1 if node i is in S, 0 if in T", + "y[i,j] >= 0 for (i,j) in A: cut indicator variable" + ], + "objective": { + "sense": "min", + "expression": "sum(capacity[i,j] * y[i,j] for (i,j) in A)" + }, + "constraints": [ + "x[source] == 1", + "x[sink] == 0", + "y[i,j] >= x[i] - x[j] for all (i,j) in A" + ] +} +``` + +### Common Pitfalls +- Forgetting to enforce `x[source] == 1` and `x[sink] == 0`, which can lead to trivial cuts (e.g., all nodes in S or all in T). +- Omitting the non-negativity constraint on `y[i,j]`, which may cause the solver to assign negative values. +- Using a large number of arcs without considering that the LP relaxation is naturally integral due to total unimodularity; a pure LP solver may suffice. + +## Solving stage + +### Strategy Overview +Solve the MIP formulation using a standard MIP solver (e.g., CBC, GLPK, or Gurobi) via Pyomo. The problem is totally unimodular, so the LP relaxation yields integer solutions, but a MIP solver is used for robustness. + +### Step 1 - Build the Model in Pyomo +- Import `pyomo.environ` as `pyo`. +- Create a `ConcreteModel` and define sets `N` and `A` from the input data. +- Declare binary variables `x` indexed over `N` and continuous non-negative variables `y` indexed over `A`. +- Add constraints for source/sink fixation and cut linearization. +- Set the objective expression. + +### Step 2 - Configure and Solve +- Instantiate a solver (e.g., `SolverFactory('cbc')` or `SolverFactory('glpk')`). +- Set solver options: `mipgap = 0.0` for optimality, `seconds = [TIME_LIMIT]` for a time limit. +- Call `solver.solve(model, tee=False)` and capture the results object. + +### Step 3 - Extract and Validate Results +- Check `results.solver.status` and `results.solver.termination_condition`; only proceed if status is `ok` and termination is `optimal` or `feasible`. +- Retrieve the objective value via `pyo.value(model.objective)`. +- Extract the partition: nodes with `pyo.value(x[i]) > 0.5` are in S; others are in T. +- Verify the cut by iterating over arcs and summing capacities where `x[i] > 0.5` and `x[j] < 0.5`. This sum must equal the objective value. + +### Code Usage +```python +import pyomo.environ as pyo + +def solve_min_cut_milp(nodes, arcs, source, sink, capacities): + model = pyo.ConcreteModel() + model.N = pyo.Set(initialize=nodes) + model.A = pyo.Set(initialize=arcs, dimen=2) + + model.x = pyo.Var(model.N, domain=pyo.Binary) + model.y = pyo.Var(model.A, domain=pyo.NonNegativeReals) + + # Fix source and sink + model.fix_source = pyo.Constraint(expr=model.x[source] == 1) + model.fix_sink = pyo.Constraint(expr=model.x[sink] == 0) + + # Cut linearization constraints + def cut_rule(m, i, j): + return m.y[i, j] >= m.x[i] - m.x[j] + model.cut_constr = pyo.Constraint(model.A, rule=cut_rule) + + # Objective + model.obj = pyo.Objective( + expr=sum(capacities[(i, j)] * model.y[i, j] for (i, j) in model.A), + sense=pyo.minimize + ) + + solver = pyo.SolverFactory('cbc') + solver.options['mipgap'] = 0.0 + solver.options['seconds'] = 30 + results = solver.solve(model, tee=False) + + if results.solver.status != pyo.SolverStatus.ok: + raise RuntimeError(f"Solver failed: {results.solver.status}") + if results.solver.termination_condition not in (pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible): + raise RuntimeError(f"No feasible solution: {results.solver.termination_condition}") + + cut_value = pyo.value(model.obj) + S = [i for i in model.N if pyo.value(model.x[i]) > 0.5] + T = [i for i in model.N if pyo.value(model.x[i]) < 0.5] + cut_arcs = [(i, j) for (i, j) in model.A if pyo.value(model.x[i]) > 0.5 and pyo.value(model.x[j]) < 0.5] + + # Verification + computed_cut_value = sum(capacities[(i, j)] for (i, j) in cut_arcs) + if abs(cut_value - computed_cut_value) > 1e-6: + raise RuntimeError(f"Cut verification failed: objective {cut_value} != computed cut {computed_cut_value}") + + return {"status": "success", "cut_value": cut_value, "S": S, "T": T, "cut_arcs": cut_arcs} +``` + +### Common Pitfalls +- Not checking solver status before reading results, leading to runtime errors from uninitialized variables. +- Using a solver that does not support binary variables (e.g., pure LP solver) without verifying integrality. +- Setting an overly tight time limit that prevents the solver from reaching optimality for large networks. + +# Workflow 2 (Max-Flow Reduction) + +## Modeling stage + +### Strategy Overview +Apply the max-flow min-cut theorem: the minimum s-t cut capacity equals the maximum flow from source to sink. Model the problem as a flow network and compute the maximum flow using a dedicated max-flow algorithm, avoiding explicit binary variables. + +### Step 1 - Define the Flow Network +- Represent the directed graph with nodes indexed from 0 to N-1, where the source is node `source` and the sink is node `sink`. +- Each directed arc `(i, j)` has a capacity `capacity[i,j]`. Only capacities are needed; ignore any extraneous costs. + +### Step 2 - Formulate as Maximum Flow +- The objective is to maximize the flow from source to sink subject to capacity constraints and flow conservation at intermediate nodes. +- The maximum flow value directly gives the minimum cut capacity. + +### Step 3 - Implicit Partition via Residual Graph +- After computing the maximum flow, the partition (S, T) is determined by reachability from the source in the residual graph. +- Nodes reachable from the source via arcs with positive residual capacity belong to S; all others belong to T. + +### Formulation Template +```json +{ + "sets": ["N: set of nodes", "A: set of directed arcs (i,j)"], + "parameters": ["source: source node index", "sink: sink node index", "capacity[i,j]: capacity of arc (i,j) for (i,j) in A"], + "decision_variables": ["flow[i,j] >= 0 for (i,j) in A: flow on each arc"], + "objective": { + "sense": "max", + "expression": "net flow out of source" + }, + "constraints": [ + "flow[i,j] <= capacity[i,j] for all (i,j) in A", + "sum(flow[i,j] for j) - sum(flow[j,i] for j) == 0 for all i not in {source, sink}" + ] +} +``` + +### Common Pitfalls +- Assuming the graph is undirected; the formulation requires directed arcs. For undirected edges, replace each with two directed arcs of equal capacity. +- Forgetting that the max-flow min-cut theorem applies only to a single source-sink pair; multi-source/multi-sink problems require a super-source/super-sink transformation. +- Using a solver that does not handle integer capacities correctly; max-flow algorithms work with integer or floating-point capacities. + +## Solving stage + +### Strategy Overview +Use a dedicated max-flow solver (e.g., OR-Tools' `SimpleMaxFlow`) to compute the maximum flow efficiently. The solver handles the flow conservation and capacity constraints internally, returning the maximum flow value and enabling partition extraction via residual graph traversal. + +### Step 1 - Build the Max-Flow Solver +- Import `SimpleMaxFlow` from `ortools.graph.python.max_flow`. +- Create a `SimpleMaxFlow` object. +- For each directed arc `(i, j)` with capacity `c`, call `solver.add_arc_with_capacity(i, j, c)`. + +### Step 2 - Solve and Check Status +- Call `solver.solve(source, sink)`. +- Check that the return status is `solver.OPTIMAL`; otherwise, the problem is infeasible or unbounded. + +### Step 3 - Extract Results +- Retrieve the maximum flow value via `solver.optimal_flow()`, which equals the minimum cut capacity. +- Extract the partition by performing a BFS/DFS on the residual graph: start from the source, traverse forward arcs where `solver.capacity(tail, head) - solver.flow(tail, head) > 0` and backward arcs where `solver.flow(head, tail) > 0`. Nodes reachable form S; the rest form T. +- Verify the cut by summing capacities of arcs from S to T; this sum must equal the max flow value. + +### Code Usage +```python +from ortools.graph.python.max_flow import SimpleMaxFlow +from collections import deque + +def solve_min_cut_maxflow(nodes, arcs, source, sink, capacities): + smf = SimpleMaxFlow() + for (i, j) in arcs: + smf.add_arc_with_capacity(i, j, capacities[(i, j)]) + + status = smf.solve(source, sink) + if status != smf.OPTIMAL: + raise RuntimeError(f"Max flow solver failed with status {status}") + + max_flow = smf.optimal_flow() + + # Extract partition via BFS on residual graph + S = set([source]) + queue = deque([source]) + while queue: + node = queue.popleft() + for arc in range(smf.num_arcs()): + tail = smf.tail(arc) + head = smf.head(arc) + if tail == node and head not in S: + if smf.capacity(arc) - smf.flow(arc) > 0: # residual capacity + S.add(head) + queue.append(head) + if head == node and tail not in S: + if smf.flow(arc) > 0: # reverse flow + S.add(tail) + queue.append(tail) + + T = [n for n in nodes if n not in S] + cut_arcs = [(i, j) for (i, j) in arcs if i in S and j not in S] + cut_value = sum(capacities[(i, j)] for (i, j) in cut_arcs) + + # Verification: cut value should equal max flow + if abs(cut_value - max_flow) > 1e-6: + raise RuntimeError(f"Cut verification failed: cut value {cut_value} != max flow {max_flow}") + + return {"status": "success", "cut_value": cut_value, "S": list(S), "T": T, "cut_arcs": cut_arcs} +``` + +### Common Pitfalls +- Not handling the case where the source and sink are the same node; the solver may return zero flow. +- Assuming the residual graph traversal is symmetric; always check both forward and backward arcs. +- Using `solver.num_arcs()` incorrectly if arcs were added dynamically; ensure all arcs are added before solving. diff --git a/skills/optskills/skill_library/minimum_group_assignment_with_conflict_separation.md b/skills/optskills/skill_library/minimum_group_assignment_with_conflict_separation.md new file mode 100644 index 0000000..6742d85 --- /dev/null +++ b/skills/optskills/skill_library/minimum_group_assignment_with_conflict_separation.md @@ -0,0 +1,290 @@ +--- +name: Minimum Group Assignment with Conflict Separation +description: | + Models and solves the problem of assigning items to the minimum number of groups such that conflicting pairs are never placed in the same group, using either binary assignment variables with group usage indicators or integer assignment variables with reified constraints. +--- + +# Workflow 1 (Binary Assignment with Group Indicators) + +## Modeling stage + +### Strategy Overview +Use a binary assignment variable for each item-group pair, a binary group usage indicator, and linear constraints to enforce each item is assigned exactly once, conflicting items are separated, and group usage is correctly linked. The objective minimizes the sum of group usage indicators. + +### Step 1 - Define Sets and Parameters +- Define the set of items `I` and an upper bound on groups `K` (e.g., `|I|`). +- Define the set of conflict pairs `C` as tuples `(u, v)` where items cannot share a group. + +### Step 2 - Create Decision Variables +- Create binary variable `x[i, k]` for each item `i` in `I` and each group `k` in `K`, indicating assignment. +- Create binary variable `y[k]` for each group `k` in `K`, indicating if the group is used. + +### Step 3 - Add Assignment Constraints +- For each item `i`, enforce `sum(x[i, k] for k in K) == 1` to ensure exactly one group per item. + +### Step 4 - Add Conflict Separation Constraints +- For each conflict pair `(u, v)` in `C` and each group `k` in `K`, enforce `x[u, k] + x[v, k] <= 1`. + +### Step 5 - Link Group Usage +- For each item `i` and group `k`, enforce `x[i, k] <= y[k]` to activate `y[k]` when any item is assigned to group `k`. + +### Step 6 - Set Objective +- Minimize `sum(y[k] for k in K)` to minimize the number of used groups. + +### Formulation Template +```json +{ + "sets": ["I: items", "K: groups (0..|I|-1)", "C: conflict pairs (u,v)"], + "parameters": [], + "decision_variables": [ + "x[i,k] binary: item i assigned to group k", + "y[k] binary: group k is used" + ], + "objective": { + "sense": "min", + "expression": "sum(y[k] for k in K)" + }, + "constraints": [ + "sum(x[i,k] for k in K) == 1 for all i in I", + "x[u,k] + x[v,k] <= 1 for all (u,v) in C, k in K", + "x[i,k] <= y[k] for all i in I, k in K" + ] +} +``` + +### Common Pitfalls +- Setting the upper bound on groups too low, causing infeasibility; always use `|I|` as a safe upper bound. +- Forgetting to add conflict constraints for both orderings `(u,v)` and `(v,u)` if the conflict list is not symmetric. +- Omitting the `x[i,k] <= y[k]` constraint, which allows the solver to use a group without counting it in the objective. + +## Solving stage + +### Strategy Overview +Use a constraint programming solver (OR-Tools CP-SAT) that efficiently handles binary variables and linear constraints. Configure time limits and parallelism for practical performance. + +### Step 1 - Initialize Solver and Model +- Import `cp_model` from `ortools.sat.python`. +- Create a `CpModel()` instance. + +### Step 2 - Build Variables and Constraints +- Use `model.NewBoolVar(name)` for each `x[i,k]` and `y[k]`. +- Add constraints using `model.Add(expression)`. + +### Step 3 - Configure Solver Parameters +- Set `solver.parameters.max_time_in_seconds` to a reasonable limit (e.g., 30.0). +- Set `solver.parameters.num_search_workers` to 8 for parallel search. +- Optionally set `solver.parameters.random_seed` for reproducibility. + +### Step 4 - Solve and Extract Solution +- Call `status = solver.Solve(model)`. +- Check if `status == cp_model.OPTIMAL` or `status == cp_model.FEASIBLE`. +- Extract assignment: for each item `i`, find the group `k` where `solver.Value(x[i,k]) > 0`. +- Count used groups: `sum(1 for k in K if solver.Value(y[k]) > 0)`. + +### Step 5 - Validate and Output +- Verify no conflict pair shares the same group by checking the extracted assignment. +- Output a JSON payload with `status`, `objective` (as float), and the assignment dictionary. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# Problem data +items = list(range(num_items)) +conflict_pairs = [(u, v), ...] # list of tuples +max_groups = len(items) +groups = list(range(max_groups)) + +model = cp_model.CpModel() + +# Variables +x = {} +for i in items: + for k in groups: + x[i, k] = model.NewBoolVar(f'x_{i}_{k}') +y = {} +for k in groups: + y[k] = model.NewBoolVar(f'y_{k}') + +# Constraints +for i in items: + model.Add(sum(x[i, k] for k in groups) == 1) + +for u, v in conflict_pairs: + for k in groups: + model.Add(x[u, k] + x[v, k] <= 1) + +for i in items: + for k in groups: + model.Add(x[i, k] <= y[k]) + +# Objective +model.Minimize(sum(y[k] for k in groups)) + +# Solve +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 8 +status = solver.Solve(model) + +if status == cp_model.OPTIMAL or status == cp_model.FEASIBLE: + obj_val = float(sum(1 for k in groups if solver.Value(y[k]) > 0)) + assignment = {} + for i in items: + for k in groups: + if solver.Value(x[i, k]) > 0: + assignment[i] = k + break + # Verify no conflicts + for u, v in conflict_pairs: + assert assignment[u] != assignment[v], f"Conflict violation: {u}, {v}" + print(f"RESULT:{obj_val}") +else: + print('{"status": "failed", "reason": "infeasible_or_timeout"}') +``` + +### Common Pitfalls +- Not checking for `FEASIBLE` status in addition to `OPTIMAL`, which may discard valid solutions found within the time limit. +- Using `solver.Value()` on variables that were not part of the solution (e.g., after infeasibility), causing runtime errors. +- Forgetting to set a time limit, causing the solver to run indefinitely on large instances. + +# Workflow 2 (Integer Assignment with Reified Constraints) + +## Modeling stage + +### Strategy Overview +Use an integer variable for each item representing its group assignment, with reified Boolean variables to track group usage. This reduces the number of decision variables compared to the binary approach, but requires more complex linking constraints. + +### Step 1 - Define Sets and Parameters +- Define the set of items `I` and an upper bound on groups `K` (e.g., `|I|`). +- Define the set of conflict pairs `C` as tuples `(u, v)`. + +### Step 2 - Create Decision Variables +- Create integer variable `color[i]` for each item `i` in `I` with domain `[0, max_groups-1]`. +- Create Boolean variable `used[k]` for each group `k` in `K` to indicate if the group is used. + +### Step 3 - Add Conflict Separation Constraints +- For each conflict pair `(u, v)` in `C`, enforce `color[u] != color[v]`. + +### Step 4 - Link Group Usage with Reified Constraints +- For each item `i` and group `k`, create a Boolean variable `eq[i][k]` that is true iff `color[i] == k`. +- Use `model.Add(color[i] == k).OnlyEnforceIf(eq[i][k])` and `model.Add(color[i] != k).OnlyEnforceIf(eq[i][k].Not())` to enforce the equivalence. +- For each item `i` and group `k`, enforce `model.Add(used[k] == 1).OnlyEnforceIf(eq[i][k])` to activate `used[k]` when any item is assigned to group `k`. + +### Step 5 - Set Objective +- Minimize `sum(used[k] for k in K)` to minimize the number of used groups. + +### Formulation Template +```json +{ + "sets": ["I: items", "K: groups (0..|I|-1)", "C: conflict pairs (u,v)"], + "parameters": [], + "decision_variables": [ + "color[i] integer in [0, |I|-1]: group assignment for item i", + "used[k] binary: group k is used", + "eq[i,k] binary: indicator that color[i] == k" + ], + "objective": { + "sense": "min", + "expression": "sum(used[k] for k in K)" + }, + "constraints": [ + "color[u] != color[v] for all (u,v) in C", + "color[i] == k iff eq[i,k] == 1 for all i in I, k in K", + "used[k] == 1 if eq[i,k] == 1 for all i in I, k in K" + ] +} +``` + +### Common Pitfalls +- Creating too many `eq[i,k]` variables (O(|I|^2)) which can blow up memory; only create them for items and groups that are relevant. +- Forgetting the `OnlyEnforceIf` for the negative case (`color[i] != k`), which can leave the equivalence unconstrained. +- Using `model.Add(color[i] == k)` directly without reification, which forces a specific assignment rather than allowing the solver to choose. + +## Solving stage + +### Strategy Overview +Use a constraint programming solver (OR-Tools CP-SAT) that supports reified constraints via `OnlyEnforceIf`. This approach is well-suited for problems with integer variables and logical implications. + +### Step 1 - Initialize Solver and Model +- Import `cp_model` from `ortools.sat.python`. +- Create a `CpModel()` instance. + +### Step 2 - Build Variables and Constraints +- Use `model.NewIntVar(0, max_groups-1, name)` for each `color[i]`. +- Use `model.NewBoolVar(name)` for each `used[k]` and `eq[i,k]`. +- Add constraints using `model.Add(expression).OnlyEnforceIf(condition)`. + +### Step 3 - Configure Solver Parameters +- Set `solver.parameters.max_time_in_seconds` to a reasonable limit (e.g., 30.0). +- Set `solver.parameters.num_search_workers` to 8 for parallel search. +- Optionally set `solver.parameters.random_seed` for reproducibility. + +### Step 4 - Solve and Extract Solution +- Call `status = solver.Solve(model)`. +- Check if `status == cp_model.OPTIMAL` or `status == cp_model.FEASIBLE`. +- Extract assignment: for each item `i`, get `solver.Value(color[i])`. +- Count used groups: `sum(1 for k in K if solver.Value(used[k]) > 0)`. + +### Step 5 - Validate and Output +- Verify no conflict pair shares the same group by checking the extracted assignment. +- Output a JSON payload with `status`, `objective` (as float), and the assignment dictionary. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# Problem data +items = list(range(num_items)) +conflict_pairs = [(u, v), ...] # list of tuples +max_groups = len(items) +groups = list(range(max_groups)) + +model = cp_model.CpModel() + +# Variables +color = {} +for i in items: + color[i] = model.NewIntVar(0, max_groups - 1, f'color_{i}') +used = {} +for k in groups: + used[k] = model.NewBoolVar(f'used_{k}') +eq = {} +for i in items: + for k in groups: + eq[i, k] = model.NewBoolVar(f'eq_{i}_{k}') + +# Constraints +for u, v in conflict_pairs: + model.Add(color[u] != color[v]) + +for i in items: + for k in groups: + model.Add(color[i] == k).OnlyEnforceIf(eq[i, k]) + model.Add(color[i] != k).OnlyEnforceIf(eq[i, k].Not()) + model.Add(used[k] == 1).OnlyEnforceIf(eq[i, k]) + +# Objective +model.Minimize(sum(used[k] for k in groups)) + +# Solve +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 8 +status = solver.Solve(model) + +if status == cp_model.OPTIMAL or status == cp_model.FEASIBLE: + obj_val = float(sum(1 for k in groups if solver.Value(used[k]) > 0)) + assignment = {i: solver.Value(color[i]) for i in items} + # Verify no conflicts + for u, v in conflict_pairs: + assert assignment[u] != assignment[v], f"Conflict violation: {u}, {v}" + print(f"RESULT:{obj_val}") +else: + print('{"status": "failed", "reason": "infeasible_or_timeout"}') +``` + +### Common Pitfalls +- Using `model.Add(color[i] == k)` without `OnlyEnforceIf`, which forces the solver to assign item `i` to group `k` unconditionally. +- Not creating the `eq[i,k]` variables for all `(i,k)` pairs, which can lead to missing constraints that link group usage. +- Forgetting to set `OnlyEnforceIf(eq[i,k].Not())` for the negative case, which can cause the solver to incorrectly infer `color[i] == k` when `eq[i,k]` is false. diff --git a/skills/optskills/skill_library/minimum_unique_label_assignment_via_graph_coloring.md b/skills/optskills/skill_library/minimum_unique_label_assignment_via_graph_coloring.md new file mode 100644 index 0000000..676dc29 --- /dev/null +++ b/skills/optskills/skill_library/minimum_unique_label_assignment_via_graph_coloring.md @@ -0,0 +1,277 @@ +--- +name: Minimum Unique Label Assignment via Graph Coloring +description: | + Models a conflict graph where each vertex must be assigned a unique label (e.g., frequency) such that adjacent vertices receive different labels, and minimizes the total number of distinct labels used. + +--- +# Workflow 1 (MILP with Binary Assignment Variables) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) using binary assignment variables for each vertex-label pair and binary usage variables for each label. The objective minimizes the sum of used labels, with constraints ensuring each vertex gets exactly one label and no adjacent vertices share a label. + +### Step 1 - Define Sets and Parameters +- Define the set of vertices `V`. +- Define the set of edges `E` representing conflicts between vertex pairs. +- Define the set of possible labels `K` with cardinality equal to `|V|` (a safe upper bound). + +### Step 2 - Create Decision Variables +- Create binary variable `x[i,k]` for each vertex `i` and label `k`, equal to 1 if vertex `i` is assigned label `k`. +- Create binary variable `y[k]` for each label `k`, equal to 1 if label `k` is used by any vertex. + +### Step 3 - Enforce Assignment Completeness +- For each vertex `i`, add constraint: `sum_{k in K} x[i,k] == 1`. + +### Step 4 - Enforce Neighbor Conflict Constraints +- For each edge `(i,j)` in `E` and each label `k` in `K`, add constraint: `x[i,k] + x[j,k] <= 1`. + +### Step 5 - Link Usage Variables +- For each vertex `i` and label `k`, add constraint: `x[i,k] <= y[k]`. + +### Step 6 - Set Objective +- Minimize the sum of `y[k]` over all labels `k`. + +### Formulation Template +```json +{ + "sets": ["V: vertices", "E: edges (conflicts)", "K: labels (size = |V|)"], + "parameters": [], + "decision_variables": [ + "x[i,k] binary: 1 if vertex i assigned label k", + "y[k] binary: 1 if label k is used" + ], + "objective": { + "sense": "min", + "expression": "sum_{k in K} y[k]" + }, + "constraints": [ + "sum_{k in K} x[i,k] == 1, for all i in V", + "x[i,k] + x[j,k] <= 1, for all (i,j) in E, k in K", + "x[i,k] <= y[k], for all i in V, k in K" + ] +} +``` + +### Common Pitfalls +- Setting `|K|` too small, making the model infeasible when a feasible solution exists with more labels. +- Forgetting to link `x` and `y` variables, causing the solver to use labels without counting them in the objective. +- Using continuous variables instead of binary, which can lead to fractional assignments. + +## Solving stage + +### Strategy Overview +Use a MILP solver (e.g., Gurobi, CPLEX, or SCIP) with binary variables. Set a time limit and zero MIP gap for optimality. After solving, extract the objective value and decode the assignment for each vertex. + +### Step 1 - Initialize Solver and Model +- Import the solver library (e.g., `gurobipy`, `pulp`, or `ortools`). +- Create a model instance and set parameters: time limit (e.g., `[TIME_LIMIT]` seconds), MIP gap (0), random seed. + +### Step 2 - Build and Solve +- Add variables and constraints as defined in the modeling stage. +- Call the solve method. + +### Step 3 - Check Status and Extract Results +- Verify solver status is `optimal` or `feasible`. +- Extract the objective value as the minimum number of unique labels. +- For each vertex `i`, find the label `k` where `x[i,k].X > 0.5`. + +### Step 4 - Validate Solution +- Iterate over all edges `(i,j)` and confirm `label[i] != label[j]`. +- Confirm every vertex has exactly one label assigned. + +### Code Usage +```python +import pulp + +# Define data +vertices = list(range(num_vertices)) +edges = [(u, v) for u, v in conflict_pairs] +labels = list(range(num_vertices)) # upper bound + +# Create model +model = pulp.LpProblem("MinLabels", pulp.LpMinimize) + +# Variables +x = pulp.LpVariable.dicts("x", (vertices, labels), cat="Binary") +y = pulp.LpVariable.dicts("y", labels, cat="Binary") + +# Objective +model += pulp.lpSum(y[k] for k in labels) + +# Constraints +for i in vertices: + model += pulp.lpSum(x[i][k] for k in labels) == 1 + +for (i, j) in edges: + for k in labels: + model += x[i][k] + x[j][k] <= 1 + +for i in vertices: + for k in labels: + model += x[i][k] <= y[k] + +# Solve +model.solve(pulp.PULP_CBC_CMD(msg=False, timeLimit=[TIME_LIMIT])) + +# Check status +if pulp.LpStatus[model.status] in ["Optimal", "Feasible"]: + min_labels = int(pulp.value(model.objective)) + assignment = {} + for i in vertices: + for k in labels: + if pulp.value(x[i][k]) > 0.5: + assignment[i] = k + break + # Validate + for (u, v) in edges: + assert assignment[u] != assignment[v], f"Conflict at edge ({u},{v})" +else: + raise RuntimeError(f"Solver failed with status: {pulp.LpStatus[model.status]}") +``` + +### Common Pitfalls +- Not checking solver status before accessing variable values, causing runtime errors. +- Assuming the solver will always return an optimal solution; always handle infeasible or unbounded statuses. +- Using a time limit too short for large instances, resulting in suboptimal or no solution. + +# Workflow 2 (Constraint Programming with CP-SAT) + +## Modeling stage + +### Strategy Overview +Formulate as a Constraint Satisfaction/Optimization Problem (CSP/COP) using integer variables for each vertex's label assignment. Use pairwise inequality constraints on edges. Minimize the number of distinct labels by introducing auxiliary boolean variables to track label usage and an objective variable for the count. + +### Step 1 - Define Sets and Parameters +- Define the set of vertices `V`. +- Define the set of edges `E`. +- Define an upper bound `K_max` equal to `|V|`. + +### Step 2 - Create Decision Variables +- Create integer variable `label[i]` for each vertex `i`, with domain `[0, K_max-1]`. +- Create boolean variable `used[k]` for each possible label `k`, indicating if any vertex uses that label. +- Create integer variable `num_labels` representing the count of distinct labels used. + +### Step 3 - Enforce Neighbor Conflict Constraints +- For each edge `(i,j)` in `E`, add constraint: `label[i] != label[j]`. + +### Step 4 - Link Used Variables to Labels +- For each vertex `i` and label `k`, create an intermediate boolean variable `is_equal[i][k]`. +- Add constraints: `label[i] == k` implies `is_equal[i][k] == true`, and `label[i] != k` implies `is_equal[i][k] == false`. +- For each label `k`, enforce `used[k] >= is_equal[i][k]` for all vertices `i`. This ensures `used[k]` is true if any vertex uses label `k`. +- Set `num_labels` equal to the sum of `used[k]`. + +### Step 5 - Set Objective +- Minimize `num_labels`. + +### Formulation Template +```json +{ + "sets": ["V: vertices", "E: edges (conflicts)"], + "parameters": ["K_max: upper bound on labels (|V|)"], + "decision_variables": [ + "label[i] integer in [0, K_max-1]: assigned label for vertex i", + "is_equal[i][k] boolean: true if label[i] == k", + "used[k] boolean: true if label k is used", + "num_labels integer: total distinct labels used" + ], + "objective": { + "sense": "min", + "expression": "num_labels" + }, + "constraints": [ + "label[i] != label[j], for all (i,j) in E", + "label[i] == k => is_equal[i][k] == true, for all i in V, k in [0, K_max-1]", + "label[i] != k => is_equal[i][k] == false, for all i in V, k in [0, K_max-1]", + "used[k] >= is_equal[i][k], for all i in V, k in [0, K_max-1]", + "num_labels == sum_{k} used[k]" + ] +} +``` + +### Common Pitfalls +- Using a domain for `label[i]` that is too small, causing infeasibility. +- Not linking `used[k]` to `label[i]` properly, leading to an incorrect count of distinct labels. +- Forgetting that CP-SAT requires integer variables; using boolean variables for `used[k]` and `is_equal[i][k]` is fine. + +## Solving stage + +### Strategy Overview +Use OR-Tools CP-SAT solver, which is well-suited for combinatorial optimization with integer variables. Set a time limit and enable solution logging if needed. After solving, extract the objective value and decode the assignment. + +### Step 1 - Initialize Solver and Model +- Import `ortools.sat.python.cp_model`. +- Create a `CpModel()` instance. + +### Step 2 - Build and Solve +- Add variables and constraints as defined. +- Add the objective to minimize `num_labels`. +- Create a `CpSolver()` and set parameters: time limit (`[TIME_LIMIT]` seconds), number of search workers. + +### Step 3 - Check Status and Extract Results +- Check solver status: `OPTIMAL` or `FEASIBLE`. +- Extract `num_labels` value and `label[i]` for each vertex. + +### Step 4 - Validate Solution +- Verify all edge constraints are satisfied. +- Confirm the number of distinct labels matches the objective value. + +### Step 5 - Prove Optimality via Feasibility Check (Optional) +- After finding a solution with `K` colors, to prove optimality, attempt to solve the problem with `K-1` colors (by adding a constraint `num_labels <= K-1`). If the solver returns `INFEASIBLE`, then `K` is optimal. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# Define data +vertices = list(range(num_vertices)) +edges = [(u, v) for u, v in conflict_pairs] +K_max = num_vertices + +# Create model +model = cp_model.CpModel() + +# Variables +label = [model.NewIntVar(0, K_max - 1, f"label_{i}") for i in vertices] +used = [model.NewBoolVar(f"used_{k}") for k in range(K_max)] +num_labels = model.NewIntVar(0, K_max, "num_labels") + +# Conflict constraints +for (u, v) in edges: + model.Add(label[u] != label[v]) + +# Link used to labels +for k in range(K_max): + vertex_uses_k = [] + for i in vertices: + is_equal = model.NewBoolVar(f"eq_{i}_{k}") + model.Add(label[i] == k).OnlyEnforceIf(is_equal) + model.Add(label[i] != k).OnlyEnforceIf(is_equal.Not()) + vertex_uses_k.append(is_equal) + for i in vertices: + model.Add(used[k] >= vertex_uses_k[i]) + +model.Add(num_labels == sum(used)) +model.Minimize(num_labels) + +# Solve +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = 8 +status = solver.Solve(model) + +# Check status +if status in [cp_model.OPTIMAL, cp_model.FEASIBLE]: + min_labels = solver.Value(num_labels) + assignment = {i: solver.Value(label[i]) for i in vertices} + # Validate + for (u, v) in edges: + assert assignment[u] != assignment[v], f"Conflict at edge ({u},{v})" +else: + raise RuntimeError(f"Solver failed with status: {solver.StatusName(status)}") +``` + +### Common Pitfalls +- Not setting a time limit, causing the solver to run indefinitely on hard instances. +- Using `OnlyEnforceIf` incorrectly with boolean variables; ensure the implication direction is correct. +- Forgetting to check for `FEASIBLE` status in addition to `OPTIMAL` when a time limit is used. diff --git a/skills/optskills/skill_library/minimumcostcirculation.md b/skills/optskills/skill_library/minimumcostcirculation.md new file mode 100644 index 0000000..1074c7b --- /dev/null +++ b/skills/optskills/skill_library/minimumcostcirculation.md @@ -0,0 +1,207 @@ +--- +name: MinimumCostCirculation +description: | + Model and solve minimum-cost circulation problems on directed networks with flow conservation, arc bounds, and linear costs using either a direct solver API or a modeling framework. + +--- +# Workflow 1 (Direct Solver API - OR-Tools) + +## Modeling stage + +### Strategy Overview +Structure the problem as a linear program (LP) by directly creating variables with bounds and adding flow conservation constraints node-by-node. This approach uses a solver's native API for efficient, low-overhead model construction. + +### Step 1 - Define Network Data Structure +- Store nodes as a list of unique identifiers. +- Store arcs as a list of tuples `(from_node, to_node)`. +- Create parallel lists or dictionaries for arc properties: `cost`, `lower_bound`, `upper_bound`. + +### Step 2 - Create Variables with Embedded Bounds +- For each arc, create a continuous solver variable `flow[arc]`. +- Directly set the variable's lower and upper bounds using the provided `lower_bound[arc]` and `upper_bound[arc]` during creation. + +### Step 3 - Formulate Flow Conservation Constraints +- For each node `n`, identify all arcs where `to_node == n` (inflow) and where `from_node == n` (outflow). +- Add a linear constraint: `sum(inflow_flows) == sum(outflow_flows)`. + +### Step 4 - Define Linear Objective +- Set the objective to minimize `sum(cost[arc] * flow[arc] for arc in arcs)`. + +### Formulation Template +```json +{ + "sets": [ + {"name": "nodes", "description": "Set of all nodes in the network."}, + {"name": "arcs", "description": "Set of directed arcs, each defined as (from_node, to_node)."} + ], + "parameters": [ + {"name": "cost", "set": "arcs", "description": "Unit cost of flow per arc."}, + {"name": "lower_bound", "set": "arcs", "description": "Minimum required flow on an arc."}, + {"name": "upper_bound", "set": "arcs", "description": "Maximum allowed flow on an arc."} + ], + "decision_variables": [ + {"name": "flow", "set": "arcs", "type": "continuous", "description": "Amount of flow on each arc."} + ], + "objective": { + "sense": "min", + "expression": "sum(cost[a] * flow[a] for a in arcs)" + }, + "constraints": [ + {"name": "flow_conservation", "set": "nodes", "expression": "sum(flow[a] for a in arcs if arc_to[a]==n) == sum(flow[a] for a in arcs if arc_from[a]==n)"}, + {"name": "arc_bounds", "set": "arcs", "expression": "lower_bound[a] <= flow[a] <= upper_bound[a]"} + ] +} +``` + +### Common Pitfalls +- Forgetting to handle arcs with no explicit upper bound, leading to unbounded variables. Always define a finite upper bound, even if large. +- Incorrectly identifying inflow/outflow arcs due to mismatched node indexing. Use consistent data structures. +- Creating duplicate constraints for nodes with no incident arcs, which may cause solver errors. Filter nodes based on incident arcs. + +## Solving stage + +### Strategy Overview +Use a dedicated LP solver (like GLOP) through its Python API. Focus on building the model efficiently, solving, and rigorously checking the solution status before extracting results. + +### Step 1 - Initialize Solver and Build Model +- Create a solver instance (e.g., `pywraplp.Solver.CreateSolver('GLOP')`). +- Follow the modeling steps to add variables, constraints, and the objective. + +### Step 2 - Solve and Check Status +- Call `solver.Solve()`. +- Check the return status. Accept `OPTIMAL` or `FEASIBLE`. Handle `INFEASIBLE`, `UNBOUNDED`, or `ABNORMAL` statuses with informative error messages. + +### Step 3 - Extract and Verify Solution +- If the status is acceptable, retrieve the objective value. +- Retrieve variable values for all arcs. Optionally, filter and print only active arcs (flow > tolerance). +- Programmatically verify key properties: flow conservation at each node and adherence to arc bounds within a small tolerance. + +### Step 4 - Output Results +- Format the output as required (e.g., a simple `print(f"RESULT:{objective_value}")` or a JSON dictionary of flows). +- Ensure the output is parseable by downstream processes. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('GLOP') +flow_vars = {} +for a in arcs: + flow_vars[a] = solver.NumVar(lower_bound[a], upper_bound[a], f'flow_{a}') +# ... add constraints and objective + +# solve with status / termination checks +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + objective_value = solver.Objective().Value() + solution = {a: flow_vars[a].solution_value() for a in arcs} +else: + raise Exception(f"Solver failed with status: {status}") +``` + +### Common Pitfalls +- Assuming `FEASIBLE` status guarantees optimality. It does not; it only confirms a feasible solution was found. +- Not using a tolerance (e.g., `1e-6`) when checking flow conservation equality due to floating-point arithmetic. +- Extracting variable values without first confirming a successful solve status. + +# Workflow 2 (Modeling Framework - Pyomo) + +## Modeling stage + +### Strategy Overview +Use a modeling framework (Pyomo) to declaratively define sets, parameters, variables, and constraints. This approach separates the problem formulation from the solver interface, improving readability and maintainability for complex networks. + +### Step 1 - Define Abstract Sets and Parameters +- Declare `pyo.Set` objects for `nodes` and `arcs` (with `dimen=2`). +- Declare `pyo.Param` objects for `cost`, `lower_bound`, and `upper_bound`, indexed by `arcs`. + +### Step 2 - Create Variables with Rule-Based Bounds +- Create a `pyo.Var` object `flow` indexed by `arcs` within the `NonNegativeReals` domain. +- Use a rule or loop to set the `lower_bound` and `upper_bound` attributes for each variable based on the corresponding parameters. + +### Step 3 - Declare Flow Conservation Constraints +- Define a `pyo.Constraint` rule indexed by `nodes`. +- Within the rule for a node `n`, sum flows on arcs where the destination is `n` (inflow) and equate it to the sum of flows on arcs where the origin is `n` (outflow). + +### Step 4 - Define the Objective +- Create a `pyo.Objective` rule to minimize the sum of `cost[arc] * flow[arc]` over all arcs. + +### Formulation Template +```json +{ + "sets": [ + {"name": "N", "description": "Set of nodes."}, + {"name": "A", "description": "Set of directed arcs (i,j).", "dimen": 2} + ], + "parameters": [ + {"name": "c", "set": "A", "description": "Cost per unit flow on arc."}, + {"name": "l", "set": "A", "description": "Lower bound for flow on arc."}, + {"name": "u", "set": "A", "description": "Upper bound (capacity) for flow on arc."} + ], + "decision_variables": [ + {"name": "x", "set": "A", "type": "continuous", "description": "Flow on each arc."} + ], + "objective": { + "sense": "min", + "expression": "sum(c[a] * x[a] for a in A)" + }, + "constraints": [ + {"name": "balance", "set": "N", "expression": "sum(x[(i,n)] for (i,n) in A) == sum(x[(n,j)] for (n,j) in A)"} + ] +} +``` + +### Common Pitfalls +- Defining parameter dictionaries with missing keys for some arcs, causing KeyError during model instantiation. Ensure all arcs have defined values, using `.get()` with defaults if needed. +- Inefficient constraint rules that iterate over all arcs for every node, leading to O(|N|*|A|) complexity. Use pre-computed dictionaries mapping nodes to incident arcs. +- Forgetting to call `model_instance = model.create_instance(data)` when using an abstract model, leaving the model un-initialized. + +## Solving stage + +### Strategy Overview +Use Pyomo's `SolverFactory` to interface with a capable LP solver (e.g., HiGHS, CBC). Configure solver options for performance and reliability, then solve the instance and meticulously inspect the results object. + +### Step 1 - Instantiate Model and Select Solver +- Create a concrete model instance with the provided network data. +- Initialize the solver: `solver = pyo.SolverFactory('highs')`. + +### Step 2 - Configure Solver and Solve +- Set solver options such as `time_limit`, `threads`, and `mip_rel_gap` (if applicable). +- Execute `results = solver.solve(model, tee=False)`. + +### Step 3 - Validate Solution Status +- Check `results.solver.status` is `SolverStatus.ok`. +- Check `results.solver.termination_condition` is `TerminationCondition.optimal` or `TerminationCondition.feasible`. +- If status is not ok or termination is not acceptable, analyze logs and raise an error. + +### Step 4 - Extract and Process Solution +- Load the solution into the model instance: `model.solutions.load_from(results)`. +- Access the objective value via `pyo.value(model.obj)`. +- Iterate over the `flow` variable to extract values, applying a tolerance filter (e.g., `> 1e-6`) to identify active arcs. + +### Step 5 - Report and Verify +- Format the output (objective value and active flows). +- Optionally, run a post-solve verification script to check flow conservation and bound adherence programmatically. + +### Code Usage +```python +# build model from formulation +model = pyo.ConcreteModel() +model.N = pyo.Set(initialize=nodes) +model.A = pyo.Set(initialize=arcs, dimen=2) +model.x = pyo.Var(model.A, domain=pyo.NonNegativeReals, bounds=arc_bounds_rule) +# ... define constraints and objective + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') +results = solver.solve(model) +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in [pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible]): + objective_value = pyo.value(model.obj) +else: + raise Exception(f"Solver failed: {results.solver.termination_condition}") +``` + +### Common Pitfalls +- Confusing `SolverStatus` (communication status) with `TerminationCondition` (solution quality). Both must be checked. +- Attempting to access variable values (`pyo.value(model.x[arc])`) before loading the solution, which returns `None`. +- Not setting `tee=True` during initial debugging to see the solver's log output. diff --git a/skills/optskills/skill_library/multi_commodity_flow_allocation.md b/skills/optskills/skill_library/multi_commodity_flow_allocation.md new file mode 100644 index 0000000..6335f77 --- /dev/null +++ b/skills/optskills/skill_library/multi_commodity_flow_allocation.md @@ -0,0 +1,223 @@ +--- +name: Multi-Commodity Flow Allocation +description: | + Model and solve multi-source, multi-sink, multi-commodity flow problems with linear profit objectives and exact demand satisfaction using linear programming. +--- + +# Workflow 1 (Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +This workflow uses Pyomo's abstract modeling capabilities to define a multi-dimensional linear program. It structures the problem as a multi-commodity flow with explicit sets for sources, sinks, and commodities, enabling clear constraint expression and easy data integration. + +### Step 1 - Define Sets and Parameters +- Define three fundamental sets: `sources`, `sinks`, and `commodities`. +- Create parameter dictionaries: `profit[(source, commodity, sink)]` for unit profit and `demand[(sink, commodity)]` for required quantities. +- Use tuple-indexed dictionaries for efficient data lookup during model construction. + +### Step 2 - Create Decision Variables +- Instantiate a three-dimensional variable `x[source, commodity, sink]` representing the flow quantity. +- Set the variable domain to `pyo.NonNegativeReals` to enforce non-negativity constraints automatically. + +### Step 3 - Formulate the Objective Function +- Define a linear objective to maximize total profit: `sum(profit[s,c,m] * x[s,c,m] for all s,c,m)`. +- Set the objective sense to `pyo.maximize`. + +### Step 4 - Implement Demand Satisfaction Constraints +- For each sink-commodity pair, create a linear equality constraint: `sum(x[s,c,m] for s in sources) == demand[m,c]`. +- This ensures the total supply from all sources exactly matches the demand at each sink for each commodity. + +### Formulation Template +```json +{ + "sets": ["sources", "sinks", "commodities"], + "parameters": [ + {"name": "profit", "index": ["source", "commodity", "sink"], "type": "float"}, + {"name": "demand", "index": ["sink", "commodity"], "type": "float"} + ], + "decision_variables": [ + {"name": "x", "index": ["source", "commodity", "sink"], "domain": "NonNegativeReals"} + ], + "objective": { + "sense": "max", + "expression": "sum(profit[s,c,m] * x[s,c,m] for s in sources for c in commodities for m in sinks)" + }, + "constraints": [ + {"name": "demand_satisfaction", "index": ["sink", "commodity"], "expression": "sum(x[s,c,m] for s in sources) == demand[m,c]"} + ] +} +``` + +### Common Pitfalls +- Forgetting to initialize all required indices in parameter dictionaries, leading to KeyError during model building. +- Using inconsistent indexing order (e.g., `profit[source, sink, commodity]`) between parameters and variables, causing incorrect objective coefficients. +- Neglecting to set the variable domain to non-negative, which allows infeasible negative flows. + +## Solving stage + +### Strategy Overview +This stage solves the Pyomo model using the HiGHS or CBC LP solver, with robust status checking and solution validation. It includes configuration for performance and reliability, and extracts results into a structured format. + +### Step 1 - Configure and Execute the Solver +- Instantiate the solver factory (e.g., `SolverFactory("highs")` or `SolverFactory("cbc")`). +- Set practical options: enable presolve, set a time limit, and specify optimality gap target (e.g., `ratio=0.0`). +- Solve the model with `tee=False` for clean output unless debugging. + +### Step 2 - Verify Solver Status and Termination +- Check that the solver status is `SolverStatus.ok`. +- Verify the termination condition is `TerminationCondition.optimal` or `.feasible` before extracting results. +- If status is not ok or termination is not acceptable, handle the error and do not proceed to solution extraction. + +### Step 3 - Extract and Validate the Solution +- Extract the objective value using `pyo.value(model.obj)`. +- Iterate through all variables to collect values, filtering out near-zero allocations (e.g., `value > 1e-6`) for clarity. +- Programmatically verify that extracted flows satisfy all demand constraints within a small numerical tolerance. + +### Step 4 - Structure and Output Results +- Compose a result dictionary containing the status, objective value, and a dictionary of non-zero flows. +- Output the results in a structured format like JSON for easy parsing and integration. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Build model (following modeling stage steps) +model = pyo.ConcreteModel() +# ... define sets, parameters, variables, objective, constraints + +# Solve with status / termination checks +solver = pyo.SolverFactory("highs") # or "cbc" +solver.options["time_limit"] = 30 +solver.options["presolve"] = "on" +# For CBC: solver.options["seconds"] = 30; solver.options["ratio"] = 0.0 + +results = solver.solve(model, tee=False) + +status = results.solver.status +termination = results.solver.termination_condition + +if status == SolverStatus.ok and termination in {TerminationCondition.optimal, TerminationCondition.feasible}: + objective_value = float(pyo.value(model.obj)) + solution = {} + for idx in model.x: + val = pyo.value(model.x[idx]) + if val > 1e-6: + solution[idx] = val + # Verification logic here + final_result = {"status": "optimal", "objective": objective_value, "solution": solution} +else: + final_result = {"status": "failed", "message": f"Solver terminated with status: {status}, condition: {termination}"} +``` + +### Common Pitfalls +- Attempting to access variable values (`pyo.value`) before confirming the solver terminated successfully, which may raise errors. +- Not filtering near-zero values in the solution output, resulting in cluttered, unreadable results. +- Omitting verification of demand constraints, potentially missing numerical inaccuracies that indicate solver issues. + +# Workflow 2 (OR-Tools with GLOP) + +## Modeling stage + +### Strategy Overview +This workflow uses Google OR-Tools' linear solver wrapper (`pywraplp`) to construct and solve the model imperatively. It is well-suited for rapid prototyping and deployment in environments where Pyomo is not available, offering direct control over variable and constraint creation. + +### Step 1 - Initialize Solver and Data Structures +- Create a solver instance using `pywraplp.Solver.CreateSolver('GLOP')`. +- Organize input data as nested lists or dictionaries: `profit[source][commodity][sink]` and `demand[sink][commodity]`. + +### Step 2 - Create Variables with Bounds +- Use nested loops over sources, commodities, and sinks to create variables: `x[s][c][m] = solver.NumVar(0, solver.infinity(), name)`. +- Setting the lower bound to 0 and upper bound to infinity enforces non-negativity and lack of capacity limits. + +### Step 3 - Build Demand Satisfaction Constraints +- For each sink `m` and commodity `c`, create a constraint object: `constraint = solver.Constraint(demand[m][c], demand[m][c])`. +- This creates a linear equality constraint with both lower and upper bound set to the demand value. +- Add the coefficient from each source's corresponding variable to this constraint. + +### Step 4 - Define the Linear Objective +- Initialize the objective: `objective = solver.Objective()`. +- In nested loops, set each variable's coefficient in the objective using `objective.SetCoefficient(x[s][c][m], profit[s][c][m])`. +- Set the optimization sense to maximization: `objective.SetMaximization()`. + +### Formulation Template +```json +{ + "sets": ["sources", "sinks", "commodities"], + "parameters": [ + {"name": "profit", "index": ["source", "commodity", "sink"], "type": "float"}, + {"name": "demand", "index": ["sink", "commodity"], "type": "float"} + ], + "decision_variables": [ + {"name": "x", "index": ["source", "commodity", "sink"], "lower_bound": 0, "upper_bound": "infinity"} + ], + "objective": { + "sense": "max", + "expression": "sum(profit[s][c][m] * x[s][c][m] for s,c,m)" + }, + "constraints": [ + {"name": "demand_satisfaction", "index": ["sink", "commodity"], "expression": "sum(x[s][c][m] for s in sources) == demand[m][c]"} + ] +} +``` + +### Common Pitfalls +- Creating constraints in the wrong loop order, leading to missing coefficients or incorrectly defined constraints. +- Using `solver.infinity()` for an upper bound when there are actual capacity limits, which should be modeled explicitly. +- Not using descriptive variable names, making debugging difficult for larger instances. + +## Solving stage + +### Strategy Overview +This stage solves the model using the GLOP linear programming solver, extracts the solution, and performs validation. It focuses on efficient solution retrieval and provides a simple, functional output structure. + +### Step 1 - Solve and Check Basic Status +- Execute the solver with `solver.Solve()`. +- Check the result status: `pywraplp.Solver.OPTIMAL` or `FEASIBLE` indicates a successful solve. +- If the status is not acceptable, handle the failure appropriately without attempting to extract variable values. + +### Step 2 - Extract Objective and Solution Values +- Retrieve the objective value using `objective.Value()`. +- Iterate through all created variables, using `var.solution_value()` to get the flow quantity. +- Apply a tolerance (e.g., `value > 1e-6`) to filter out effectively zero allocations for a cleaner solution report. + +### Step 3 - Validate Solution Correctness +- Recompute the total supply for each sink-commodity pair from the extracted solution. +- Verify that each computed supply equals the corresponding demand within a small numerical tolerance (e.g., `abs(supply - demand) < 1e-5`). +- Recalculate the total profit from the solution and compare it to the reported objective value. + +### Step 4 - Format and Return Results +- Package the results into a dictionary containing the status, objective value, and a list or dictionary of non-zero allocations. +- The output should be easily serializable (e.g., to JSON) for downstream use. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# Build model from formulation +solver = pywraplp.Solver.CreateSolver('GLOP') +# ... create variables, constraints, and objective as per modeling stage + +# Solve with status / termination checks +status = solver.Solve() +result_status = {pywraplp.Solver.OPTIMAL: 'optimal', pywraplp.Solver.FEASIBLE: 'feasible'}.get(status, 'failed') + +if result_status in ['optimal', 'feasible']: + objective_value = solver.Objective().Value() + solution = {} + # Assume `variables` is a dict or list storing all created variable objects + for var_key, var in variables.items(): + val = var.solution_value() + if val > 1e-6: + solution[var_key] = val + # Add verification logic here + final_result = {"status": result_status, "objective": objective_value, "solution": solution} +else: + final_result = {"status": "failed", "message": f"Solver returned status code: {status}"} +``` + +### Common Pitfalls +- Assuming `solver.Solve()` always returns an optimal solution without checking the status code. +- Not using a tolerance when checking variable values, which may include extremely small non-zero values due to numerical precision. +- Failing to verify the solution against the original demand data, missing potential solver inaccuracies or modeling errors. diff --git a/skills/optskills/skill_library/multi_commodity_flow_allocation_2.md b/skills/optskills/skill_library/multi_commodity_flow_allocation_2.md new file mode 100644 index 0000000..b9d207d --- /dev/null +++ b/skills/optskills/skill_library/multi_commodity_flow_allocation_2.md @@ -0,0 +1,250 @@ +--- +name: Multi-Commodity Flow Allocation +description: | + Model and solve linear programs for allocating flows from multiple sources of multiple products to multiple destinations to maximize profit, with exact demand satisfaction and non-negative continuous variables. +--- + +# Workflow 1 (Exact Demand Fulfillment with HiGHS) + +## Modeling stage + +### Strategy Overview +Model the problem as a multi-source, multi-sink, multi-commodity flow linear program. Enforce exact demand satisfaction via equality constraints and maximize total profit. Use Pyomo's ConcreteModel with explicit set definitions and dictionary-based parameter indexing for clarity and maintainability. + +### Step 1 - Define Sets and Indices +- Define the sets of sources (`model.SOURCES`), products (`model.PRODUCTS`), and destinations (`model.DESTINATIONS`) using Pyomo's `Set` component. + +### Step 2 - Define Parameters +- Define a profit parameter `profit[source, product, destination]` representing the unit profit for each flow. +- Define a demand parameter `demand[destination, product]` representing the required quantity for each product at each destination. +- Use Pyomo's `Param` component initialized with nested dictionaries for intuitive multi-dimensional indexing. + +### Step 3 - Define Decision Variables +- Define a continuous, non-negative decision variable `flow[source, product, destination]` representing the quantity allocated. +- Use `Var` with `domain=pyo.NonNegativeReals` and initialize it over the Cartesian product of the defined sets. + +### Step 4 - Formulate Objective Function +- Formulate the objective to maximize total profit: `sum(profit[s, p, d] * flow[s, p, d] for s in SOURCES for p in PRODUCTS for d in DESTINATIONS)`. +- Attach it to the model as `model.obj` with `sense=pyo.maximize`. + +### Step 5 - Formulate Demand Constraints +- For each product and destination pair, create a linear equality constraint ensuring the sum of flows from all sources equals the demand. +- Formulate as: `sum(flow[s, p, d] for s in SOURCES) == demand[d, p]`. + +### Formulation Template +```json +{ + "sets": ["SOURCES", "PRODUCTS", "DESTINATIONS"], + "parameters": [ + {"name": "profit", "dimensions": ["SOURCES", "PRODUCTS", "DESTINATIONS"]}, + {"name": "demand", "dimensions": ["DESTINATIONS", "PRODUCTS"]} + ], + "decision_variables": [ + {"name": "flow", "dimensions": ["SOURCES", "PRODUCTS", "DESTINATIONS"], "domain": "NonNegativeReals"} + ], + "objective": { + "sense": "max", + "expression": "sum(profit[s, p, d] * flow[s, p, d] for all s, p, d)" + }, + "constraints": [ + {"name": "demand_satisfaction", "relation": "==", "expression": "sum(flow[s, p, d] for all s) == demand[d, p]", "for_all": ["d in DESTINATIONS", "p in PRODUCTS"]} + ] +} +``` + +### Common Pitfalls +- Using equality constraints (`==`) when the problem allows supply to exceed demand (`<=`), which can make the model infeasible if supply is insufficient. +- Not defining capacity constraints for sources, which may lead to unrealistic solutions if sources have production limits. +- Hardcoding data with nested lists instead of dictionaries, reducing readability and making the model harder to maintain. + +## Solving stage + +### Strategy Overview +Solve the LP using the HiGHS solver via Pyomo, configured for performance and reliability. Include robust solution status checking, extraction of non-zero results, and a verification step to ensure correctness. + +### Step 1 - Initialize Solver and Set Options +- Create a solver instance using `pyo.SolverFactory("highs")`. +- Configure solver options: set a time limit `[TIME_LIMIT]`, optimality gap tolerance to `0.0` for LP, and number of threads for parallel processing. + +### Step 2 - Solve and Check Status +- Execute the solve command on the model instance. +- Check if the solver status is `SolverStatus.ok` and the termination condition is `optimal` or `feasible`. Proceed only if both checks pass. + +### Step 3 - Extract and Process Solution +- Extract the objective function value using `pyo.value(model.obj)`. +- Iterate through all `model.flow` variables, collecting indices and values where the value exceeds a small tolerance (e.g., `1e-6`) to create a concise solution summary. + +### Step 4 - Verify Solution Correctness +- For each product and destination pair, recalculate the total flow from the solution and verify it matches the demand parameter within a small tolerance (e.g., `1e-6`). +- Optionally, recalculate the total profit from the extracted flows to verify against the solver's reported objective. + +### Step 5 - Output Results +- Print a human-readable summary including the objective value and key allocation statistics. +- Output a structured, machine-readable payload (e.g., JSON) containing the solve status, objective value, and the non-zero solution dictionary. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# ... (model building code as per Modeling Stage) + +# Initialize solver +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = [TIME_LIMIT] +solver.options['mip_rel_gap'] = 0.0 # For pure LP +solver.options['threads'] = [N_THREADS] + +# Solve +results = solver.solve(model, tee=False) # Set tee=True for solver log + +# Check status +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + obj_val = pyo.value(model.obj) + # Extract non-zero flows + solution = {} + for idx in model.flow: + val = pyo.value(model.flow[idx]) + if val > 1e-6: + solution[idx] = val + # Verification + for d in model.DESTINATIONS: + for p in model.PRODUCTS: + calculated_demand = sum(pyo.value(model.flow[s, p, d]) for s in model.SOURCES) + # Assert or log verification + # assert abs(calculated_demand - model.demand[d, p]) < 1e-6 + # Output + print(f"RESULT:{obj_val}") + # ... output JSON payload +else: + print(f"Solver failed. Status: {status}, Termination: {term}") +``` + +### Common Pitfalls +- Running redundant verification code that duplicates the solver's internal validation, adding unnecessary runtime. +- Not checking for alternative optimal solutions in degenerate problems (common in transportation-type LPs). +- Using Unicode characters in console output, which may cause encoding issues in some environments. + +# Workflow 2 (Exact Demand Fulfillment with CBC) + +## Modeling stage + +### Strategy Overview +This workflow follows the same multi-commodity flow formulation but emphasizes Pyomo's rule-based constraint definitions and parameter initialization with explicit multi-dimensional keys. It is designed for compatibility with the COIN-OR CBC solver. + +### Step 1 - Define Model and Sets +- Instantiate a `pyo.ConcreteModel()`. +- Define sets `model.S`, `model.P`, `model.M` for sources, products, and markets using `pyo.Set(initialize=...)`. + +### Step 2 - Initialize Parameters with Dictionaries +- Create parameter dictionaries `profit_dict` and `demand_dict` where keys are tuples `(source, product, market)` and `(market, product)` respectively. +- Use `pyo.Param(model.S, model.P, model.M, initialize=profit_dict)` and `pyo.Param(model.M, model.P, initialize=demand_dict, mutable=True)`. + +### Step 3 - Declare Variables with Rule-Based Domains +- Declare variable `model.x` indexed over `model.S * model.P * model.M`. +- Specify the domain as `domain=pyo.NonNegativeReals`. + +### Step 4 - Build Objective with Summation +- Define the objective rule as `pyo.Objective(expr=sum(model.profit[s,p,m] * model.x[s,p,m] for s in model.S for p in model.P for m in model.M), sense=pyo.maximize)`. + +### Step 5 - Build Constraints with Rule Functions +- Define a function `demand_rule(model, m, p)` that returns `sum(model.x[s,p,m] for s in model.S) == model.demand[m,p]`. +- Create the constraint using `pyo.Constraint(model.M, model.P, rule=demand_rule)`. + +### Formulation Template +```json +{ + "sets": ["S", "P", "M"], + "parameters": [ + {"name": "profit", "dimensions": ["S", "P", "M"], "structure": "dictionary with tuple keys"}, + {"name": "demand", "dimensions": ["M", "P"], "mutable": true} + ], + "decision_variables": [ + {"name": "x", "dimensions": ["S", "P", "M"], "domain": "NonNegativeReals"} + ], + "objective": { + "sense": "max", + "expression": "sum(profit[s, p, m] * x[s, p, m] for all s, p, m)" + }, + "constraints": [ + {"name": "demand_constraint", "relation": "==", "expression": "sum(x[s, p, m] for all s) == demand[m, p]", "for_all": ["m in M", "p in P"], "definition": "rule-based"} + ] +} +``` + +### Common Pitfalls +- Assuming unlimited supply from all sources without modeling capacity constraints, which can lead to infeasible or unrealistic solutions if data changes. +- Using equality constraints rigidly; consider if `<=` constraints are more appropriate if oversupply is allowed. +- Not verifying that all indices in constraint rules align perfectly with the dimensions of the variables and parameters. + +## Solving stage + +### Strategy Overview +Solve the model using the CBC solver, configured via Pyomo. Include explicit handling of solver failures and structured output of results and diagnostics. + +### Step 1 - Configure CBC Solver +- Create solver instance with `pyo.SolverFactory("cbc")`. +- Set solver options such as maximum seconds `[TIME_LIMIT]`, optimality gap (ratio) to `0.0`, and number of threads. + +### Step 2 - Execute Solve with Diagnostics +- Call `solver.solve(model)`. Consider using `tee=True` for small problems to see the solver log for diagnostics. +- Capture the results object. + +### Step 3 - Validate Solution Status +- Check `results.solver.status` and `results.solver.termination_condition`. +- Proceed only if status is `ok` and termination is `optimal` or `feasible`. Otherwise, handle the error. + +### Step 4 - Extract and Summarize Solution +- Retrieve the objective value. +- Iterate over the model's variable index set, storing variable values that are significantly greater than zero (e.g., `> 1e-6`). +- Optionally, compute aggregate statistics (e.g., total flow per source). + +### Step 5 - Verify and Report +- Perform a post-solve verification by iterating through the demand constraints and ensuring the sum of solution flows matches the demand within a small tolerance. +- Print a formatted summary of the solution, including status, objective, and a sample of allocations. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# ... (model building code as per Modeling Stage) + +# Configure solver +solver = pyo.SolverFactory('cbc') +solver.options['seconds'] = [TIME_LIMIT] +solver.options['ratio'] = 0.0 +solver.options['threads'] = [N_THREADS] + +# Solve +results = solver.solve(model, tee=False) + +# Validate and process +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + obj_val = pyo.value(model.obj) + # Build solution dictionary + sol = {idx: pyo.value(model.x[idx]) for idx in model.x if pyo.value(model.x[idx]) > 1e-6} + # Verification loop + for m in model.M: + for p in model.P: + flow_to_market = sum(pyo.value(model.x[s, p, m]) for s in model.S) + # Optional: print or assert verification + # print(f"Market {m}, Product {p}: Demand={model.demand[m,p]}, Supplied={flow_to_market}") + # Output + print(f"Solver terminated with {term}. Objective: {obj_val:.2f}") + # ... (structured output) +else: + print(f"Solve unsuccessful. Status: {status}, Termination: {term}") + # Handle failure (e.g., return empty result, raise warning) +``` + +### Common Pitfalls +- Solving the same problem twice in sequence without changing parameters, wasting computational resources. +- Not checking for multiple optimal solutions in degenerate linear programs. +- Hardcoding data structures in a way that makes the model difficult to adapt to different problem scales or data formats. diff --git a/skills/optskills/skill_library/multi_commodity_flow_min_max_utilization.md b/skills/optskills/skill_library/multi_commodity_flow_min_max_utilization.md new file mode 100644 index 0000000..85ce011 --- /dev/null +++ b/skills/optskills/skill_library/multi_commodity_flow_min_max_utilization.md @@ -0,0 +1,281 @@ +--- +name: Multi-Commodity Flow Min-Max Utilization +description: | + Models and solves a multi-commodity flow problem where the objective is to minimize the maximum arc utilization across the network, using either direct LP solver APIs or algebraic modeling frameworks. +--- + +# Workflow 1 (Direct LP Solver API) + +## Modeling stage + +### Strategy Overview +This workflow uses a direct solver API (e.g., Google OR-Tools) to build the optimization model programmatically. The min-max objective is linearized by introducing an auxiliary variable `z` representing the maximum utilization ratio across all arcs, then constraining each arc's total flow to be at most its capacity times `z`. + +### Step 1 - Define Sets and Parameters +- Define the set of directed arcs `(i,j)` with associated capacity `capacity[i,j]`. +- Define the set of commodities `k`, each with a source node `source[k]`, sink node `sink[k]`, and demand `demand[k]`. +- Represent undirected edges as two directed arcs with identical capacity. + +### Step 2 - Create Decision Variables +- For each directed arc `(i,j)` and each commodity `k`, create a continuous non-negative variable `flow_on_arc[i,j,k]`. +- Create a single continuous non-negative auxiliary variable `z` representing the maximum utilization ratio. + +### Step 3 - Formulate Flow Conservation Constraints +- For each commodity `k` and each node `n`: + - Compute outflow: sum of `flow_on_arc[i,j,k]` for arcs where `i == n`. + - Compute inflow: sum of `flow_on_arc[i,j,k]` for arcs where `j == n`. + - Enforce `outflow - inflow == demand[k]` if `n == source[k]`. + - Enforce `outflow - inflow == -demand[k]` if `n == sink[k]`. + - Enforce `outflow - inflow == 0` for all other nodes. + +### Step 4 - Formulate Capacity Constraints with Utilization +- For each directed arc `(i,j)`, constrain the total flow across all commodities to be at most `capacity[i,j] * z`. +- Use `solver.Add(sum_k flow_on_arc[i,j,k] <= capacity[i,j] * z)`. + +### Step 5 - Set Objective +- Set the objective to minimize `z` using `solver.Objective().SetCoefficient(z, 1)` and `solver.Objective().SetMinimization()`. + +### Formulation Template +```json +{ + "sets": ["ARCS: directed arcs (i,j)", "COMMODITIES: commodities k"], + "parameters": ["capacity[i,j]: capacity of arc (i,j)", "source[k]: source node for commodity k", "sink[k]: sink node for commodity k", "demand[k]: demand for commodity k"], + "decision_variables": ["flow_on_arc[i,j,k] >= 0: flow of commodity k on arc (i,j)", "z >= 0: maximum utilization ratio"], + "objective": { + "sense": "min", + "expression": "z" + }, + "constraints": [ + "flow_conservation: for each k, n: outflow - inflow == demand[k] if n==source[k], -demand[k] if n==sink[k], 0 otherwise", + "capacity_utilization: for each (i,j): sum_k flow_on_arc[i,j,k] <= capacity[i,j] * z" + ] +} +``` + +### Common Pitfalls +- Forgetting to represent undirected edges as two directed arcs, leading to missing flow directions. +- Using integer variables for flow when continuous variables are sufficient, unnecessarily increasing solve time. +- Misplacing the sign of demand in flow conservation (positive at source, negative at sink). + +## Solving stage + +### Strategy Overview +The model is solved using a linear programming solver (e.g., GLOP from OR-Tools) since all variables are continuous. The solver status is checked before extracting results, and arc utilization is computed post-solve for verification. + +### Step 1 - Select and Configure Solver +- Use `pywraplp.Solver.CreateSolver("GLOP")` for pure linear programming problems. +- Optionally set a time limit using `solver.SetTimeLimit([TIME_LIMIT_MS])` (milliseconds). + +### Step 2 - Solve the Model +- Call `status = solver.Solve()` to execute the optimization. +- Check the status: `status == pywraplp.Solver.OPTIMAL` or `status == pywraplp.Solver.FEASIBLE` before accessing results. + +### Step 3 - Extract and Validate Results +- Retrieve the optimal objective value using `z.solution_value()` and print with a clear prefix: `print(f"RESULT:{z.solution_value()}")`. +- For each arc, compute utilization as `sum_k flow_on_arc[i,j,k].solution_value() / capacity[i,j]` and verify it does not exceed the optimal `z` value. +- Identify bottleneck arcs where utilization equals the optimal `z` value (within a small tolerance). +- Validate flow conservation at source and sink nodes to ensure demands are met. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# Data definition (use placeholders) +arcs = [(0,1), (1,2), (0,2)] # list of (i,j) +capacity = {(0,1): [CAPACITY], (1,2): [CAPACITY], (0,2): [CAPACITY]} +commodities = [0, 1] +source = {0: 0, 1: 1} +sink = {0: 2, 1: 2} +demand = {0: [DEMAND], 1: [DEMAND]} +nodes = set([i for i,j in arcs] + [j for i,j in arcs]) + +# Create solver +solver = pywraplp.Solver.CreateSolver("GLOP") +if not solver: + raise Exception("Solver not created") + +# Decision variables +flow_on_arc = {} +for (i,j) in arcs: + for k in commodities: + flow_on_arc[(i,j,k)] = solver.NumVar(0, solver.infinity(), f'flow_{i}_{j}_{k}') +z = solver.NumVar(0, solver.infinity(), 'z') + +# Flow conservation constraints +for k in commodities: + for n in nodes: + outflow = sum(flow_on_arc[(i,j,k)] for (i,j) in arcs if i == n) + inflow = sum(flow_on_arc[(i,j,k)] for (i,j) in arcs if j == n) + if n == source[k]: + solver.Add(outflow - inflow == demand[k]) + elif n == sink[k]: + solver.Add(outflow - inflow == -demand[k]) + else: + solver.Add(outflow - inflow == 0) + +# Capacity constraints with utilization +for (i,j) in arcs: + solver.Add(sum(flow_on_arc[(i,j,k)] for k in commodities) <= capacity[(i,j)] * z) + +# Objective +solver.Minimize(z) + +# Solve +status = solver.Solve() +if status == pywraplp.Solver.OPTIMAL or status == pywraplp.Solver.FEASIBLE: + print(f"RESULT:{z.solution_value()}") + # Optional: print arc utilizations and identify bottlenecks + for (i,j) in arcs: + total_flow = sum(flow_on_arc[(i,j,k)].solution_value() for k in commodities) + util = total_flow / capacity[(i,j)] + print(f"Arc ({i},{j}): utilization = {util:.3f}") +else: + print("No feasible solution found") +``` + +### Common Pitfalls +- Not checking solver status before accessing solution values, leading to runtime errors. +- Using an MIP solver (e.g., CBC) when the problem is purely linear, causing unnecessary overhead. +- Forgetting to set a time limit for large instances, causing indefinite solve times. + +# Workflow 2 (Algebraic Modeling Framework) + +## Modeling stage + +### Strategy Overview +This workflow uses an algebraic modeling framework (e.g., Pyomo) to declare the optimization model declaratively. The min-max objective is linearized identically to Workflow 1, but the model is built using symbolic set indexing and constraint expressions, which improves readability and maintainability for larger problems. + +### Step 1 - Define Sets and Parameters +- Create Pyomo `Set` objects for directed arcs, commodities, and nodes. +- Create Pyomo `Param` objects for arc capacities, commodity source/sink nodes, and demand values. +- Represent undirected edges as two directed arcs in the arcs set. + +### Step 2 - Create Decision Variables +- Define a continuous non-negative variable `flow_on_arc` indexed by arcs and commodities using `pyomo.Var(arcs, commodities, domain=pyomo.NonNegativeReals)`. +- Define a continuous non-negative auxiliary variable `z` using `pyomo.Var(domain=pyomo.NonNegativeReals)`. + +### Step 3 - Formulate Flow Conservation Constraints +- For each commodity `k` and each node `n`, create a constraint using `pyomo.Constraint` that enforces: outflow minus inflow equals demand at the source, negative demand at the sink, and zero otherwise. +- Use conditional logic within the constraint rule to determine the right-hand side based on node type. + +### Step 4 - Formulate Capacity Constraints with Utilization +- For each directed arc `(i,j)`, create a constraint: sum of flows of all commodities on that arc is less than or equal to `capacity[i,j] * z`. +- Use `pyomo.Constraint(arcs, rule=lambda model, i, j: sum(model.flow_on_arc[i,j,k] for k in model.commodities) <= model.capacity[i,j] * model.z)`. + +### Step 5 - Set Objective +- Set the objective to minimize `z` using `pyomo.Objective(expr=model.z, sense=pyomo.minimize)`. + +### Formulation Template +```json +{ + "sets": ["ARCS: directed arcs (i,j)", "COMMODITIES: commodities k", "NODES: all network nodes"], + "parameters": ["capacity[i,j]: capacity of arc (i,j)", "source[k]: source node for commodity k", "sink[k]: sink node for commodity k", "demand[k]: demand for commodity k"], + "decision_variables": ["flow_on_arc[i,j,k] >= 0: flow of commodity k on arc (i,j)", "z >= 0: maximum utilization ratio"], + "objective": { + "sense": "min", + "expression": "z" + }, + "constraints": [ + "flow_conservation: for each k, n: outflow - inflow == demand[k] if n==source[k], -demand[k] if n==sink[k], 0 otherwise", + "capacity_utilization: for each (i,j): sum_k flow_on_arc[i,j,k] <= capacity[i,j] * z" + ] +} +``` + +### Common Pitfalls +- Using mutable parameters inside constraint rules without proper Pyomo `value()` calls, causing expression errors. +- Forgetting to declare all sets before using them in variable or constraint indexing. +- Using Python loops inside constraint rules that inadvertently create multiple constraints instead of a single indexed constraint. + +## Solving stage + +### Strategy Overview +The model is solved using an LP solver (e.g., HiGHS or GLPK) via Pyomo's solver interface. Solver status and termination condition are checked before extracting results, and a time limit is set to prevent indefinite solves. + +### Step 1 - Select and Configure Solver +- Use `pyomo.SolverFactory("highs")` or `pyomo.SolverFactory("glpk")` for LP problems. +- Set solver options such as time limit: `solver.options["time_limit"] = [TIME_LIMIT_SEC]` (seconds for HiGHS) or `solver.options["tmlim"] = [TIME_LIMIT_SEC]` (for GLPK). + +### Step 2 - Solve the Model +- Call `result = solver.solve(model, tee=False)` to execute the optimization. +- Check the solver status: `result.solver.status == pyomo.SolverStatus.ok` and `result.solver.termination_condition == pyomo.TerminationCondition.optimal` (or `feasible`). + +### Step 3 - Extract and Validate Results +- Retrieve the optimal objective value using `pyomo.value(model.z)` and print with a clear prefix: `print(f"RESULT:{pyomo.value(model.z)}")`. +- For each arc, compute utilization as `sum_k pyomo.value(model.flow_on_arc[i,j,k]) / pyomo.value(model.capacity[i,j])` and verify it does not exceed the optimal `z` value. +- Identify bottleneck arcs where utilization equals the optimal `z` value (within a small tolerance). +- If the solver fails, output a structured error message with solver status and termination condition. + +### Code Usage +```python +import pyomo.environ as pyomo + +# Data definition (use placeholders) +arcs_list = [(0,1), (1,2), (0,2)] +capacity_data = {(0,1): [CAPACITY], (1,2): [CAPACITY], (0,2): [CAPACITY]} +commodities_list = [0, 1] +source_data = {0: 0, 1: 1} +sink_data = {0: 2, 1: 2} +demand_data = {0: [DEMAND], 1: [DEMAND]} +nodes_list = list(set([i for i,j in arcs_list] + [j for i,j in arcs_list])) + +# Create model +model = pyomo.ConcreteModel() + +# Sets +model.ARCS = pyomo.Set(initialize=arcs_list, dimen=2) +model.COMMODITIES = pyomo.Set(initialize=commodities_list) +model.NODES = pyomo.Set(initialize=nodes_list) + +# Parameters +model.capacity = pyomo.Param(model.ARCS, initialize=capacity_data) +model.source = pyomo.Param(model.COMMODITIES, initialize=source_data) +model.sink = pyomo.Param(model.COMMODITIES, initialize=sink_data) +model.demand = pyomo.Param(model.COMMODITIES, initialize=demand_data) + +# Decision variables +model.flow_on_arc = pyomo.Var(model.ARCS, model.COMMODITIES, domain=pyomo.NonNegativeReals) +model.z = pyomo.Var(domain=pyomo.NonNegativeReals) + +# Flow conservation constraints +def flow_conservation_rule(model, k, n): + outflow = sum(model.flow_on_arc[i,j,k] for (i,j) in model.ARCS if i == n) + inflow = sum(model.flow_on_arc[i,j,k] for (i,j) in model.ARCS if j == n) + if n == model.source[k]: + return outflow - inflow == model.demand[k] + elif n == model.sink[k]: + return outflow - inflow == -model.demand[k] + else: + return outflow - inflow == 0 +model.flow_conservation = pyomo.Constraint(model.COMMODITIES, model.NODES, rule=flow_conservation_rule) + +# Capacity constraints with utilization +def capacity_utilization_rule(model, i, j): + return sum(model.flow_on_arc[i,j,k] for k in model.COMMODITIES) <= model.capacity[i,j] * model.z +model.capacity_utilization = pyomo.Constraint(model.ARCS, rule=capacity_utilization_rule) + +# Objective +model.obj = pyomo.Objective(expr=model.z, sense=pyomo.minimize) + +# Solve +solver = pyomo.SolverFactory("highs") +solver.options["time_limit"] = [TIME_LIMIT_SEC] +result = solver.solve(model, tee=False) + +# Check status +if (result.solver.status == pyomo.SolverStatus.ok and + result.solver.termination_condition in [pyomo.TerminationCondition.optimal, pyomo.TerminationCondition.feasible]): + print(f"RESULT:{pyomo.value(model.z)}") + # Optional: print arc utilizations and identify bottlenecks + for (i,j) in model.ARCS: + total_flow = sum(pyomo.value(model.flow_on_arc[i,j,k]) for k in model.COMMODITIES) + util = total_flow / pyomo.value(model.capacity[i,j]) + print(f"Arc ({i},{j}): utilization = {util:.3f}") +else: + print(f"Solver failed: status={result.solver.status}, termination={result.solver.termination_condition}") +``` + +### Common Pitfalls +- Not converting Pyomo parameter values to floats when computing utilization (use `pyomo.value()`). +- Using `tee=True` in production code, which floods output with solver logs. +- Forgetting to handle the case where the solver returns feasible but not optimal (e.g., due to time limit), which may still provide a usable solution. diff --git a/skills/optskills/skill_library/multi_commodity_resource_allocation_with_shared_capacity.md b/skills/optskills/skill_library/multi_commodity_resource_allocation_with_shared_capacity.md new file mode 100644 index 0000000..b2b1528 --- /dev/null +++ b/skills/optskills/skill_library/multi_commodity_resource_allocation_with_shared_capacity.md @@ -0,0 +1,218 @@ +--- +name: Multi-Commodity Resource Allocation with Shared Capacity +description: | + Model and solve integer or linear programs for allocating multiple products across shared, capacity-constrained resources to maximize linear profit, using explicit demand limits and resource usage matrices. +--- + +# Workflow 1 (Pyomo with HiGHS/CBC for Integer Programming) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) using Pyomo's abstract modeling capabilities. This workflow is suited for problems requiring integer solutions (e.g., discrete units) and leverages open-source solvers like HiGHS or CBC for robust solving. + +### Step 1 - Define Sets and Parameters +- Define clear sets for commodities (e.g., products, packages) and resources (e.g., routes, machines). +- Store parameters in dictionaries: per-unit revenue, demand limit per commodity, and capacity per resource. +- Define a binary or coefficient matrix mapping each commodity to the resources it consumes. + +### Step 2 - Create Bounded Integer Variables +- Instantiate decision variables as `pyo.NonNegativeIntegers` for each commodity. +- Embed demand limits directly as variable upper bounds using the `bounds` argument or separate constraints. + +### Step 3 - Formulate Shared Capacity Constraints +- For each resource, create a linear inequality constraint. +- Sum the consumption across all commodities using the resource, weighted by the usage coefficient. +- Ensure the total does not exceed the resource's capacity. + +### Step 4 - Define Linear Profit Objective +- Create an objective to maximize total revenue: the sum of per-unit revenue multiplied by the decision variable for each commodity. + +### Formulation Template +```json +{ + "sets": ["commodities", "resources"], + "parameters": { + "revenue": {"index": "commodities", "type": "float"}, + "demand_limit": {"index": "commodities", "type": "float"}, + "capacity": {"index": "resources", "type": "float"}, + "usage": {"index": ["commodities", "resources"], "type": "float"} + }, + "decision_variables": [ + {"name": "x", "index": "commodities", "domain": "NonNegativeIntegers", "bounds": "(0, demand_limit)"} + ], + "objective": { + "sense": "max", + "expression": "sum(revenue[c] * x[c] for c in commodities)" + }, + "constraints": [ + {"name": "capacity", "index": "resources", "expression": "sum(usage[c, r] * x[c] for c in commodities) <= capacity[r]"} + ] +} +``` + +### Common Pitfalls +- Assuming missing resource usage data can be guessed; always require a complete usage matrix. +- Creating random or arbitrary constraint coefficients to force feasibility, which changes the fundamental problem. +- Setting an optimality gap (`mip_rel_gap`) to 0.0 for large problems without a time limit, causing excessive solve times. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the HiGHS or CBC solver via `SolverFactory`. Configure practical limits, verify solution status rigorously, and implement post-solve validation to ensure feasibility and correctness. + +### Step 1 - Configure and Execute Solver +- Instantiate the solver: `solver = pyo.SolverFactory("highs")` or `solver = pyo.SolverFactory("cbc")`. +- Set options: `time_limit` for runtime control, `mip_rel_gap` (or `ratio`) for optimality tolerance, and `threads` for parallelism. +- Execute the solve with `results = solver.solve(model, tee=False)`. + +### Step 2 - Verify Solver Status and Termination +- Check if `results.solver.status` is `SolverStatus.ok`. +- Check if `results.solver.termination_condition` is `TerminationCondition.optimal` or `TerminationCondition.feasible`. +- If not acceptable, analyze logs or infeasibility; do not proceed to extract a solution. + +### Step 3 - Extract and Validate Solution +- Extract the objective value: `obj_val = float(pyo.value(model.obj))`. +- Extract variable values, converting to integers if needed: `sol = {c: int(pyo.value(model.x[c])) for c in model.commodities}`. +- Programmatically verify all constraints: recalculate resource usage and compare against capacities with a small tolerance. + +### Step 4 - Report Structured Results +- Output key metrics: objective value, solution vector, and resource utilization percentages. +- For binding constraints (usage ≈ capacity), report dual values or shadow prices if available. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Build model (following formulation steps) +model = pyo.ConcreteModel() +# ... [model construction code] + +# Solve +solver = pyo.SolverFactory("highs") # or "cbc" +solver.options["time_limit"] = 30 +solver.options["mip_rel_gap"] = 0.0 +solver.options["threads"] = 4 +results = solver.solve(model, tee=False) + +# Status / termination checks +status = results.solver.status +term = results.solver.termination_condition +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + total_revenue = float(pyo.value(model.obj)) + solution = {c: int(pyo.value(model.x[c])) for c in model.commodities} + # ... [validation and output] +else: + # Handle failure, e.g., raise error or return status details + output = {"status": "failed", "termination_condition": str(term)} +``` + +### Common Pitfalls +- Trusting a non-optimal or unknown solver status and outputting pseudo numeric answers. +- Not verifying integer solution feasibility against constraints due to solver tolerances. +- Using the same solver settings across all problem scales without adjustment. + +# Workflow 2 (OR-Tools with SCIP/CBC for Direct API Control) + +## Modeling stage + +### Strategy Overview +Formulate the problem using Google's OR-Tools linear solver wrapper (`pywraplp`). This workflow provides direct API control, efficient constraint building via coefficient setting, and is well-suited for prototyping or deployment in environments where Pyomo is not available. + +### Step 1 - Initialize Solver and Define Data +- Create a solver instance: `solver = pywraplp.Solver.CreateSolver("SCIP")` or `"CBC_MIXED_INTEGER_PROGRAMMING"`. +- Define data arrays/lists for revenue, demand limits, capacities, and a sparse representation of the resource usage matrix. + +### Step 2 - Create Bounded Decision Variables +- For each commodity, create an integer variable with explicit lower and upper bounds: `solver.IntVar(0, demand_limit[i], f"x_{i}")`. +- This encodes non-negativity and demand limits directly. + +### Step 3 - Build Capacity Constraints Efficiently +- For each resource, create a constraint object with an upper bound: `constraint = solver.Constraint(0, capacity[r])`. +- Iterate through commodities and use the usage matrix to selectively set coefficients: `if usage[r][i] == 1: constraint.SetCoefficient(x[i], 1)`. + +### Step 4 - Set Linear Maximization Objective +- Create the objective: `objective = solver.Objective()`. +- Set all coefficients using `objective.SetCoefficient(x[i], revenue[i])`. +- Call `objective.SetMaximization()`. + +### Formulation Template +```json +{ + "sets": ["commodities", "resources"], + "parameters": { + "revenue": {"index": "commodities", "type": "float"}, + "demand_limit": {"index": "commodities", "type": "float"}, + "capacity": {"index": "resources", "type": "float"}, + "usage": {"index": ["resources", "commodities"], "type": "binary"} + }, + "decision_variables": [ + {"name": "x", "index": "commodities", "type": "IntVar", "bounds": "[0, demand_limit]"} + ], + "objective": { + "sense": "max", + "expression": "sum(revenue[i] * x[i] for i in commodities)" + }, + "constraints": [ + {"name": "capacity", "index": "resources", "expression": "sum(usage[r][i] * x[i] for i in commodities) <= capacity[r]"} + ] +} +``` + +### Common Pitfalls +- Modifying the problem structure (e.g., changing binary usage to fractional) to artificially relax constraints. +- Hardcoding random seeds for data generation in production code without validation. +- Iterating over all commodity-resource pairs without using a sparse representation, hurting performance for large problems. + +## Solving stage + +### Strategy Overview +Solve the model using the configured OR-Tools solver. Set appropriate limits, extract the solution, and perform verification. The direct API allows fine-grained control and immediate access to solution values. + +### Step 1 - Configure Solver Parameters +- Set a time limit: `solver.SetTimeLimit(30000)` for 30 seconds. +- Set the number of threads: `solver.SetNumThreads(4)`. +- For optimality gap, use solver-specific parameters: `solver.SetSolverSpecificParametersAsString("limits/gap=0.0001")`. + +### Step 2 - Execute Solve and Check Status +- Execute: `status = solver.Solve()`. +- Check for optimal or feasible status: `status == pywraplp.Solver.OPTIMAL` or `status == pywraplp.Solver.FEASIBLE`. +- If status is not acceptable, do not extract variable values. + +### Step 3 - Extract Solution and Compute Metrics +- Extract the objective value: `obj_val = objective.Value()`. +- Extract variable values: `sol = [x[i].solution_value() for i in range(num_commodities)]`. +- Compute resource usage for each constraint to verify feasibility. + +### Step 4 - Output Structured Results +- Return a dictionary or JSON containing status, objective value, solution vector, and constraint utilization. +- Include verification flags to indicate if all constraints are satisfied within tolerance. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# Build model (following formulation steps) +solver = pywraplp.Solver.CreateSolver("SCIP") +# ... [variable and constraint creation] + +# Configure solver +solver.SetTimeLimit(30000) +solver.SetNumThreads(4) + +# Solve with status / termination checks +status = solver.Solve() +if status in [pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE]: + total_revenue = solver.Objective().Value() + solution = [x[i].solution_value() for i in range(num_commodities)] + # ... [validation and output] +else: + # Handle failure + output = {"status": "failed", "solver_status": status} +``` + +### Common Pitfalls +- Treating solver infeasibility as a random seed issue rather than analyzing constraint feasibility. +- Not checking solver status before accessing `.solution_value()`, which can cause errors. +- Using the same time limit across all problem instances without considering complexity. diff --git a/skills/optskills/skill_library/multi_commodity_transportation_network_flow.md b/skills/optskills/skill_library/multi_commodity_transportation_network_flow.md new file mode 100644 index 0000000..923ab0b --- /dev/null +++ b/skills/optskills/skill_library/multi_commodity_transportation_network_flow.md @@ -0,0 +1,213 @@ +--- +name: Multi-Commodity Transportation Network Flow +description: | + Model and solve multi-product flow problems on capacitated networks, minimizing total transportation cost while respecting supply limits, demand requirements, and shared arc capacities. +--- + +# Workflow 1 (LP Solver with pywraplp) + +## Modeling stage + +### Strategy Overview +Model the problem as a pure Linear Program (LP) using a direct solver API (e.g., OR-Tools' `pywraplp`). This approach is procedural, builds the model coefficient-by-coefficient, and is well-suited for integration into scripts or applications requiring fine-grained control. + +### Step 1 - Define Data Structures +- Organize all input data as nested dictionaries or 2D/3D lists for consistent indexing. Use `supply[i][p]`, `demand[j][p]`, `cost[i][j][p]`, and `arc_capacity[i][j]`. +- Pre-validate data by checking total system balance: `sum(supply[i][p]) >= sum(demand[j][p])` for feasibility. + +### Step 2 - Create Decision Variables +- Instantiate a three-dimensional array of non-negative continuous variables `x[i][j][p]`. Use `solver.NumVar(0, solver.infinity(), f'x_{i}_{j}_{p}')` to create variables with descriptive names. + +### Step 3 - Build Supply and Demand Constraints +- For each origin `i` and product `p`, create a supply constraint: `sum_{j} x[i][j][p] <= supply[i][p]`. +- For each destination `j` and product `p`, create a demand constraint: `sum_{i} x[i][j][p] >= demand[j][p]`. + +### Step 4 - Build Arc Capacity Constraints +- For each origin-destination pair `(i, j)`, create a capacity constraint: `sum_{p} x[i][j][p] <= arc_capacity[i][j]`. + +### Step 5 - Set Objective Function +- Define the objective to minimize total cost: `sum_{i,j,p} cost[i][j][p] * x[i][j][p]`. Use `solver.Minimize()`. + +### Formulation Template +```json +{ + "sets": ["origins", "destinations", "products"], + "parameters": [ + "supply[origin][product]", + "demand[destination][product]", + "cost[origin][destination][product]", + "arc_capacity[origin][destination]" + ], + "decision_variables": ["flow[origin][destination][product] >= 0"], + "objective": { + "sense": "min", + "expression": "sum(cost[i][j][p] * flow[i][j][p])" + }, + "constraints": [ + "sum_j flow[i][j][p] <= supply[i][p] for all i, p", + "sum_i flow[i][j][p] >= demand[j][p] for all j, p", + "sum_p flow[i][j][p] <= arc_capacity[i][j] for all i, j" + ] +} +``` + +### Common Pitfalls +- Using inconsistent data structures (e.g., lists for supply but dicts for cost) leading to indexing errors. +- Forgetting to aggregate across the correct index in capacity constraints (should sum over products `p`). +- Not including origins with zero supply in the model structure, which can simplify data handling. + +## Solving stage + +### Strategy Overview +Solve the LP using a high-performance solver like GLOP. Implement robust status checking and post-solution verification to ensure the solution is both optimal and feasible within numerical tolerances. + +### Step 1 - Configure Solver +- Instantiate the solver: `solver = pywraplp.Solver.CreateSolver('GLOP')`. +- Set practical limits: `solver.SetTimeLimit(30000)` for a 30-second timeout. + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve()`. +- Check for acceptable statuses: `status in (solver.OPTIMAL, solver.FEASIBLE)`. Handle other statuses (e.g., `INFEASIBLE`, `UNBOUNDED`) with appropriate error messages. + +### Step 3 - Extract and Validate Solution +- If the status is acceptable, extract variable values: `flow_val = x[i][j][p].solution_value()`. +- Programmatically verify all constraints with a tolerance (e.g., `1e-6`). Calculate total supply used, demand met, and arc flows, comparing against limits. +- Compute and report aggregate statistics (e.g., total cost, supply utilization per product). + +### Step 4 - Output Results +- Print the objective value in a parseable format (e.g., `RESULT:{objective_value}`). +- Optionally, output detailed flow values or constraint violation reports for debugging. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('GLOP') +# ... (build variables, constraints, objective as per modeling stage) + +# solve with status / termination checks +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + objective_value = solver.Objective().Value() + # Validate solution + for i in origins: + for p in products: + total_shipped = sum(x[i][j][p].solution_value() for j in destinations) + if total_shipped > supply[i][p] + 1e-6: + print(f"Supply violation at ({i},{p})") + print(f'RESULT:{objective_value}') +else: + print('Solver failed with status:', status) +``` + +### Common Pitfalls +- Accepting a solution without numerical verification, potentially missing small constraint violations. +- Misinterpreting solver status codes (e.g., treating `FEASIBLE` as an error). +- Not setting a time limit, risking the solver hanging on large or pathological instances. + +# Workflow 2 (Modeling Language with Pyomo and HiGHS) + +## Modeling stage + +### Strategy Overview +Model the problem declaratively using Pyomo, separating model definition from solver invocation. This approach enhances readability, maintainability, and leverages automatic presolve reductions from solvers like HiGHS. + +### Step 1 - Declare Abstract Sets and Parameters +- Define Pyomo Sets for `origins`, `destinations`, and `products`. +- Define Pyomo Parameters for `supply`, `demand`, `cost`, and `arc_capacity` using nested dictionaries or rule-based initialization. + +### Step 2 - Define Decision Variables +- Declare a Pyomo `Var` indexed over the three sets: `model.x = pyo.Var(model.origins, model.destinations, model.products, domain=pyo.NonNegativeReals)`. + +### Step 3 - Construct Constraints via Rules +- Create a `ConstraintList` or use `pyo.Constraint` with rule functions for each constraint family. +- Supply rule: `def supply_rule(model, i, p): return sum(model.x[i, j, p] for j in model.destinations) <= model.supply[i, p]`. +- Demand rule: Use `>=` for demand satisfaction. +- Capacity rule: Sum over products `p` for each arc `(i, j)`. + +### Step 4 - Define the Objective +- Use `pyo.Objective(expr=sum(model.cost[i,j,p] * model.x[i,j,p] for i,j,p), sense=pyo.minimize)`. + +### Formulation Template +```json +{ + "sets": ["I (origins)", "J (destinations)", "P (products)"], + "parameters": [ + "supply[i,p] for i in I, p in P", + "demand[j,p] for j in J, p in P", + "cost[i,j,p] for i in I, j in J, p in P", + "capacity[i,j] for i in I, j in J" + ], + "decision_variables": ["x[i,j,p] >= 0 for i in I, j in J, p in P"], + "objective": { + "sense": "min", + "expression": "sum( cost[i,j,p] * x[i,j,p] )" + }, + "constraints": [ + "supply_con[i,p]: sum_j x[i,j,p] <= supply[i,p]", + "demand_con[j,p]: sum_i x[i,j,p] >= demand[j,p]", + "capacity_con[i,j]: sum_p x[i,j,p] <= capacity[i,j]" + ] +} +``` + +### Common Pitfalls +- Defining parameter rules that are overly complex or slow for large datasets; prefer dictionary initialization. +- Mixing up index order in constraint rules (e.g., summing over origins when destinations are required). +- Using equality (`=`) for demand constraints when the problem allows over-satisfaction; `>=` is more flexible. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the HiGHS LP solver via the `pyo.SolverFactory` interface. Configure solver options for performance and reliability, and implement a structured post-solution analysis. + +### Step 1 - Instantiate and Configure Solver +- Create solver object: `solver = pyo.SolverFactory('highs')`. +- Set options: `solver.options['time_limit'] = 30`, `solver.options['threads'] = 4`. For exact solutions, set `'mip_rel_gap' = 0.0` (relevant if extensions introduce integer variables). + +### Step 2 - Solve and Inspect Termination +- Execute `results = solver.solve(model, tee=False)`. +- Check termination condition: `results.solver.termination_condition` should be `optimal` or `feasible`. Also verify `results.solver.status` is `ok`. + +### Step 3 - Extract and Verify Solution +- Access the objective value: `obj_val = pyo.value(model.obj)`. +- Implement a verification function that iterates over all constraints, computing left-hand side values using `pyo.value(model.x[i,j,p])` and comparing to right-hand side limits with a tolerance. +- Report any violations and compute operational metrics (e.g., capacity utilization rates). + +### Step 4 - Output Structured Results +- Print the objective value in a consistent format (e.g., `RESULT:{obj_val}`). +- For non-optimal terminations, log the solver status and termination condition for debugging. + +### Code Usage +```python +# build model from formulation +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=origins) +model.J = pyo.Set(initialize=destinations) +model.P = pyo.Set(initialize=products) +# ... (define parameters, variables, constraints, objective as per modeling stage) + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = 30 +results = solver.solve(model) + +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in [pyo.TerminationCondition.optimal, + pyo.TerminationCondition.feasible]): + objective_value = pyo.value(model.obj) + # Perform verification + tolerance = 1e-6 + for i in model.I: + for p in model.P: + lhs = sum(pyo.value(model.x[i, j, p]) for j in model.J) + if lhs > model.supply[i, p] + tolerance: + print(f"Supply violation at ({i},{p})") + print(f'RESULT:{objective_value}') +else: + print('Solver failed:', results.solver.termination_condition) +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition`, leading to acceptance of failed solves. +- Using placeholder or incorrect data during verification (e.g., a different `cost` dictionary); always use the model's own parameters. +- Ignoring the benefits of solver presolve; trust the reduced problem statistics reported by HiGHS. diff --git a/skills/optskills/skill_library/multi_index_flow_allocation.md b/skills/optskills/skill_library/multi_index_flow_allocation.md new file mode 100644 index 0000000..b6434b7 --- /dev/null +++ b/skills/optskills/skill_library/multi_index_flow_allocation.md @@ -0,0 +1,272 @@ +--- +name: Multi-Index Flow Allocation +description: | + Model and solve linear profit maximization problems for allocating flows across multiple origins, types, and destinations with exact demand satisfaction. +--- + +# Workflow 1 (Google OR-Tools LP) + +## Modeling stage + +### Strategy Overview +Formulate the allocation problem as a pure Linear Program (LP) using the OR-Tools linear solver wrapper. This approach is suitable for continuous, non-negative flow variables with linear equality constraints and a linear objective. + +### Step 1 - Define Data Structures +- Organize input data in nested lists or dictionaries that align with the multi-dimensional nature of the problem (e.g., origins, product types, destinations). +- Store profit coefficients in a 3D structure `profit[origin][type][destination]` and demand requirements in a 2D structure `demand[type][destination]`. + +### Step 2 - Create Decision Variables +- Instantiate non-negative continuous variables for each flow path using `solver.NumVar(lb, ub, name)`. +- Use a naming convention that embeds indices (e.g., `f_o_t_d`) for traceability. +- Set lower bound to 0 and upper bound to `solver.infinity()` to enforce non-negativity without explicit capacity. + +### Step 3 - Formulate Demand Satisfaction Constraints +- For each product type and destination pair, create a linear equality constraint. +- Use `solver.Add(sum(variables) == demand_value)` to enforce that the total flow from all origins equals the exact demand. +- Iterate over all type-destination combinations to add the full set of constraints. + +### Step 4 - Define Linear Objective +- Create the objective expression using `solver.Objective()`. +- Iterate through all variables, setting their coefficients with `objective.SetCoefficient(var, profit_coefficient)`. +- Set the objective sense to maximization. + +### Formulation Template +```json +{ + "sets": [ + "origins (O)", + "types (T)", + "destinations (D)" + ], + "parameters": [ + "profit[o][t][d]: unit profit for flow from origin o of type t to destination d", + "demand[t][d]: required quantity of type t at destination d" + ], + "decision_variables": [ + "x[o][t][d] >= 0: flow quantity from origin o of type t to destination d" + ], + "objective": { + "sense": "max", + "expression": "sum_{o in O, t in T, d in D} profit[o][t][d] * x[o][t][d]" + }, + "constraints": [ + "demand_satisfaction[t in T, d in D]: sum_{o in O} x[o][t][d] == demand[t][d]" + ] +} +``` + +### Common Pitfalls +- Mismatching indices between profit data and variable creation loops, leading to incorrect objective coefficients. +- Forgetting to set the objective sense, defaulting to minimization. +- Using integer or boolean variable types for a continuous flow problem, unnecessarily complicating the solve. + +## Solving stage + +### Strategy Overview +Solve the LP model using the OR-Tools GLOP solver, which is designed for linear programming. Implement robust status checking, solution verification, and result extraction. + +### Step 1 - Initialize Solver and Solve +- Create the solver instance: `solver = pywraplp.Solver.CreateSolver('GLOP')`. +- Invoke `solver.Solve()` and capture the result status. + +### Step 2 - Check Solver Status +- Check the status against `solver.OPTIMAL` first, then `solver.FEASIBLE`. +- If status is not optimal or feasible, handle the error by reporting the status code and terminating gracefully. + +### Step 3 - Extract and Verify Solution +- Retrieve the objective value using `solver.Objective().Value()`. +- For each demand constraint, compute the sum of solution values for the relevant variables and compare to the demand parameter within a small tolerance (e.g., 1e-6). +- Optionally, compute a theoretical upper bound (e.g., sum of max profit per demand) to sense-check optimality. + +### Step 4 - Report Results +- Extract and print only non-zero flow variables (e.g., `solution_value() > 1e-6`) to reduce output clutter. +- Structure output for easy parsing, clearly stating the objective value and key allocations. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('GLOP') +if not solver: + raise Exception('Solver backend not available.') + +# Create variables +x = {} +for o in origins: + for t in types: + for d in destinations: + x[o,t,d] = solver.NumVar(0, solver.infinity(), f'x_{o}_{t}_{d}') + +# Add constraints +for t in types: + for d in destinations: + ct = solver.Constraint(demand[t][d], demand[t][d]) + for o in origins: + ct.SetCoefficient(x[o,t,d], 1) + +# Set objective +objective = solver.Objective() +for o in origins: + for t in types: + for d in destinations: + objective.SetCoefficient(x[o,t,d], profit[o][t][d]) +objective.SetMaximization() + +# solve with status / termination checks +status = solver.Solve() +if status == solver.OPTIMAL: + print(f'Optimal objective value = {objective.Value()}') + # Verification and result extraction + for t in types: + for d in destinations: + total_flow = sum(x[o,t,d].solution_value() for o in origins) + assert abs(total_flow - demand[t][d]) < 1e-6, f'Demand not met for {t},{d}' + # Print non-zero flows + for (o,t,d), var in x.items(): + val = var.solution_value() + if val > 1e-6: + print(f' {var.name()} = {val}') +elif status == solver.FEASIBLE: + print(f'Feasible solution found, objective = {objective.Value()}') +else: + print('No optimal or feasible solution found.') +``` + +### Common Pitfalls +- Assuming the solver status is `OPTIMAL` without checking, leading to errors when accessing `solution_value()` on an unsolved model. +- Not verifying constraint satisfaction numerically, which can mask modeling errors or solver inaccuracies. +- Using a loose tolerance for checking non-zero flows, potentially filtering out small but meaningful values. + +# Workflow 2 (Pyomo with Open-Source Solver) + +## Modeling stage + +### Strategy Overview +Model the problem using Pyomo's abstract or concrete modeling paradigm, defining sets, parameters, variables, and constraints in a declarative style. This separates the model definition from the solver interface, enhancing reusability. + +### Step 1 - Define Pyomo Sets and Parameters +- Create Pyomo `Set` objects for origins, product types, and destinations. +- Define `Param` components for profit and demand, indexed over the appropriate sets. Use dictionaries for initialization. + +### Step 2 - Declare Decision Variables +- Instantiate a `Var` component indexed over the Cartesian product of the sets. +- Specify `domain=pyo.NonNegativeReals` to enforce non-negativity. +- Optionally, set bounds here if capacity constraints exist. + +### Step 3 - Construct Demand Constraints +- Use Pyomo's `Constraint` component with a rule function. +- The rule should, for each fixed type and destination, return the equality expression: `sum(model.x[o,t,d] for o in model.origins) == model.demand[t,d]`. + +### Step 4 - Formulate the Objective +- Define an `Objective` component with a rule that sums `model.profit[o,t,d] * model.x[o,t,d]` over all indices. +- Set the `sense` to maximize. + +### Formulation Template +```json +{ + "sets": [ + "O: set of origins", + "T: set of product types", + "D: set of destinations" + ], + "parameters": [ + "profit[o in O, t in T, d in D]: unit profit", + "demand[t in T, d in D]: required quantity" + ], + "decision_variables": [ + "x[o in O, t in T, d in D] >= 0: flow quantity" + ], + "objective": { + "sense": "max", + "expression": "sum( profit[o,t,d] * x[o,t,d] for o in O, t in T, d in D )" + }, + "constraints": [ + "forall t in T, d in D: sum( x[o,t,d] for o in O ) == demand[t,d]" + ] +} +``` + +### Common Pitfalls +- Defining parameters as plain Python dictionaries instead of Pyomo `Param` objects, which prevents proper indexing in constraints. +- Incorrectly nesting summation loops in constraint rules, leading to scalar constraints instead of indexed ones. +- Not initializing all set elements before using them in parameter or variable declarations. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using an open-source LP solver (e.g., CBC, GLPK) via Pyomo's `SolverFactory`. Configure solver options, check termination conditions rigorously, and extract solution values safely. + +### Step 1 - Configure and Execute Solver +- Instantiate the solver: `solver = pyo.SolverFactory('solver_name')`. +- Set options like time limit (`seconds`) and optimality tolerance (`ratio` for CBC). +- Call `results = solver.solve(model, tee=False)`. + +### Step 2 - Inspect Solver Status and Termination Condition +- Check `results.solver.status` is `SolverStatus.ok`. +- Check `results.solver.termination_condition` is `TerminationCondition.optimal` (or `.feasible` for a sub-optimal but valid solution). +- If checks fail, output diagnostic information and do not proceed to value extraction. + +### Step 3 - Validate Solution and Extract Values +- Use `pyo.value(model.objective)` to get the objective value. +- Programmatically verify each demand constraint by summing `pyo.value(model.x[o,t,d])` for fixed t,d and comparing to the demand parameter. +- Iterate through variables to collect non-zero allocations. + +### Step 4 - Report and Handle Errors +- Print results in a structured format (e.g., key-value pairs or JSON) for automated parsing. +- In case of solver failure, output a JSON payload containing the status, termination condition, and any error messages. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo + +model = pyo.ConcreteModel() +model.O = pyo.Set(initialize=origins_list) +model.T = pyo.Set(initialize=types_list) +model.D = pyo.Set(initialize=destinations_list) + +model.profit = pyo.Param(model.O, model.T, model.D, initialize=profit_dict) +model.demand = pyo.Param(model.T, model.D, initialize=demand_dict) + +model.x = pyo.Var(model.O, model.T, model.D, domain=pyo.NonNegativeReals) + +def demand_rule(model, t, d): + return sum(model.x[o, t, d] for o in model.O) == model.demand[t, d] +model.demand_con = pyo.Constraint(model.T, model.D, rule=demand_rule) + +def obj_rule(model): + return sum(model.profit[o, t, d] * model.x[o, t, d] for o in model.O for t in model.T for d in model.D) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.maximize) + +# solve with status / termination checks +solver = pyo.SolverFactory('cbc') +solver.options['seconds'] = 30 +results = solver.solve(model) + +if results.solver.status == pyo.SolverStatus.ok: + if results.solver.termination_condition == pyo.TerminationCondition.optimal: + print(f'RESULT:{pyo.value(model.obj)}') + # Verification + for t in model.T: + for d in model.D: + total = sum(pyo.value(model.x[o, t, d]) for o in model.O) + assert abs(total - model.demand[t, d]) < 1e-6 + # Print non-zero flows + for index in model.x.index_set(): + val = pyo.value(model.x[index]) + if val > 1e-6: + print(f' x{index} = {val}') + elif results.solver.termination_condition == pyo.TerminationCondition.feasible: + print(f'Feasible solution found: {pyo.value(model.obj)}') + else: + print('Solver did not converge to an optimal solution.') + payload = {'status': str(results.solver.status), + 'termination': str(results.solver.termination_condition)} + print(f'RESULT_JSON:{json.dumps(payload)}') +else: + print('Solver failed.') +``` + +### Common Pitfalls +- Accessing `pyo.value()` on variables or objectives without first confirming a feasible solution exists, causing exceptions. +- Not setting a time limit for the solver, risking long runtimes on large instances. +- Confusing `SolverStatus` (ok/error) with `TerminationCondition` (optimal/feasible/infeasible). Both must be checked. diff --git a/skills/optskills/skill_library/multi_item_allocation_with_capacity_constraints.md b/skills/optskills/skill_library/multi_item_allocation_with_capacity_constraints.md new file mode 100644 index 0000000..34620ae --- /dev/null +++ b/skills/optskills/skill_library/multi_item_allocation_with_capacity_constraints.md @@ -0,0 +1,231 @@ +--- +name: Multi-Item Allocation with Capacity Constraints +description: | + Model and solve integer linear programs for allocating items under individual demand limits and multiple linear capacity constraints to maximize linear revenue. +--- + +# Workflow 1 (Direct Solver API - OR-Tools) + +## Modeling stage + +### Strategy Overview +This workflow uses a procedural, solver-centric API (e.g., OR-Tools) to directly construct the model. Variables and constraints are added imperatively, which is efficient for problems with a straightforward structure and allows for fine-grained control over variable bounds. + +### Step 1 - Define Sets and Parameters +- Declare the set of item types and the set of capacity constraints. +- Define parameters: revenue per item, individual demand limit per item, capacity limit per constraint, and a binary coefficient matrix indicating which items belong to which constraint. +- Use placeholders like `ITEMS`, `CONSTRAINTS`, `revenue`, `demand_limit`, `capacity_limit`, and `coeff_matrix`. + +### Step 2 - Create Decision Variables +- Instantiate non-negative integer variables for each item type. +- Directly incorporate individual upper bounds by setting the variable's upper bound to its demand limit during creation, which is more efficient than adding separate constraints. +- Use `solver.IntVar(lower, upper, name)`. + +### Step 3 - Formulate Capacity Constraints +- For each capacity constraint, create a linear inequality by summing the relevant variables. +- Use the binary coefficient matrix to determine which variables participate in each sum. +- Add the constraint to the solver: `solver.Add(sum(coeff[c][i] * x[i] for i in ITEMS) <= capacity_limit[c])`. + +### Step 4 - Define the Objective +- Create a linear expression for total revenue: sum of `revenue[i] * x[i]` for all items. +- Set the objective to maximize this expression using `solver.Maximize()`. + +### Formulation Template +```json +{ + "sets": ["ITEMS", "CONSTRAINTS"], + "parameters": [ + {"name": "revenue", "type": "float", "index": "ITEMS"}, + {"name": "demand_limit", "type": "int", "index": "ITEMS"}, + {"name": "capacity_limit", "type": "float", "index": "CONSTRAINTS"}, + {"name": "coeff_matrix", "type": "int", "index": ["CONSTRAINTS", "ITEMS"]} + ], + "decision_variables": [ + {"name": "x", "type": "integer_nonnegative", "index": "ITEMS", "bounds": [0, "demand_limit[i]"]} + ], + "objective": { + "sense": "max", + "expression": "sum(revenue[i] * x[i] for i in ITEMS)" + }, + "constraints": [ + {"name": "capacity", "expression": "sum(coeff_matrix[c][i] * x[i] for i in ITEMS) <= capacity_limit[c]", "index": "CONSTRAINTS"} + ] +} +``` + +### Common Pitfalls +- Forgetting to set variable upper bounds, leading to unbounded or unrealistic solutions. +- Incorrectly populating the coefficient matrix, which can silently create wrong constraints. +- Using floating-point numbers for capacity limits when integer limits are more appropriate, potentially causing precision issues. + +## Solving stage + +### Strategy Overview +Solve the model using a dedicated MIP solver (e.g., SCIP, CBC) via the OR-Tools wrapper. The focus is on configuring the solver for performance, robustly checking the solution status, and extracting and verifying the results. + +### Step 1 - Initialize and Configure Solver +- Create a solver instance: `solver = pywraplp.Solver.CreateSolver("SCIP")`. +- Set practical limits: `solver.SetTimeLimit(30000)` (in milliseconds) and `solver.SetNumThreads(4)`. +- Optionally set a relative optimality gap: `solver.SetRelativeGapTolerance(0.0)` for exact solutions. + +### Step 2 - Solve and Check Status +- Execute the solve: `status = solver.Solve()`. +- Check the result status against `solver.OPTIMAL` and `solver.FEASIBLE`. Handle `solver.INFEASIBLE` or `solver.UNBOUNDED` appropriately. + +### Step 3 - Extract and Verify Solution +- If the status is acceptable, retrieve the objective value: `obj_val = solver.Objective().Value()`. +- Extract variable values: `sol = {i: x[i].solution_value() for i in ITEMS}`. +- Programmatically verify that all capacity constraints and demand bounds are satisfied. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# 1. Initialize solver +solver = pywraplp.Solver.CreateSolver("SCIP") +if not solver: + raise Exception("Solver not available.") + +# 2. Configure solver (optional) +solver.SetTimeLimit(30000) # 30 seconds in milliseconds +solver.SetNumThreads(4) + +# 3. Build model (following Modeling stage steps) +# ... (Define variables, constraints, objective) + +# 4. Solve and check status +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + # 5. Extract solution + objective_value = solver.Objective().Value() + solution = {i: x[i].solution_value() for i in ITEMS} + # 6. Optional verification + for c in CONSTRAINTS: + lhs = sum(coeff_matrix[c][i] * solution[i] for i in ITEMS) + assert lhs <= capacity_limit[c] + 1e-6, f"Constraint {c} violated." + print(f"RESULT:{objective_value}") +else: + print(f"No feasible solution found. Status: {status}") +``` + +### Common Pitfalls +- Not checking solver availability, which can cause runtime errors. +- Misinterpreting the solver status (e.g., treating `FEASIBLE` as `OPTIMAL` without noting potential sub-optimality). +- Extracting variable values without first confirming a feasible status, leading to errors. + +# Workflow 2 (Algebraic Modeling Language - Pyomo) + +## Modeling stage + +### Strategy Overview +This workflow uses a declarative Algebraic Modeling Language (AML) like Pyomo. The model is defined abstractly using sets, parameters, variables, and rules, which promotes clarity, maintainability, and is ideal for problems with complex indexing or those that are part of larger systems. + +### Step 1 - Declare Abstract Sets and Parameters +- Define Pyomo `Set` objects for items and constraints. +- Define `Param` objects for revenue, demand limits, capacity limits, and the constraint coefficient matrix. Initialize them from data dictionaries. + +### Step 2 - Define Decision Variables +- Declare a `Var` for each item with domain `pyo.NonNegativeIntegers`. +- Implement individual upper bounds either as variable bounds `bounds=(0, demand_limit[i])` or as separate constraints for clarity. + +### Step 3 - Construct Capacity Constraints via Rules +- Define a function (rule) for each capacity constraint or a single rule indexed by the constraint set. +- The rule should return the expression `sum(coeff[c, i] * model.x[i] for i in model.I) <= capacity_limit[c]`. + +### Step 4 - Formulate the Objective +- Define an `Objective` with the expression `sum(revenue[i] * model.x[i] for i in model.I)` and sense `pyo.maximize`. + +### Formulation Template +```json +{ + "sets": ["I (items)", "C (constraints)"], + "parameters": [ + {"name": "revenue", "type": "float", "index": "I"}, + {"name": "demand_limit", "type": "int", "index": "I"}, + {"name": "capacity_limit", "type": "float", "index": "C"}, + {"name": "coeff", "type": "int", "index": ["C", "I"]} + ], + "decision_variables": [ + {"name": "x", "type": "NonNegativeIntegers", "index": "I"} + ], + "objective": { + "sense": "max", + "expression": "sum(revenue[i] * x[i] for i in I)" + }, + "constraints": [ + {"name": "DemandBound", "expression": "x[i] <= demand_limit[i]", "index": "I"}, + {"name": "Capacity", "expression": "sum(coeff[c,i] * x[i] for i in I) <= capacity_limit[c]", "index": "C"} + ] +} +``` + +### Common Pitfalls +- Confusing 1-based and 0-based indexing when initializing parameters from data. +- Defining constraint rules incorrectly so they reference model attributes that don't exist yet. +- Using mutable default arguments (like lists) in Pyomo rule functions. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using an external MILP solver (e.g., HiGHS, CBC). The workflow emphasizes proper solver configuration, robust handling of solution loading, and post-solution analysis to verify feasibility and understand constraint utilization. + +### Step 1 - Instantiate Solver and Set Options +- Create a solver object: `solver = pyo.SolverFactory("highs")`. +- Configure options: set time limit (`time_limit`), optimality gap (`mip_rel_gap`), number of threads (`threads`), and enable presolve. + +### Step 2 - Solve with Solution Loading Control +- Solve the model with `load_solutions=False` to first check termination status without loading potentially invalid results. +- Capture the results object: `results = solver.solve(model, load_solutions=False, tee=False)`. + +### Step 3 - Check Termination Status +- Verify the solver status: `assert results.solver.status == pyo.SolverStatus.ok`. +- Check the termination condition: `if results.solver.termination_condition in [pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible]:`. + +### Step 4 - Load and Extract Solution +- If termination is acceptable, load the solution: `model.solutions.load_from(results)`. +- Extract the objective value: `obj_val = pyo.value(model.obj)`. +- Extract variable values, converting to appropriate types (e.g., `int` for quantities). + +### Step 5 - Perform Post-Solution Analysis +- Compute the utilization of each capacity constraint and the usage of each demand limit. +- Identify binding constraints (where usage equals limit) for insight. + +### Code Usage +```python +import pyomo.environ as pyo + +# 1. Build model (following Modeling stage steps) +model = pyo.ConcreteModel() +# ... (Define sets, params, variables, constraints, objective) + +# 2. Instantiate and configure solver +solver = pyo.SolverFactory("highs") +solver.options["time_limit"] = 30 +solver.options["mip_rel_gap"] = 0.0 +solver.options["threads"] = 4 + +# 3. Solve with controlled solution loading +results = solver.solve(model, load_solutions=False, tee=False) + +# 4. Check solver status and termination condition +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in [pyo.TerminationCondition.optimal, + pyo.TerminationCondition.feasible]): + # 5. Load solution + model.solutions.load_from(results) + # 6. Extract results + objective_value = pyo.value(model.obj) + solution = {i: int(pyo.value(model.x[i])) for i in model.I} + # 7. Optional analysis + for c in model.C: + usage = sum(pyo.value(model.coeff[c, i]) * solution[i] for i in model.I) + print(f"Constraint {c} usage: {usage} / {pyo.value(model.capacity_limit[c])}") + print(f"RESULT:{objective_value}") +else: + print(f"Solver failed: {results.solver.termination_condition}") +``` + +### Common Pitfalls +- Loading solutions automatically without checking termination condition, which can raise exceptions for infeasible models. +- Forgetting to convert Pyomo variable values (which are floats) to integers for integer variables. +- Not setting the `domain` of variables correctly, leading to continuous instead of integer solutions. diff --git a/skills/optskills/skill_library/multi_machine_job_scheduling_with_precedence_and_no_overlap.md b/skills/optskills/skill_library/multi_machine_job_scheduling_with_precedence_and_no_overlap.md new file mode 100644 index 0000000..1e89976 --- /dev/null +++ b/skills/optskills/skill_library/multi_machine_job_scheduling_with_precedence_and_no_overlap.md @@ -0,0 +1,367 @@ +--- +name: Multi-Machine Job Scheduling with Precedence and No-Overlap +description: | + Model and solve scheduling problems where jobs must be processed on multiple machines in a fixed sequence, with each machine handling only one job at a time, to minimize the overall completion time (makespan). + +--- +# Workflow 1 (Constraint Programming with Interval Variables) + +## Modeling stage + +### Strategy Overview +This workflow uses a Constraint Programming (CP) paradigm, leveraging native interval variables and global no-overlap constraints. It is highly effective for pure scheduling problems, offering a declarative and solver-efficient model. + +### Step 1 - Define Problem Entities +- Define the set of jobs `J` and the set of machines `M`. +- Define the processing time `p[j][m]` for each job `j` on each machine `m`. +- Define the job routing `route[j]` as an ordered list of machines for each job. +- Calculate a reasonable time horizon `H` (e.g., sum of all processing times) for variable bounds. + +### Step 2 - Create Interval Variables +- For each job `j` and each machine `m` in its routing `route[j]`, create an interval variable `interval[j][m]`. +- The interval is defined by its start time, processing duration (`p[j][m]`), and end time. +- This single construct implicitly links start, duration, and end, simplifying constraint writing. + +### Step 3 - Enforce Precedence Chain +- For each job `j`, enforce that its operation on machine `m_{k+1}` starts after its operation on machine `m_k` finishes, according to its routing. +- Add constraints: `end_before_start(interval[j][m_k], interval[j][m_{k+1}])` for all consecutive machines in `route[j]`. + +### Step 4 - Enforce Machine Capacity +- For each machine `m`, collect all interval variables for that machine into a list `intervals_on_m`. +- Add a single `no_overlap(intervals_on_m)` constraint. This ensures intervals do not overlap in time on the same resource. + +### Step 5 - Define Makespan Objective +- Create a single integer or continuous variable `makespan`. +- For each job `j`, constrain `makespan` to be greater than or equal to the end time of its final operation: `makespan >= end_of(interval[j][last_machine])`. +- Set the objective to minimize `makespan`. + +### Formulation Template +```json +{ + "sets": [ + "J: Set of jobs.", + "M: Set of machines.", + "route[j]: Ordered list of machines for job j." + ], + "parameters": [ + "p[j][m]: Processing time of job j on machine m." + ], + "decision_variables": [ + "interval[j][m]: Interval variable representing the processing of job j on machine m.", + "makespan: Variable representing the maximum completion time." + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + "Precedence: end_before_start(interval[j][m_k], interval[j][m_{k+1}]) for all j in J, for consecutive machines m_k, m_{k+1} in route[j].", + "MachineNoOverlap: no_overlap([interval[j][m] for j in J if m in route[j]]) for all m in M.", + "MakespanDefinition: makespan >= end_of(interval[j][route[j][-1]]) for all j in J." + ] +} +``` + +### Common Pitfalls +- Forgetting to bound the start time domains of interval variables, which can lead to inefficient search. Always provide a sensible upper bound (horizon `H`). +- Misindexing the machine sequence in precedence constraints, especially if job routings are not uniform. +- Defining `makespan` as a parameter instead of a variable, preventing its minimization. + +## Solving stage + +### Strategy Overview +Solve the model using a CP solver (e.g., OR-Tools CP-SAT) that natively supports interval variables and global constraints. Configure search parameters for a balance between solution speed and quality. + +### Step 1 - Solver and Model Initialization +- Instantiate the CP model object (e.g., `CpModel`). +- Define the horizon `H` as the sum of all processing times plus a buffer. + +### Step 2 - Variable and Constraint Creation +- Create interval variables using the solver's factory method (e.g., `NewIntervalVar`), providing lower/upper bounds for start and end. +- Add precedence and no-overlap constraints using the model's methods. +- Define the `makespan` variable and its linking constraints. + +### Step 3 - Solver Configuration +- Set a time limit (e.g., `max_time_in_seconds`) appropriate for the problem size. +- Enable parallel search by setting `num_search_workers` to the number of available CPU cores. +- Optionally set a `random_seed` for reproducible results. + +### Step 4 - Execute Solve and Check Status +- Call the solver's `Solve` method on the model. +- Check the returned status (e.g., `OPTIMAL`, `FEASIBLE`, `INFEASIBLE`). +- If the status is not `FEASIBLE` or `OPTIMAL`, implement fallback logic (e.g., relax constraints, increase time limit). + +### Step 5 - Extract and Validate Solution +- If feasible, extract the start and end times for each interval variable. +- Programmatically verify that all constraints hold: precedence is respected, no intervals on the same machine overlap, and the reported makespan matches the maximum end time. +- Format the schedule into a readable structure (e.g., list of (job, machine, start, end) tuples). + +### Code Usage +```python +# Example using OR-Tools CP-SAT +from ortools.sat.python import cp_model + +# 1. Initialize Model +model = cp_model.CpModel() +horizon = sum(p[j][m] for j in J for m in route[j]) + +# 2. Create Variables +intervals = {} +for j in J: + for m in route[j]: + start_var = model.NewIntVar(0, horizon, f'start_{j}_{m}') + end_var = model.NewIntVar(0, horizon, f'end_{j}_{m}') + interval_var = model.NewIntervalVar(start_var, p[j][m], end_var, f'interval_{j}_{m}') + intervals[(j, m)] = (start_var, end_var, interval_var) + +makespan = model.NewIntVar(0, horizon, 'makespan') + +# 3. Add Constraints +# Precedence +for j in J: + r = route[j] + for idx in range(len(r) - 1): + m1, m2 = r[idx], r[idx+1] + model.Add(intervals[(j, m2)][0] >= intervals[(j, m1)][1]) + +# No-overlap per machine +for m in M: + machine_intervals = [intervals[(j, m)][2] for j in J if m in route[j]] + if machine_intervals: + model.AddNoOverlap(machine_intervals) + +# Makespan definition +for j in J: + last_machine = route[j][-1] + model.Add(makespan >= intervals[(j, last_machine)][1]) + +# 4. Set Objective +model.Minimize(makespan) + +# 5. Configure and Solve +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 8 +# solver.parameters.random_seed = 42 # Optional +status = solver.Solve(model) + +# 6. Process Result +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + print(f'Makespan: {solver.Value(makespan)}') + schedule = [] + for j in J: + for m in route[j]: + start = solver.Value(intervals[(j, m)][0]) + end = solver.Value(intervals[(j, m)][1]) + schedule.append((j, m, start, end)) + # ... process schedule +else: + print('No feasible solution found.') +``` + +### Common Pitfalls +- Not checking solver status before extracting variable values, leading to runtime errors. +- Setting an overly restrictive time horizon, which can make the problem infeasible. +- Ignoring solver logs; enabling `log_search_progress` can provide insights into search stagnation. + +# Workflow 2 (Mixed-Integer Linear Programming with Disjunctive Constraints) + +## Modeling stage + +### Strategy Overview +This workflow uses a Mixed-Integer Linear Programming (MILP) formulation with binary sequencing variables and big-M constraints to model disjunctive machine capacity. It is portable across many MILP solvers and allows for integration with other linear constraints. + +### Step 1 - Define Problem Entities +- Define the set of jobs `J` and the set of machines `M`. +- Define the processing time `p[j][m]` for each job `j` on each machine `m`. +- Define the job routing `route[j]` as an ordered list of machines for each job. +- Calculate a large constant `BigM` (e.g., sum of all processing times). + +### Step 2 - Create Continuous Decision Variables +- Create continuous (or integer) variables `start[j][m]` representing the start time of job `j` on machine `m`. +- Optionally create `completion[j][m]` variables linked by constraint: `completion[j][m] == start[j][m] + p[j][m]`. +- Create a continuous variable `makespan`. + +### Step 3 - Enforce Precedence Chain +- For each job `j` and consecutive machines `m_k`, `m_{k+1}` in its routing `route[j]`, add constraint: `start[j][m_{k+1}] >= start[j][m_k] + p[j][m_k]`. + +### Step 4 - Model Machine Capacity with Binary Variables +- For each machine `m` and each pair of distinct jobs `j` and `k` that both require machine `m`, create a binary variable `precedes[j][k][m]`. +- `precedes[j][k][m] = 1` indicates job `j` is processed before job `k` on machine `m`. +- Enforce mutual exclusivity: `precedes[j][k][m] + precedes[k][j][m] == 1` for each unordered job pair. + +### Step 5 - Implement Disjunctive Constraints via Big-M +- For each machine `m` and job pair `j, k` that both require `m`, add two conditional constraints using `BigM`: + - If `precedes[j][k][m] == 1`, then `start[k][m] >= completion[j][m]`. + - This is enforced as: `start[k][m] >= completion[j][m] - BigM * (1 - precedes[j][k][m])`. + - The symmetric constraint for the opposite order is also added. + +### Step 6 - Define Makespan Objective +- For each job `j`, add constraint: `makespan >= completion[j][last_machine]`. +- Set the objective to minimize `makespan`. + +### Formulation Template +```json +{ + "sets": [ + "J: Set of jobs.", + "M: Set of machines.", + "route[j]: Ordered list of machines for job j." + ], + "parameters": [ + "p[j][m]: Processing time of job j on machine m.", + "BigM: A sufficiently large number (e.g., sum(p[j][m] for all j, m))." + ], + "decision_variables": [ + "start[j][m]: Continuous variable for start time of job j on machine m.", + "completion[j][m]: Continuous variable for completion time of job j on machine m.", + "precedes[j][k][m]: Binary variable, 1 if job j precedes job k on machine m (defined only if both jobs require m).", + "makespan: Continuous variable for the maximum completion time." + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + "ProcessingTime: completion[j][m] == start[j][m] + p[j][m] for all j in J, m in route[j].", + "Precedence: start[j][m_{k+1}] >= completion[j][m_k] for all j in J, for consecutive machines m_k, m_{k+1} in route[j].", + "MutualExclusion: precedes[j][k][m] + precedes[k][j][m] == 1 for all m in M, j,k in J where j= completion[j][m] - BigM * (1 - precedes[j][k][m]) for all m in M, j,k in J, j!=k, where both require m.", + "Disjunctive2: start[j][m] >= completion[k][m] - BigM * (1 - precedes[k][j][m]) for all m in M, j,k in J, j!=k, where both require m.", + "MakespanDef: makespan >= completion[j][route[j][-1]] for all j in J." + ] +} +``` + +### Common Pitfalls +- Choosing a `BigM` value that is too small, which can cut off valid solutions. Use a safe, large upper bound like the sum of all processing times. +- Choosing a `BigM` value that is excessively large, which weakens the linear relaxation and slows down the solver. +- Forgetting to enforce mutual exclusivity (`precedes[j][k][m] + precedes[k][j][m] == 1`) for all job pairs, leading to incorrect or infeasible models. +- Creating binary variables for job pairs that do not share a machine, unnecessarily increasing model size. + +## Solving stage + +### Strategy Overview +Solve the MILP model using a standard solver like Gurobi, CPLEX, or CBC. Configure MIP parameters to manage solution time and quality, and implement logic to handle non-optimal terminations. + +### Step 1 - Model Building with a Modeling Language +- Use a modeling library (e.g., Pyomo, PuLP) to define sets, parameters, variables, and constraints based on the formulation. +- Ensure all indices and summations are correctly implemented, creating binary variables only for job pairs that share a machine. + +### Step 2 - Solver Configuration +- Set a MIP gap tolerance (`MIPGap`) to define acceptable optimality (e.g., `[TARGET_GAP]`). +- Set a time limit (`TimeLimit`) appropriate for the problem size. +- Configure the number of threads (`Threads`) for parallel solving. +- Set a random seed (`Seed`) for reproducibility, if supported. + +### Step 3 - Execute Solve and Check Termination +- Invoke the solver on the model instance. +- Check the termination condition (e.g., `optimal`, `feasible`, `timeLimit`). +- If the status is not optimal, assess the best bound and incumbent solution to gauge quality. + +### Step 4 - Extract and Interpret Solution +- If a feasible solution exists, retrieve the values of `start[j][m]` and `precedes[j][k][m]` variables. +- Reconstruct the job sequence on each machine from the `precedes` variables. +- Validate the solution by checking all constraints programmatically. + +### Step 5 - Analyze and Report +- Calculate the achieved makespan. +- Generate a Gantt chart or schedule table from the start times. +- Report the optimality gap if the solver terminated due to time limits. + +### Code Usage +```python +# Example using Pyomo with Gurobi +import pyomo.environ as pyo + +# 1. Create Concrete Model +model = pyo.ConcreteModel() + +# 2. Define Sets and Parameters +model.J = pyo.Set(initialize=J) # Set of jobs +model.M = pyo.Set(initialize=M) # Set of machines +# Define processing times and routing +model.p = pyo.Param(model.J, model.M, initialize=p_data, default=0) # default 0 for unused pairs +# Determine which jobs require which machine +def job_requires_m(model, j, m): + return m in route_data[j] +model.JobsOnMachine = pyo.Set(model.M, initialize=lambda model, m: [j for j in model.J if job_requires_m(model, j, m)]) + +BigM = sum(p_data[j][m] for j in J for m in route_data[j]) + +# 3. Define Variables +model.start = pyo.Var(model.J, model.M, domain=pyo.NonNegativeReals) +model.completion = pyo.Var(model.J, model.M, domain=pyo.NonNegativeReals) +model.precedes = pyo.Var(model.J, model.J, model.M, domain=pyo.Binary, initialize=0) +model.makespan = pyo.Var(domain=pyo.NonNegativeReals) + +# 4. Define Constraints +# Processing time relation (only for required machines) +def processing_rule(model, j, m): + if m in route_data[j]: + return model.completion[j, m] == model.start[j, m] + model.p[j, m] + return pyo.Constraint.Skip +model.processing = pyo.Constraint(model.J, model.M, rule=processing_rule) + +# Precedence chain +def precedence_rule(model, j): + route = route_data[j] + constraints = [] + for idx in range(len(route)-1): + m1, m2 = route[idx], route[idx+1] + constraints.append(model.start[j, m2] >= model.completion[j, m1]) + return constraints +model.precedence = pyo.Constraint(model.J, rule=precedence_rule) + +# Mutual exclusion for sequencing (only for jobs sharing a machine) +def mutual_excl_rule(model, m, j, k): + if j < k and (j in model.JobsOnMachine[m] and k in model.JobsOnMachine[m]): + return model.precedes[j, k, m] + model.precedes[k, j, m] == 1 + return pyo.Constraint.Skip +model.mutual_excl = pyo.Constraint(model.M, model.J, model.J, rule=mutual_excl_rule) + +# Disjunctive constraints (only for jobs sharing a machine) +def disjunctive1_rule(model, m, j, k): + if j != k and (j in model.JobsOnMachine[m] and k in model.JobsOnMachine[m]): + return model.start[k, m] >= model.completion[j, m] - BigM * (1 - model.precedes[j, k, m]) + return pyo.Constraint.Skip +model.disjunctive1 = pyo.Constraint(model.M, model.J, model.J, rule=disjunctive1_rule) + +def disjunctive2_rule(model, m, j, k): + if j != k and (j in model.JobsOnMachine[m] and k in model.JobsOnMachine[m]): + return model.start[j, m] >= model.completion[k, m] - BigM * (1 - model.precedes[k, j, m]) + return pyo.Constraint.Skip +model.disjunctive2 = pyo.Constraint(model.M, model.J, model.J, rule=disjunctive2_rule) + +# Makespan definition +def makespan_rule(model, j): + last_m = route_data[j][-1] + return model.makespan >= model.completion[j, last_m] +model.makespan_def = pyo.Constraint(model.J, rule=makespan_rule) + +# 5. Define Objective +model.obj = pyo.Objective(expr=model.makespan, sense=pyo.minimize) + +# 6. Solve +solver = pyo.SolverFactory('gurobi') # or 'cplex', 'cbc' +solver.options['TimeLimit'] = 30 +solver.options['MIPGap'] = 0.0001 +solver.options['Threads'] = 4 +solver.options['Seed'] = 42 +results = solver.solve(model, tee=True) # tee=True prints solver log + +# 7. Process Results +if results.solver.termination_condition == pyo.TerminationCondition.optimal: + print(f'Optimal makespan: {pyo.value(model.makespan):.2f}') + # Extract schedule from model.start and model.precedes +elif results.solver.termination_condition == pyo.TerminationCondition.feasible: + print(f'Feasible solution found. Makespan: {pyo.value(model.makespan):.2f}') + print(f'Best bound: {results.problem.lower_bound:.2f}') +else: + print('No feasible solution found.') +``` + +### Common Pitfalls +- Incorrectly indexing the "next" machine in the precedence constraint for non-consecutive or job-specific machine sequences. +- Creating an excessive number of binary variables (`|J|^2 * |M|`) for large instances, leading to intractable models. The refined code creates variables only for jobs sharing a machine. +- Not using `pyo.Constraint.Skip` correctly in rule-based constraint definitions, leading to errors for invalid index combinations. diff --git a/skills/optskills/skill_library/multi_period_production_inventory_planning.md b/skills/optskills/skill_library/multi_period_production_inventory_planning.md new file mode 100644 index 0000000..c6eaec6 --- /dev/null +++ b/skills/optskills/skill_library/multi_period_production_inventory_planning.md @@ -0,0 +1,245 @@ +--- +name: Multi-Period Production-Inventory Planning +description: | + Formulate and solve multi-period production planning problems with inventory balance, capacity constraints, and terminal conditions, using both continuous and integer variable approaches. +--- + +# Workflow 1 (Linear Programming Relaxation) + +## Modeling stage + +### Strategy Overview +Model the problem as a Linear Program (LP) using continuous variables. This provides a theoretical upper bound on profit and is useful for initial feasibility checks, sensitivity analysis, or when fractional production is acceptable (e.g., in high-volume fluid production). + +### Step 1 - Define Sets and Parameters +- Define sets for `products`, `machines`, and `periods`. +- Define parameters for `profit_per_unit`, `holding_cost`, `production_time`, `machine_capacity`, `max_sales`, `max_inventory`, and `terminal_inventory_target`. + +### Step 2 - Define Continuous Decision Variables +- Define `production[product, period]` as `NonNegativeReals`. +- Define `sales[product, period]` as `NonNegativeReals`. +- Define `inventory[product, period]` as `NonNegativeReals`. + +### Step 3 - Formulate Inventory Balance Constraints +- For the first period, enforce `production[p,0] = sales[p,0] + inventory[p,0]`. +- For subsequent periods, enforce `inventory[p,t-1] + production[p,t] = sales[p,t] + inventory[p,t]`. + +### Step 4 - Formulate Capacity and Bound Constraints +- For each machine and period, enforce `sum(production_time[m][p] * production[p,t] for p in products) <= machine_capacity[m][t]`. +- For each product and period, enforce `sales[p,t] <= max_sales[p][t]`. +- For each product and period, enforce `inventory[p,t] <= max_inventory`. + +### Step 5 - Formulate Terminal Inventory and Objective +- For each product, enforce `inventory[p, final_period] = terminal_inventory_target[p]`. +- Maximize `sum(profit_per_unit[p] * sales[p,t] - holding_cost * inventory[p,t] for p in products for t in periods)`. + +### Formulation Template +```json +{ + "sets": ["products", "machines", "periods"], + "parameters": { + "profit_per_unit": {"product": "float"}, + "holding_cost": "float", + "production_time": {"machine": {"product": "float"}}, + "machine_capacity": {"machine": {"period": "float"}}, + "max_sales": {"product": {"period": "float"}}, + "max_inventory": "float", + "terminal_inventory_target": {"product": "float"} + }, + "decision_variables": { + "production": {"product": "period", "domain": "NonNegativeReals"}, + "sales": {"product": "period", "domain": "NonNegativeReals"}, + "inventory": {"product": "period", "domain": "NonNegativeReals"} + }, + "objective": { + "sense": "max", + "expression": "sum(profit_per_unit[p] * sales[p,t] - holding_cost * inventory[p,t] for p in products for t in periods)" + }, + "constraints": [ + "inventory_balance_first_period", + "inventory_balance_subsequent_periods", + "machine_capacity", + "sales_upper_bound", + "inventory_upper_bound", + "terminal_inventory" + ] +} +``` + +### Common Pitfalls +- Using continuous variables for problems requiring discrete unit decisions, leading to operationally infeasible fractional solutions. +- Overlooking the implicit integer requirement signaled by terms like "number of units" in the problem context. +- Accepting the LP solution as final without verifying if the context demands integer feasibility. + +## Solving stage + +### Strategy Overview +Solve the LP model using a high-performance solver (e.g., HiGHS) to obtain a continuous solution. Focus on verifying feasibility, analyzing constraints, and using the result as a bound for integer models. + +### Step 1 - Configure and Run Solver +- Instantiate the solver (e.g., `SolverFactory('highs')`). +- Set options for performance: `time_limit=30`, `threads=4`, `presolve='on'`. +- Solve the model with `tee=True` for initial debugging. + +### Step 2 - Check Solver Status and Load Solution +- After solving, check the solver termination condition (e.g., `optimal`, `feasible`). +- Use `load_solutions=False` initially to avoid errors on infeasible runs, then load solutions only if status is acceptable. + +### Step 3 - Verify Solution and Analyze Results +- Programmatically verify key constraints are satisfied (e.g., terminal inventory equals target, capacity limits). +- Extract and display production, sales, and inventory quantities. +- Calculate and report total revenue, holding costs, and net profit separately to validate the objective value. + +### Step 4 - Perform Bottleneck Analysis +- Calculate machine utilization per period: `sum(production_time[m][p] * production[p,t]) / machine_capacity[m][t]`. +- Identify periods and machines with 100% utilization, as these constrain further profit improvement. + +### Code Usage +```python +# build model from formulation +model = pyo.ConcreteModel() +# ... (model construction based on formulation template) + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = 30 +solver.options['threads'] = 4 +results = solver.solve(model, load_solutions=False, tee=False) + +# Check status and load solution +if results.solver.termination_condition == pyo.TerminationCondition.optimal: + model.solutions.load_from(results) + print("Optimal LP solution found.") + # ... (solution verification and analysis) +elif results.solver.termination_condition == pyo.TerminationCondition.feasible: + model.solutions.load_from(results) + print("Feasible LP solution found.") +else: + print("Solver failed. Status:", results.solver.termination_condition) + # ... (diagnostic output for infeasibility) +``` + +### Common Pitfalls +- Accepting the first solver output without examining solution feasibility (e.g., fractional values for discrete units). +- Neglecting to verify constraint satisfaction programmatically after solving. +- Switching to an integer model without first using the LP solution to identify binding constraints and performance bounds. + +# Workflow 2 (Mixed-Integer Programming) + +## Modeling stage + +### Strategy Overview +Model the problem as a Mixed-Integer Program (MIP) using integer variables for production, sales, and inventory. This yields operationally feasible solutions for contexts requiring whole units and is necessary when the LP relaxation is not implementable. + +### Step 1 - Define Sets and Parameters +- Use the same set and parameter structure as Workflow 1. + +### Step 2 - Define Integer Decision Variables +- Define `production[product, period]` as `NonNegativeIntegers`. +- Define `sales[product, period]` as `NonNegativeIntegers`. +- Define `inventory[product, period]` as `NonNegativeIntegers`. + +### Step 3 - Formulate Inventory Balance Constraints +- Apply the same inventory balance constraints as in Workflow 1, ensuring material flow consistency with integer variables. + +### Step 4 - Formulate Capacity and Bound Constraints +- Apply the same capacity, sales upper bound, and inventory upper bound constraints as in Workflow 1. + +### Step 5 - Formulate Terminal Inventory and Objective +- Enforce the same terminal inventory equality constraints. +- Use the same profit-maximizing objective function. + +### Formulation Template +```json +{ + "sets": ["products", "machines", "periods"], + "parameters": { + "profit_per_unit": {"product": "float"}, + "holding_cost": "float", + "production_time": {"machine": {"product": "float"}}, + "machine_capacity": {"machine": {"period": "float"}}, + "max_sales": {"product": {"period": "float"}}, + "max_inventory": "float", + "terminal_inventory_target": {"product": "float"} + }, + "decision_variables": { + "production": {"product": "period", "domain": "NonNegativeIntegers"}, + "sales": {"product": "period", "domain": "NonNegativeIntegers"}, + "inventory": {"product": "period", "domain": "NonNegativeIntegers"} + }, + "objective": { + "sense": "max", + "expression": "sum(profit_per_unit[p] * sales[p,t] - holding_cost * inventory[p,t] for p in products for t in periods)" + }, + "constraints": [ + "inventory_balance_first_period", + "inventory_balance_subsequent_periods", + "machine_capacity", + "sales_upper_bound", + "inventory_upper_bound", + "terminal_inventory" + ] +} +``` + +### Common Pitfalls +- Starting with an LP model when the problem context clearly requires integer decisions, wasting time on an infeasible relaxation. +- Using the same variable domain (e.g., `NonNegativeReals`) for all formulations without justifying the choice. +- Overlooking the potential for a significant gap between the LP upper bound and the integer optimum. + +## Solving stage + +### Strategy Overview +Solve the MIP model using a MIP-capable solver (e.g., HiGHS, CBC) with appropriate optimality gap settings. Focus on obtaining and verifying a proven optimal or high-quality feasible integer solution. + +### Step 1 - Configure MIP Solver with Tight Gaps +- Instantiate the solver (e.g., `SolverFactory('highs')`). +- Set MIP-specific options: `mip_rel_gap=0.000001` (or a small tolerance), `time_limit=60`. +- Enable presolving and set thread count for performance. + +### Step 2 - Solve and Rigorously Check Status +- Solve the model with `load_solutions=False`. +- Check both the solver termination condition and solution status. +- Load the solution only if status indicates optimal or integer feasible. + +### Step 3 - Verify Integer Feasibility and Constraints +- Programmatically verify that all decision variable values are integers. +- Re-check all constraints (capacity, inventory balance, bounds) to ensure the integer solution is feasible. +- Confirm terminal inventory requirements are met exactly. + +### Step 4 - Analyze Optimality and Performance +- Report the MIP optimality gap from the solver results. +- Compare the MIP objective value to the LP upper bound from Workflow 1 to understand the integrality gap. +- Extract and display the integer production plan. + +### Code Usage +```python +# build model from formulation +model = pyo.ConcreteModel() +# ... (model construction with integer variables) + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = 60 +solver.options['mip_rel_gap'] = 1e-6 +solver.options['threads'] = 4 +results = solver.solve(model, load_solutions=False, tee=False) + +# Check status and load solution +status_ok = ( + results.solver.termination_condition == pyo.TerminationCondition.optimal or + results.solver.termination_condition == pyo.TerminationCondition.feasible +) +if status_ok and results.solver.status == pyo.SolverStatus.ok: + model.solutions.load_from(results) + print("Integer solution loaded.") + # ... (integer feasibility and constraint verification) +else: + print("Solver did not find a suitable integer solution.") + print("Termination condition:", results.solver.termination_condition) +``` + +### Common Pitfalls +- Not setting a tight MIP gap, leading to premature acceptance of suboptimal solutions. +- Assuming an integer solution is optimal without checking solver statistics (e.g., nodes explored, gap). +- Neglecting to verify that the loaded solution satisfies all constraints, especially after a non-optimal termination. diff --git a/skills/optskills/skill_library/multi_product_multi_market_supply_allocation_with_integer_quantities.md b/skills/optskills/skill_library/multi_product_multi_market_supply_allocation_with_integer_quantities.md new file mode 100644 index 0000000..3a70b80 --- /dev/null +++ b/skills/optskills/skill_library/multi_product_multi_market_supply_allocation_with_integer_quantities.md @@ -0,0 +1,246 @@ +--- +name: Multi-Product Multi-Market Supply Allocation with Integer Quantities +description: | + Models and solves a profit-maximizing supply allocation problem with integer quantities across multiple products and markets, using either OR-Tools or Pyomo with equality demand constraints. +--- + +# Workflow 1 (OR-Tools SCIP Solver) + +## Modeling stage + +### Strategy Overview +Build a mixed-integer programming model using Google OR-Tools' pywraplp interface. Define integer decision variables for each source-product-market triplet, enforce demand fulfillment with equality constraints, and maximize total profit using linear coefficients. + +### Step 1 - Define Data Structures +- Organize input data as nested dictionaries: `profit[source][product][market]` for profit per unit, `demand[product][market]` for required quantities. +- Precompute an upper bound for each decision variable as the corresponding demand value to tighten the formulation. + +### Step 2 - Create Solver and Decision Variables +- Instantiate a solver with `pywraplp.Solver.CreateSolver("SCIP")`. +- For each combination of source, product, and market, create an integer variable using `solver.IntVar(0, demand[product][market], name)` to enforce non-negativity and integrality. + +### Step 3 - Add Demand Fulfillment Constraints +- For each product-market pair, sum all source variables for that pair and set equality to the demand: `solver.Add(sum(variables) == demand[product][market])`. + +### Step 4 - Build Objective Function +- Create an empty objective with `solver.Objective()`. +- Iterate over all decision variables, setting their profit coefficients with `objective.SetCoefficient(variable, profit_value)`. +- Call `objective.SetMaximization()` to set the sense. + +### Formulation Template +```json +{ + "sets": ["Sources", "Products", "Markets"], + "parameters": [ + "profit[source, product, market]", + "demand[product, market]" + ], + "decision_variables": [ + "supply[source, product, market] (integer, 0..demand[product, market])" + ], + "objective": { + "sense": "maximize", + "expression": "sum(profit[source, product, market] * supply[source, product, market] for all indices)" + }, + "constraints": [ + "for each (product, market): sum(supply[source, product, market] over sources) == demand[product, market]" + ] +} +``` + +### Common Pitfalls +- Using `solver.IntVar` without an upper bound can lead to unbounded search space; always bound by demand. +- Forgetting to call `objective.SetMaximization()` results in a default minimization objective. +- Naming variables with duplicate keys (e.g., same source-product-market) causes solver errors; ensure unique names. + +## Solving stage + +### Strategy Overview +Configure the SCIP solver with a time limit and optional parallelism, solve the model, and parse results with explicit status checks. Output results in a structured JSON format prefixed with `RESULT_JSON:` for downstream parsing. + +### Step 1 - Configure Solver Parameters +- Set a time limit with `solver.SetTimeLimit(30000)` (30 seconds in milliseconds). +- Optionally enable parallel solving with `solver.SetNumThreads(4)`. + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve()`. +- Check if status is `pywraplp.Solver.OPTIMAL` or `pywraplp.Solver.FEASIBLE` before extracting results. + +### Step 3 - Extract and Format Results +- Retrieve objective value with `objective.Value()`. +- Iterate over all variables, collecting `variable.name()` and `variable.solution_value()` for non-zero values. +- Print a JSON string with keys `"status"`, `"objective_value"`, and `"solution"`, prefixed by `RESULT_JSON:`. + +### Step 4 - Handle Failures +- If solver is `None` after creation, print `{"status": "failed", "reason": "Solver not available"}`. +- For infeasible or error statuses, print a JSON with `"status": "failed"` and include the solver status code. + +### Code Usage +```python +from ortools.linear_solver import pywraplp +import json + +def solve_supply_allocation(profit, demand, sources, products, markets): + solver = pywraplp.Solver.CreateSolver("SCIP") + if not solver: + print('RESULT_JSON:{"status": "failed", "reason": "SCIP solver not available"}') + return + + # Decision variables + supply = {} + for s in sources: + for p in products: + for m in markets: + ub = demand.get(p, {}).get(m, 0) + supply[(s, p, m)] = solver.IntVar(0, ub, f'supply_{s}_{p}_{m}') + + # Demand constraints + for p in products: + for m in markets: + solver.Add(sum(supply[(s, p, m)] for s in sources) == demand[p][m]) + + # Objective + objective = solver.Objective() + for s in sources: + for p in products: + for m in markets: + objective.SetCoefficient(supply[(s, p, m)], profit[s][p][m]) + objective.SetMaximization() + + # Solve + solver.SetTimeLimit(30000) + solver.SetNumThreads(4) + status = solver.Solve() + + if status in (pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE): + result = { + "status": "optimal" if status == pywraplp.Solver.OPTIMAL else "feasible", + "objective_value": objective.Value(), + "solution": {} + } + for (s, p, m), var in supply.items(): + val = var.solution_value() + if val > 0: + result["solution"][f"{s}_{p}_{m}"] = int(val) + print(f'RESULT_JSON:{json.dumps(result)}') + else: + print(f'RESULT_JSON:{{"status": "failed", "solver_status": {status}}}') +``` + +### Common Pitfalls +- Not checking if solver creation returns `None` leads to runtime errors on systems without SCIP. +- Using `solver.SetTimeLimit` with milliseconds instead of seconds; 30000 = 30 seconds. +- Forgetting to convert `solution_value()` to native Python types before JSON serialization. + +# Workflow 2 (Pyomo with CBC Solver) + +## Modeling stage + +### Strategy Overview +Construct a Pyomo ConcreteModel with separate Set objects for sources, products, and markets. Define a 3-index integer decision variable, enforce demand equality constraints using constraint rules, and maximize profit with a linear objective expression. + +### Step 1 - Define Index Sets +- Create Pyomo Set objects: `model.SOURCES = pyo.Set(initialize=sources)`, `model.PRODUCTS = pyo.Set(initialize=products)`, `model.MARKETS = pyo.Set(initialize=markets)`. +- Use distinct variable names for model attributes and loop variables to avoid name conflicts (e.g., use `mk` for market index, not `m`). + +### Step 2 - Declare Decision Variables +- Define `model.supply = pyo.Var(model.SOURCES, model.PRODUCTS, model.MARKETS, domain=pyo.NonNegativeIntegers)` for integer supply quantities. + +### Step 3 - Add Demand Fulfillment Constraints +- Write a constraint rule: `def demand_rule(model, p, mk): return sum(model.supply[s, p, mk] for s in model.SOURCES) == demand[p][mk]`. +- Add the constraint with `model.demand_con = pyo.Constraint(model.PRODUCTS, model.MARKETS, rule=demand_rule)`. + +### Step 4 - Build Objective Function +- Define the objective expression: `model.obj = pyo.Objective(expr=sum(profit[s][p][mk] * model.supply[s, p, mk] for s in model.SOURCES for p in model.PRODUCTS for mk in model.MARKETS), sense=pyo.maximize)`. + +### Formulation Template +```json +{ + "sets": ["SOURCES", "PRODUCTS", "MARKETS"], + "parameters": [ + "profit[source, product, market]", + "demand[product, market]" + ], + "decision_variables": [ + "supply[source, product, market] (NonNegativeIntegers)" + ], + "objective": { + "sense": "maximize", + "expression": "sum(profit[s, p, m] * supply[s, p, m] for all indices)" + }, + "constraints": [ + "for each (product, market): sum(supply[source, product, market] over sources) == demand[product, market]" + ] +} +``` + +### Common Pitfalls +- Reusing the model variable name (e.g., `m`) as a loop variable inside constraint rules or generator expressions causes attribute errors; use distinct names like `mk` for market. +- Forgetting to import `pyo.environ` or using incorrect domain names (e.g., `NonNegativeIntegers` vs `NonNegativeInteger`). +- Using mutable data structures (e.g., lists) as set elements; Pyomo requires hashable types. + +## Solving stage + +### Strategy Overview +Use the CBC solver via Pyomo's SolverFactory with optimality gap and time limit settings. Check solver status and termination condition before extracting results, and output the objective value in a parseable format. + +### Step 1 - Configure Solver +- Create solver instance: `solver = pyo.SolverFactory("cbc")`. +- Set options: `solver.options["seconds"] = 30` for time limit, `solver.options["ratio"] = 0.0` for zero MIP gap (optimality required). + +### Step 2 - Solve Model +- Call `result = solver.solve(model, tee=False)` to suppress solver output. + +### Step 3 - Check Status and Extract Results +- Verify `result.solver.status == SolverStatus.ok` and `result.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}`. +- Extract objective value: `obj_val = float(pyo.value(model.obj))`. +- Print in format: `print(f"RESULT:{obj_val}")`. +- For solution inspection, iterate over `model.supply` indices and collect non-zero values: `{str(idx): int(pyo.value(var)) for idx, var in model.supply.items() if pyo.value(var) > 0}`. + +### Step 4 - Handle Failures +- If status is not ok or termination condition is unexpected, print a JSON payload: `{"status": "failed", "reason": str(result.solver.termination_condition)}`. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +def solve_supply_allocation_pyomo(profit, demand, sources, products, markets): + model = pyo.ConcreteModel() + model.SOURCES = pyo.Set(initialize=sources) + model.PRODUCTS = pyo.Set(initialize=products) + model.MARKETS = pyo.Set(initialize=markets) + + model.supply = pyo.Var(model.SOURCES, model.PRODUCTS, model.MARKETS, domain=pyo.NonNegativeIntegers) + + def demand_rule(model, p, mk): + return sum(model.supply[s, p, mk] for s in model.SOURCES) == demand[p][mk] + model.demand_con = pyo.Constraint(model.PRODUCTS, model.MARKETS, rule=demand_rule) + + model.obj = pyo.Objective( + expr=sum(profit[s][p][mk] * model.supply[s, p, mk] + for s in model.SOURCES for p in model.PRODUCTS for mk in model.MARKETS), + sense=pyo.maximize + ) + + solver = pyo.SolverFactory("cbc") + solver.options["seconds"] = 30 + solver.options["ratio"] = 0.0 + result = solver.solve(model, tee=False) + + if result.solver.status == SolverStatus.ok and \ + result.solver.termination_condition in (TerminationCondition.optimal, TerminationCondition.feasible): + obj_val = float(pyo.value(model.obj)) + print(f"RESULT:{obj_val}") + # Optional: detailed solution + solution = {str(idx): int(pyo.value(var)) + for idx, var in model.supply.items() if pyo.value(var) > 0} + print(f"SOLUTION:{solution}") + else: + print(f'{{"status": "failed", "reason": "{result.solver.termination_condition}"}}') +``` + +### Common Pitfalls +- Not importing `SolverStatus` and `TerminationCondition` from `pyomo.opt` leads to NameError in status checks. +- Setting `ratio` to 0.0 may cause long solve times for large instances; consider relaxing to 0.01 for practical use. +- Forgetting to convert `pyo.value()` results to native Python types before printing or serialization. diff --git a/skills/optskills/skill_library/multi_resource_assignment_optimization.md b/skills/optskills/skill_library/multi_resource_assignment_optimization.md new file mode 100644 index 0000000..7478674 --- /dev/null +++ b/skills/optskills/skill_library/multi_resource_assignment_optimization.md @@ -0,0 +1,289 @@ +--- +name: Multi-Resource Assignment Optimization +description: | + Model and solve linear cost-minimization problems where discrete or continuous resources with capacity contributions must be assigned to tasks to meet demands, subject to supply limits. +--- + +# Workflow 1 (Integer Assignment with OR-Tools) + +## Modeling stage + +### Strategy Overview +Formulate the problem as an Integer Linear Program (ILP) using OR-Tools' MIP solver interface. This workflow is suited for problems requiring discrete, whole-unit assignments (e.g., assigning aircraft, machines, or personnel counts). It directly maps resource-task pairs to integer decision variables with linear constraints for supply and weighted demand coverage. + +### Step 1 - Define Data Structures +- Organize problem data into indexed lists or dictionaries for resources and tasks. +- Define parameters: `availability[i]` (supply per resource), `demand[j]` (requirement per task), `capacity[i][j]` (contribution per unit), and `cost[i][j]` (cost per unit assignment). + +### Step 2 - Create Integer Decision Variables +- For each resource `i` and task `j`, create an integer variable `x[i][j]` representing the assignment count. +- Set the variable domain to non-negative integers, optionally bounded above by the resource's availability. + +### Step 3 - Formulate Supply Constraints +- For each resource `i`, add a linear inequality constraint: the sum of all assignments for that resource must not exceed its availability (`sum_j x[i][j] <= availability[i]`). + +### Step 4 - Formulate Demand Coverage Constraints +- For each task `j`, add a linear inequality constraint: the weighted sum of assigned resources must meet or exceed the demand (`sum_i capacity[i][j] * x[i][j] >= demand[j]`). + +### Step 5 - Define Linear Cost Objective +- Define the objective to minimize total cost: `minimize sum_i sum_j cost[i][j] * x[i][j]`. + +### Formulation Template +```json +{ + "sets": ["resources", "tasks"], + "parameters": [ + "availability[resource]", + "demand[task]", + "capacity[resource][task]", + "cost[resource][task]" + ], + "decision_variables": ["x[resource][task] ∈ ℤ⁺"], + "objective": { + "sense": "min", + "expression": "∑∑ cost[resource][task] * x[resource][task]" + }, + "constraints": [ + "supply[resource]: ∑_task x[resource][task] ≤ availability[resource]", + "demand[task]: ∑_resource capacity[resource][task] * x[resource][task] ≥ demand[task]" + ] +} +``` + +### Common Pitfalls +- Forgetting to set upper bounds on integer variables, which can lead to unbounded or inefficient solving. +- Using floating-point values for `capacity` or `demand` when the solver expects integer coefficients; scale data appropriately. +- Mis-indexing parameters in nested loops when building constraints, leading to incorrect coefficient assignment. + +## Solving stage + +### Strategy Overview +Solve the ILP model using OR-Tools' wrapper for SCIP or CBC. Configure performance settings, execute the solve, and rigorously verify the solution's feasibility and integrality before extracting results. + +### Step 1 - Initialize Solver and Set Parameters +- Create a solver instance: `solver = pywraplp.Solver.CreateSolver('SCIP')`. +- Set practical limits: `solver.SetTimeLimit(30000)` for a 30-second timeout and `solver.SetNumThreads(4)` for parallel processing. + +### Step 2 - Build Model from Formulation +- Translate the modeling steps into code using loops to create variables, set objective coefficients, and add constraints via `constraint.SetCoefficient()`. + +### Step 3 - Execute Solve and Check Status +- Call `solver.Solve()`. +- Check the result status: accept solutions marked as `OPTIMAL` or `FEASIBLE`. Handle `INFEASIBLE` or `UNBOUNDED` statuses with appropriate error reporting. + +### Step 4 - Extract and Validate Solution +- If the status is acceptable, extract the objective value: `solver.Objective().Value()`. +- Extract variable values using `x[i][j].solution_value()` and convert to integers (e.g., `int(round(val))`). +- Programmatically verify constraints by recomputing total resource usage and delivered capacity per task against the original parameters. + +### Step 5 - Structure and Output Results +- Package the results (status, objective value, non-zero assignments, verification metrics) into a structured JSON or dictionary for downstream use. + +### Code Usage +```python +# Example using OR-Tools for integer assignment +from ortools.linear_solver import pywraplp + +# 1. Initialize solver +solver = pywraplp.Solver.CreateSolver('SCIP') +if not solver: + raise RuntimeError("Solver backend not available.") +solver.SetTimeLimit(30000) + +# 2. Define data (placeholders) +resources = [...] # list of resource identifiers +tasks = [...] # list of task identifiers +availability = {...} +demand = {...} +capacity = {...} # dict of dicts: capacity[resource][task] +cost = {...} # dict of dicts: cost[resource][task] + +# 3. Create variables +x = {} +for i in resources: + for j in tasks: + x[i, j] = solver.IntVar(0, solver.infinity(), f'x_{i}_{j}') + +# 4. Add supply constraints +for i in resources: + constraint = solver.Constraint(0, availability[i]) + for j in tasks: + constraint.SetCoefficient(x[i, j], 1) + +# 5. Add demand constraints +for j in tasks: + constraint = solver.Constraint(demand[j], solver.infinity()) + for i in resources: + constraint.SetCoefficient(x[i, j], capacity[i][j]) + +# 6. Set objective +objective = solver.Objective() +for i in resources: + for j in tasks: + objective.SetCoefficient(x[i, j], cost[i][j]) +objective.SetMinimization() + +# 7. Solve and check status +status = solver.Solve() +if status in (pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE): + obj_val = objective.Value() + assignments = {} + for i in resources: + for j in tasks: + val = x[i, j].solution_value() + if val > 0.5: # tolerance for integer extraction + assignments[(i, j)] = int(round(val)) + # ... verification and output ... +else: + # Handle failure + print(f"Solver failed with status: {status}") +``` + +### Common Pitfalls +- Not checking solver backend availability, which can cause runtime errors. +- Extracting variable values without verifying the solve status first, leading to access errors. +- Ignoring numerical precision when converting floating-point solution values to integers; use rounding with a tolerance. + +# Workflow 2 (Linear/Integer Assignment with Pyomo) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a ConcreteModel in Pyomo, providing a declarative, solver-agnostic definition. This workflow cleanly separates model construction from solving, supporting both continuous (LP) and integer (MILP) domains. It is ideal for prototyping and leveraging Pyomo's advanced features like sets and rules. + +### Step 1 - Define Pyomo Sets and Parameters +- Create Pyomo `Set` objects for `model.R` (resources) and `model.T` (tasks). +- Define `Param` objects or use plain dictionaries for `availability`, `demand`, `capacity`, and `cost` parameters, indexed by the appropriate sets. + +### Step 2 - Declare Decision Variables +- Create a Pyomo `Var` object `model.x` indexed over `model.R` and `model.T`. +- Choose the domain: `pyo.NonNegativeReals` for continuous allocation or `pyo.NonNegativeIntegers` for discrete assignment. + +### Step 3 - Implement Constraint Rules +- Define a rule function for supply constraints: for each resource `r`, `sum(model.x[r, t] for t in model.T) <= availability[r]`. +- Define a rule function for demand constraints: for each task `t`, `sum(capacity[r][t] * model.x[r, t] for r in model.R) >= demand[t]`. +- Optionally, add individual assignment limit constraints: `model.x[r, t] <= max_assignment[r][t]`. + +### Step 4 - Define the Objective Rule +- Create an `Objective` rule: `sum(cost[r][t] * model.x[r, t] for r in model.R for t in model.T)` with sense `minimize`. + +### Formulation Template +```json +{ + "sets": ["R (resources)", "T (tasks)"], + "parameters": [ + "availability[R]", + "demand[T]", + "capacity[R][T]", + "cost[R][T]", + "max_assignment[R][T] (optional)" + ], + "decision_variables": ["x[R, T] ∈ ℝ⁺ or ℤ⁺"], + "objective": { + "sense": "min", + "expression": "∑∑ cost[R][T] * x[R, T]" + }, + "constraints": [ + "supply[R]: ∑_T x[R, T] ≤ availability[R]", + "demand[T]: ∑_R capacity[R][T] * x[R, T] ≥ demand[T]", + "limit[R, T]: x[R, T] ≤ max_assignment[R, T] (optional)" + ] +} +``` + +### Common Pitfalls +- Defining constraint or objective rules that directly reference external data instead of model parameters, breaking model portability. +- Using mutable default arguments (like `[]`) in rule functions. +- Confusing Pyomo's 1-based indexing with Python's 0-based indexing when initializing sets from lists. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using an appropriate solver factory (e.g., `'highs'` for LP, `'cbc'` for MILP). Configure solver options, handle solution loading carefully, and implement systematic verification of the results. + +### Step 1 - Select and Configure Solver +- Instantiate the solver: `solver = pyo.SolverFactory('cbc')` for integer problems or `'highs'` for continuous ones. +- Set options: `solver.options['seconds'] = 30` (time limit), `solver.options['ratio'] = 0.0` (optimality gap). + +### Step 2 - Solve with Robust Status Handling +- Execute the solve with `load_solutions=False` to prevent errors on failed solves: `results = solver.solve(model, tee=False, load_solutions=False)`. +- Check the high-level status: `assert results.solver.status == pyo.SolverStatus.ok`. +- Check the termination condition: accept `optimal` or `feasible`. + +### Step 3 - Load and Extract Solution +- If status checks pass, load the solution: `model.solutions.load_from(results)`. +- Extract the objective value: `obj_val = pyo.value(model.obj)`. +- Iterate through variables to collect non-zero assignments, applying a tolerance (e.g., `if pyo.value(model.x[r, t]) > 1e-6`). + +### Step 4 - Verify Solution Feasibility +- Programmatically compute total resource usage and delivered capacity per task from the extracted assignments. +- Compare these computed values against the original `availability` and `demand` parameters to verify all constraints are satisfied within a small tolerance. + +### Step 5 - Package Output +- Structure the output into a dictionary or JSON containing the solve status, objective value, assignment list, and verification results. + +### Code Usage +```python +# Example using Pyomo for flexible assignment +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# 1. Build model +model = pyo.ConcreteModel() +model.R = pyo.Set(initialize=resources) +model.T = pyo.Set(initialize=tasks) + +# Parameters (using dictionaries) +availability_dict = {...} +demand_dict = {...} +capacity_dict = {...} # capacity_dict[r][t] +cost_dict = {...} # cost_dict[r][t] + +# Variables (choose domain) +model.x = pyo.Var(model.R, model.T, domain=pyo.NonNegativeIntegers) # or NonNegativeReals + +# Objective +def obj_rule(m): + return sum(cost_dict[r][t] * m.x[r, t] for r in m.R for t in m.T) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + +# Supply constraints +def supply_rule(m, r): + return sum(m.x[r, t] for t in m.T) <= availability_dict[r] +model.supply_con = pyo.Constraint(model.R, rule=supply_rule) + +# Demand constraints +def demand_rule(m, t): + return sum(capacity_dict[r][t] * m.x[r, t] for r in m.R) >= demand_dict[t] +model.demand_con = pyo.Constraint(model.T, rule=demand_rule) + +# 2. Solve +solver = pyo.SolverFactory('cbc') +solver.options['seconds'] = 30 +solver.options['ratio'] = 0.0 + +results = solver.solve(model, tee=False, load_solutions=False) + +# 3. Check status and load +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in (TerminationCondition.optimal, + TerminationCondition.feasible)): + model.solutions.load_from(results) + obj_val = pyo.value(model.obj) + assignments = {} + for r in model.R: + for t in model.T: + val = pyo.value(model.x[r, t]) + if val > 0.5: # tolerance for integer extraction + assignments[(r, t)] = int(round(val)) + # ... verification and output ... +else: + # Handle failure + print(f"Solver failed. Status: {results.solver.status}, " + f"Termination: {results.solver.termination_condition}") +``` + +### Common Pitfalls +- Attempting to access variable values (`pyo.value`) before loading the solution, resulting in `ValueError`. +- Not setting `load_solutions=False` when solving, which can cause crashes on infeasible or unbounded models. +- Overlooking the difference between `SolverStatus` and `TerminationCondition`; both must be checked for reliable solution extraction. diff --git a/skills/optskills/skill_library/multi_vehicle_routing_with_optional_fleet_usage.md b/skills/optskills/skill_library/multi_vehicle_routing_with_optional_fleet_usage.md new file mode 100644 index 0000000..6726f9a --- /dev/null +++ b/skills/optskills/skill_library/multi_vehicle_routing_with_optional_fleet_usage.md @@ -0,0 +1,335 @@ +--- +name: Multi-Vehicle Routing with Optional Fleet Usage +description: | + Formulate and solve vehicle routing problems with optional vehicle usage using binary arc variables and Miller-Tucker-Zemlin subtour elimination, producing executable routes from solver output. +--- + +# Workflow 1 (MIP Solver with Explicit Vehicle Assignment) + +## Modeling stage + +### Strategy Overview +This workflow uses a mixed-integer programming (MIP) formulation with separate binary variables for arc selection (`x[v,i,j]`) and vehicle assignment (`y[v,i]`). This explicit linking simplifies constraint expression and is well-suited for solvers like SCIP and Gurobi. The Miller-Tucker-Zemlin (MTZ) method eliminates subtours. + +### Step 1 - Define Core Sets and Parameters +- Define the set of vehicles `V`, the set of all nodes `N` (including depot at index 0), and the set of customer nodes `C = N \ {0}`. +- Define a distance matrix `dist[i][j]` for all `i, j` in `N`. +- Define parameter `M` as a sufficiently large number (e.g., `|C|` or `|N|`) for MTZ constraints. + +### Step 2 - Create Decision Variables +- Create binary variable `x[v,i,j]` for each vehicle `v` and node pair `(i,j)`. `x[v,i,j] = 1` if vehicle `v` travels directly from node `i` to node `j`. +- Create binary variable `y[v,i]` for each vehicle `v` and node `i`. `y[v,i] = 1` if vehicle `v` visits node `i`. +- Create continuous variable `u[i]` for each customer node `i` for MTZ sequencing. + +### Step 3 - Formulate Objective and Basic Constraints +- **Objective**: Minimize total travel distance: `min sum_{v,i,j} dist[i][j] * x[v,i,j]`. +- **Single Visitation**: Each customer is visited exactly once: `sum_{v,i} x[v,i,j] = 1` for all `j` in `C`. +- **Flow Conservation**: For each vehicle `v` and node `j`: `sum_i x[v,i,j] = sum_k x[v,j,k]`. +- **Link Assignment to Flow**: For each vehicle `v` and node `j`: `sum_i x[v,i,j] = y[v,j]` and `sum_k x[v,j,k] = y[v,j]`. + +### Step 4 - Implement Depot and Vehicle Usage Logic +- **Optional Departure/Return**: For each vehicle `v`: `sum_{j in C} x[v,0,j] <= 1` and `sum_{i in C} x[v,i,0] <= 1`. Use `<=` to allow vehicles to remain unused. +- **Depot Self-Loop Prohibition**: `x[v,0,0] = 0` for all `v`. +- **Customer Self-Loop Prohibition**: `x[v,i,i] = 0` for all `v`, `i` in `C`. + +### Step 5 - Apply MTZ Subtour Elimination +- For each customer pair `(i,j)` where `i != j`: `u[i] - u[j] + M * sum_v x[v,i,j] <= M - 1`. +- Set bounds: `1 <= u[i] <= |C|` for `i` in `C`. Fix `u[0] = 0` for the depot. + +### Formulation Template +```json +{ + "sets": [ + "V: set of vehicles (index v)", + "N: set of all nodes (index i, j, k), with depot at index 0", + "C: set of customer nodes, C = N \\ {0}" + ], + "parameters": [ + "dist[i][j]: distance/cost from node i to j, for i,j in N", + "M: sufficiently large constant (e.g., |C|)" + ], + "decision_variables": [ + "x[v,i,j]: binary, 1 if vehicle v travels from i to j", + "y[v,i]: binary, 1 if vehicle v visits node i", + "u[i]: continuous, MTZ sequencing variable for node i" + ], + "objective": { + "sense": "min", + "expression": "sum_{v in V} sum_{i in N} sum_{j in N} dist[i][j] * x[v,i,j]" + }, + "constraints": [ + "single_visitation: forall j in C -> sum_{v in V} sum_{i in N} x[v,i,j] = 1", + "flow_conservation: forall v in V, j in N -> sum_{i in N} x[v,i,j] = sum_{k in N} x[v,j,k]", + "link_flow_to_visit_in: forall v in V, j in N -> sum_{i in N} x[v,i,j] = y[v,j]", + "link_flow_to_visit_out: forall v in V, j in N -> sum_{k in N} x[v,j,k] = y[v,j]", + "departure_limit: forall v in V -> sum_{j in C} x[v,0,j] <= 1", + "return_limit: forall v in V -> sum_{i in C} x[v,i,0] <= 1", + "no_self_loop_depot: forall v in V -> x[v,0,0] = 0", + "no_self_loop_customer: forall v in V, i in C -> x[v,i,i] = 0", + "mtz: forall i in C, j in C, i != j -> u[i] - u[j] + M * sum_{v in V} x[v,i,j] <= M - 1", + "u_lower_bound: forall i in C -> u[i] >= 1", + "u_upper_bound: forall i in C -> u[i] <= |C|", + "u_depot: u[0] = 0" + ] +} +``` + +### Common Pitfalls +- Using `=` instead of `<=` for depot constraints, which forces all vehicles to be used unnecessarily. +- Setting `M` too small in MTZ constraints, which can cut off valid solutions. +- Forgetting to prohibit self-loops (`x[v,i,i] = 0`), which can lead to degenerate, zero-distance "routes". +- Not linking assignment variables (`y[v,i]`) to flow constraints, resulting in disconnected visits. + +## Solving stage + +### Strategy Overview +Solve the MIP model using a high-performance solver like Gurobi or SCIP via their Python APIs. Configure for deterministic performance, extract routes by tracing arcs from the solution, and validate the solution's completeness and objective value. + +### Step 1 - Configure and Solve the Model +- Instantiate the solver (e.g., `gurobipy.Model()` or `pywraplp.Solver.CreateSolver("SCIP")`). +- Set solver parameters for reproducibility and performance: time limit, optimality gap tolerance, thread count, and random seed. +- Build the model using the formulation template, add all variables and constraints. +- Call the solver's `optimize()` or `Solve()` method. + +### Step 2 - Check Solution Status and Extract Values +- Check the solver status (e.g., `model.status == GRB.OPTIMAL` or `solver.OPTIMAL`). +- If optimal or feasible, retrieve variable values: `x_val[v,i,j] = x[v,i,j].X` (Gurobi) or `x[v,i,j].solution_value()` (OR-Tools). +- Use a tolerance (e.g., `0.5`) to convert fractional values from tolerances to binary decisions. + +### Step 3 - Reconstruct Vehicle Routes +- For each vehicle `v`: + - If `sum_{j in C} x_val[v,0,j] < 0.5`, the vehicle is unused. + - Otherwise, start at the depot (`node = 0`). + - While `node` is not the depot or the route is just starting: + - Find `next_node` such that `x_val[v, node, next_node] > 0.5`. + - Append `next_node` to the route for vehicle `v`. + - Set `node = next_node`. + - The route ends when `next_node` is `0` (return to depot). +- Collect all non-empty routes. + +### Step 4 - Validate and Report the Solution +- Verify every customer node appears in exactly one reconstructed route. +- Calculate the total distance by summing `dist[i][j]` for each arc `(i,j)` in the reconstructed routes. +- Compare this calculated distance to the solver's reported objective value to catch extraction errors. +- Report the routes, used vehicle count, and total distance. + +### Code Usage +```python +# build model from formulation +import gurobipy as gp +from gurobipy import GRB + +model = gp.Model('VRP') +# ... create variables x, y, u using model.addVar() ... +# ... add objective using model.setObjective() ... +# ... add all constraints using model.addConstr() ... + +# solve with status / termination checks +model.setParam('TimeLimit', 30) +model.setParam('MIPGap', 0.0001) +model.setParam('Threads', 4) +model.setParam('Seed', 42) +model.optimize() + +if model.status == GRB.OPTIMAL or model.status == GRB.TIME_LIMIT: + # Extract solution values into a dictionary x_val + x_val = {} + for v in V: + for i in N: + for j in N: + x_val[(v,i,j)] = x[v,i,j].X + # Reconstruct routes + routes = {} + for v in V: + route = [0] + current = 0 + while True: + next_node = None + for j in N: + if x_val.get((v, current, j), 0) > 0.5: + next_node = j + break + if next_node is None or next_node == 0: + break + route.append(next_node) + current = next_node + if len(route) > 1: # Vehicle left depot + route.append(0) # Close the loop if not already at depot + routes[v] = route + # Validate and print results + # ... +else: + print("No feasible solution found") +``` + +### Common Pitfalls +- Not checking for `TIME_LIMIT` status, which may still provide a good feasible solution. +- Using a naive tolerance (e.g., `0.0`) for binary variable values, missing values slightly above zero due to solver tolerances. +- Infinite loops during route reconstruction if the solution contains subtours (indicating failed MTZ constraints). +- Forgetting to close the route by appending the depot return node in the reconstruction logic. + +# Workflow 2 (Compact Formulation with Per-Vehicle MTZ) + +## Modeling stage + +### Strategy Overview +This workflow uses a more compact MIP formulation, common in academic literature, where MTZ sequencing variables `u[v,i]` are defined per vehicle. This eliminates the need for separate assignment variables (`y[v,i]`) and can reduce model size for solvers like CBC. Vehicle usage remains optional. + +### Step 1 - Define Sets and Parameters +- Define the set of vehicles `V`, the set of all nodes `N` (depot at `0`), and customer nodes `C = N \ {0}`. +- Define distance matrix `dist[i][j]`. +- Define `N_customer` = `|C|` for use in MTZ bounds. + +### Step 2 - Create Decision Variables +- Create binary variable `x[v,i,j]` for each vehicle `v` and node pair `(i,j)`. +- Create continuous variable `u[v,i]` for each vehicle `v` and node `i`. This represents the position of node `i` in vehicle `v`'s route (0 for depot). + +### Step 3 - Formulate Objective and Visitation Constraints +- **Objective**: Minimize total distance: `min sum_{v,i,j} dist[i][j] * x[v,i,j]`. +- **Single Visitation**: Each customer is visited by exactly one vehicle: `sum_{v, j != i} x[v,i,j] = 1` for all `i` in `C`. (Sum over all outgoing arcs from `i` across vehicles). +- **Flow Conservation**: For each vehicle `v` and node `i`: `sum_j x[v,i,j] = sum_j x[v,j,i]`. + +### Step 4 - Implement Depot and Vehicle Usage Logic +- **Optional Departure**: For each vehicle `v`: `sum_{j in C} x[v,0,j] <= 1`. +- **Optional Return**: For each vehicle `v`: `sum_{i in C} x[v,i,0] <= 1`. +- **Depot Flow Balance**: For each vehicle `v`: `sum_{j in C} x[v,0,j] = sum_{i in C} x[v,i,0]`. Ensures a vehicle that leaves depot must return. +- **No Self-Loops**: `x[v,i,i] = 0` for all `v`, `i`. + +### Step 5 - Apply Per-Vehicle MTZ Subtour Elimination +- For each vehicle `v` and customer pair `(i,j)` with `i != j`: `u[v,i] - u[v,j] + N_customer * x[v,i,j] <= N_customer - 1`. +- Set bounds: For customers `i` in `C`: `1 <= u[v,i] <= N_customer`. For the depot: `u[v,0] = 0`. + +### Formulation Template +```json +{ + "sets": [ + "V: set of vehicles (index v)", + "N: set of all nodes (index i, j), depot at 0", + "C: set of customer nodes, C = N \\ {0}" + ], + "parameters": [ + "dist[i][j]: distance from node i to j", + "N_customer: number of customer nodes, |C|" + ], + "decision_variables": [ + "x[v,i,j]: binary, 1 if vehicle v travels from i to j", + "u[v,i]: continuous, position of node i in vehicle v's route (MTZ)" + ], + "objective": { + "sense": "min", + "expression": "sum_{v in V} sum_{i in N} sum_{j in N} dist[i][j] * x[v,i,j]" + }, + "constraints": [ + "single_visitation: forall i in C -> sum_{v in V} sum_{j in N, j != i} x[v,i,j] = 1", + "flow_conservation: forall v in V, i in N -> sum_{j in N} x[v,i,j] = sum_{j in N} x[v,j,i]", + "departure_limit: forall v in V -> sum_{j in C} x[v,0,j] <= 1", + "return_limit: forall v in V -> sum_{i in C} x[v,i,0] <= 1", + "depot_balance: forall v in V -> sum_{j in C} x[v,0,j] = sum_{i in C} x[v,i,0]", + "no_self_loop: forall v in V, i in N -> x[v,i,i] = 0", + "mtz: forall v in V, i in C, j in C, i != j -> u[v,i] - u[v,j] + N_customer * x[v,i,j] <= N_customer - 1", + "u_customer_lb: forall v in V, i in C -> u[v,i] >= 1", + "u_customer_ub: forall v in V, i in C -> u[v,i] <= N_customer", + "u_depot: forall v in V -> u[v,0] = 0" + ] +} +``` + +### Common Pitfalls +- Using the total number of nodes `|N|` instead of `|C|` in the MTZ `N_customer` constant, making the constraint too weak. +- Omitting the depot flow balance constraint, which can lead to paths that start but do not end at the depot, or vice-versa. +- Applying MTZ constraints for all `i,j` in `N` (including depot), which is unnecessary and can cause infeasibility. +- Forgetting to fix `u[v,0]=0`, which is required for the MTZ formulation to work correctly. + +## Solving stage + +### Strategy Overview +Solve the model using an open-source MIP solver like CBC via a modeling library such as Pyomo. Emphasize model construction clarity, solver configuration for practical performance, and robust solution extraction that handles unused vehicles. + +### Step 1 - Build Model with Pyomo and Configure Solver +- Define a Pyomo `ConcreteModel`. +- Create `model.x` as a `Var` indexed by `(v,i,j)` with domain `Binary`. +- Create `model.u` as a `Var` indexed by `(v,i)` with bounds. +- Add the objective and all constraints using Pyomo's `Constraint` and `Objective` components. +- Create a solver object (e.g., `SolverFactory('cbc')`) and set options: time limit, relative gap tolerance. + +### Step 2 - Solve and Check Termination Condition +- Execute `solver.solve(model)`. +- Check the solver termination condition (`model.solutions[0].termination_condition`). Accept `optimal`, `feasible`, or `maxTimeLimit` with a solution. +- Check the solver status (`model.solutions[0].status`). Accept `ok`. + +### Step 3 - Extract Solution and Reconstruct Routes +- Load the solution into the model object. +- Retrieve variable values: `x_val = value(model.x[v,i,j])`. +- For each vehicle `v`, trace a route starting from the depot if `sum_{j} value(model.x[v,0,j]) > 0.5`. +- Follow the sequence of arcs where `x_val > 0.5` until returning to the depot. +- Collect routes for vehicles that left the depot. + +### Step 4 - Validate Solution and Compute Metrics +- Verify all customer nodes appear in exactly one extracted route. +- Compute the total distance by summing the distances of arcs in the extracted routes. +- Compare this computed distance to `value(model.objective)` to ensure consistency. +- Report the list of routes, number of used vehicles, and total distance. + +### Code Usage +```python +# build model from formulation +from pyomo.environ import ConcreteModel, Set, Param, Var, Binary, NonNegativeReals, Objective, Constraint, SolverFactory, value + +model = ConcreteModel('VRP_MTZ') +# Define sets +model.V = Set(initialize=range(num_vehicles)) +model.N = Set(initialize=range(num_nodes)) +model.C = Set(initialize=range(1, num_nodes)) # customers +# ... define parameters (dist) as a Param model.dist ... +# Variables +model.x = Var(model.V, model.N, model.N, domain=Binary) +model.u = Var(model.V, model.N, domain=NonNegativeReals, bounds=(0, num_customers)) +# Objective +model.obj = Objective(expr=sum(model.dist[i,j] * model.x[v,i,j] for v in model.V for i in model.N for j in model.N)) +# Add constraints (examples) +def single_visit_rule(model, i): + return sum(model.x[v,i,j] for v in model.V for j in model.N if j != i) == 1 +model.single_visit = Constraint(model.C, rule=single_visit_rule) +# ... add all other constraints ... + +# solve with status / termination checks +solver = SolverFactory('cbc') +solver.options['seconds'] = 30 +solver.options['ratio'] = 0.0001 +results = solver.solve(model) + +if results.solver.termination_condition == 'optimal' or results.solver.termination_condition == 'feasible': + if results.solver.status == 'ok': + # Extract solution + routes = {} + for v in model.V: + route = [0] + current = 0 + visited = set() + while True: + next_node = None + for j in model.N: + if value(model.x[v, current, j]) > 0.5: + next_node = j + break + if next_node is None or next_node in visited: # prevent cycles + break + if next_node == 0: + route.append(0) + break + route.append(next_node) + visited.add(next_node) + current = next_node + if len(route) > 1: + routes[v] = route + # Validate and report + # ... +else: + print("Solver did not find a feasible solution.") +``` + +### Common Pitfalls +- Not setting proper bounds on `u[v,i]` variables in Pyomo, leading to unbounded variables. +- Using `value()` on variables before loading the solution, resulting in `None` or default values. +- Inefficient constraint rule definitions that slow down model construction for large instances. +- Not handling the case where the solver hits a time limit but returns a feasible solution (`termination_condition == 'maxTimeLimit'`). diff --git a/skills/optskills/skill_library/multicommodityflowallocation.md b/skills/optskills/skill_library/multicommodityflowallocation.md new file mode 100644 index 0000000..739cd4a --- /dev/null +++ b/skills/optskills/skill_library/multicommodityflowallocation.md @@ -0,0 +1,225 @@ +--- +name: MultiCommodityFlowAllocation +description: | + Model and solve multi-source, multi-destination, multi-commodity allocation problems with exact demand satisfaction and linear profit maximization using structured LP formulations. +--- + +# Workflow 1 (OR-Tools GLOP for Dense LP) + +## Modeling stage + +### Strategy Overview +Formulate the allocation problem as a dense linear program using a three-index continuous variable. Leverage OR-Tools' GLOP solver for efficient solving of continuous LPs with equality constraints, using direct coefficient setting for clarity and control. + +### Step 1 - Define Dimensions and Data Structure +- Identify the three fundamental sets: sources (e.g., companies), destinations (e.g., regions), and commodities (e.g., products). +- Organize input parameters as nested lists or dictionaries: `profit[source][destination][commodity]` and `demand[destination][commodity]`. Ensure consistent indexing. + +### Step 2 - Create Decision Variables +- Instantiate a three-dimensional decision variable `x[source][destination][commodity]` representing the allocation quantity. +- Use `solver.NumVar(lower_bound, upper_bound, name)` with `lower_bound=0` and `upper_bound=solver.infinity()` to enforce non-negativity implicitly. +- Employ systematic naming (e.g., `f"x_{s}_{d}_{c}"`) for debugging and solution interpretation. + +### Step 3 - Formulate Demand Satisfaction Constraints +- For each destination-commodity pair, create an exact equality constraint: `sum(x[s][d][c] for all sources) == demand[d][c]`. +- Use `solver.Constraint(rhs, rhs)` to create a fixed constraint, then add coefficients via `constraint.SetCoefficient(variable, 1.0)` in a loop over sources. + +### Step 4 - Define Linear Profit Objective +- Construct the objective to maximize total profit: `sum(profit[s][d][c] * x[s][d][c] for all indices)`. +- Use `solver.Maximize()` or `solver.Minimize()` and set coefficients using `objective.SetCoefficient(variable, coefficient)` in a triple-nested loop. + +### Formulation Template +```json +{ + "sets": ["sources", "destinations", "commodities"], + "parameters": [ + "profit[sources][destinations][commodities]", + "demand[destinations][commodities]" + ], + "decision_variables": ["x[sources][destinations][commodities] >= 0"], + "objective": { + "sense": "max", + "expression": "sum(profit[s][d][c] * x[s][d][c])" + }, + "constraints": [ + "for each d in destinations, c in commodities: sum(x[s][d][c] for s in sources) == demand[d][c]" + ] +} +``` + +### Common Pitfalls +- Inconsistent indexing between profit coefficients and variable creation loops, leading to incorrect objective values. +- Forgetting to set the upper bound to `solver.infinity()` for unbounded variables, which is safe for allocation quantities. +- Creating constraints with incorrect right-hand side values by not matching the demand parameter's indexing. + +## Solving stage + +### Strategy Overview +Solve the constructed LP model using the GLOP backend, implement robust status checking, extract and verify the solution, and output structured results. + +### Step 1 - Initialize Solver and Solve +- Create the solver instance: `solver = pywraplp.Solver.CreateSolver('GLOP')`. +- Invoke `solver.Solve()` to obtain the solution status. + +### Step 2 - Check Solver Status +- Verify the solution status is either `solver.OPTIMAL` or `solver.FEASIBLE` before proceeding. +- If status is `solver.INFEASIBLE` or `solver.ABNORMAL`, handle the error by returning a diagnostic payload with the status code. + +### Step 3 - Extract and Validate Solution +- Retrieve the objective value using `solver.Objective().Value()`. +- Iterate through all variables, using `variable.solution_value()` to get allocations. Filter values above a small tolerance (e.g., `1e-6`) to focus on meaningful flows. +- Optionally, verify constraint satisfaction by recomputing total allocation per destination-commodity pair and comparing to original demand within tolerance. + +### Step 4 - Output Structured Results +- Package the results into a structured dictionary or JSON object containing the solver status, objective value, and a list of non-zero allocations with their indices and values. + +### Code Usage +```python +# build model from formulation +from ortools.linear_solver import pywraplp + +solver = pywraplp.Solver.CreateSolver('GLOP') +# ... (variable creation, constraint and objective building as per modeling stage) + +# solve with status / termination checks +status = solver.Solve() +result_payload = {} + +if status in (solver.OPTIMAL, solver.FEASIBLE): + result_payload['status'] = 'OPTIMAL' if status == solver.OPTIMAL else 'FEASIBLE' + result_payload['objective_value'] = solver.Objective().Value() + allocations = [] + for s in sources: + for d in destinations: + for c in commodities: + var = x[s][d][c] + val = var.solution_value() + if val > 1e-6: + allocations.append({'source': s, 'dest': d, 'commodity': c, 'value': val}) + result_payload['allocations'] = allocations + # Optional verification loop + for d in destinations: + for c in commodities: + total = sum(x[s][d][c].solution_value() for s in sources) + # assert abs(total - demand[d][c]) < 1e-6 +else: + result_payload['status'] = 'FAILED' + result_payload['solver_status_code'] = status +``` + +### Common Pitfalls +- Accessing `variable.solution_value()` without checking solver status first, which may cause errors. +- Using an inappropriate tolerance for filtering near-zero values, either missing small allocations or including numerical noise. +- Not providing a fallback error payload, making integration with automated systems difficult. + +# Workflow 2 (Pyomo with Open-Source Solver) + +## Modeling stage + +### Strategy Overview +Model the problem using Pyomo's abstract or concrete model paradigm, defining explicit Sets, Parameters, Variables, Constraints, and Objective. This approach is solver-agnostic and facilitates use with open-source solvers like GLPK or CBC. + +### Step 1 - Define Pyomo Sets and Parameters +- Declare Pyomo Set objects for `model.sources`, `model.destinations`, `model.commodities`. +- Define Pyomo Param objects or use standard Python dictionaries for `model.profit` and `model.demand`, ensuring they are indexed by the appropriate Set tuples. + +### Step 2 - Create Decision Variables with Domain +- Instantiate a Pyomo Var `model.x` indexed over the three sets: `model.x = pyo.Var(model.sources, model.destinations, model.commodities, domain=pyo.NonNegativeReals)`. +- Using `domain=pyo.NonNegativeReals` enforces non-negativity bounds efficiently. + +### Step 3 - Formulate Demand Constraints via Rules +- Define a constraint rule `demand_rule(model, d, c)` that returns the equality: `sum(model.x[s, d, c] for s in model.sources) == model.demand[d, c]`. +- Create a Pyomo Constraint object indexed over destinations and commodities using this rule. + +### Step 4 - Construct Linear Objective +- Define the objective expression using a sum over all indices: `sum(model.profit[s, d, c] * model.x[s, d, c] for s in model.sources for d in model.destinations for c in model.commodities)`. +- Use `sense=pyo.maximize` in the Objective constructor. + +### Formulation Template +```json +{ + "sets": ["sources", "destinations", "commodities"], + "parameters": [ + "profit[(sources, destinations, commodities)]", + "demand[(destinations, commodities)]" + ], + "decision_variables": ["x[sources, destinations, commodities] in NonNegativeReals"], + "objective": { + "sense": "max", + "expression": "sum(profit[s, d, c] * x[s, d, c])" + }, + "constraints": [ + "demand_constraint[destinations, commodities]: sum(x[s, d, c] for s in sources) == demand[d, c]" + ] +} +``` + +### Common Pitfalls +- Mixing Pyomo Param indexing with Python dictionary lookups, causing key errors during model construction. +- Defining constraint rules with incorrect indentation or scope, leading to uninitialized model components. +- Forgetting to initialize all Sets before using them to index Parameters or Variables. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a configured open-source solver (GLPK or CBC), perform comprehensive status and termination checks, load the solution, and extract results with validation. + +### Step 1 - Configure and Execute Solver +- Create a solver factory: `solver = pyo.SolverFactory('glpk')` (or `'cbc'`). +- Set practical options: `solver.options['tmlim'] = 30` for time limit, `solver.options['mipgap'] = 0.0` for optimality tolerance. +- Solve the model: `results = solver.solve(model, tee=False)`. + +### Step 2 - Verify Solver Status and Termination +- Check that `results.solver.status == pyo.SolverStatus.ok`. +- Verify `results.solver.termination_condition` is either `pyo.TerminationCondition.optimal` or `pyo.TerminationCondition.feasible`. +- If checks fail, inspect the termination condition and status for error diagnosis. + +### Step 3 - Load Solution and Extract Values +- If `load_solutions=False` was used, call `model.solutions.load_from(results)`. +- Retrieve the objective value via `pyo.value(model.obj)`. +- Iterate through `model.x` to get variable values using `pyo.value(model.x[s, d, c])`, filtering near-zero values. + +### Step 4 - Validate and Structure Output +- Optionally, verify constraint satisfaction by recomputing sums per destination-commodity pair. +- Package results into a dictionary containing status, objective value, and a list of non-zero allocations. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +model = pyo.ConcreteModel() +model.sources = pyo.Set(initialize=sources_list) +model.destinations = pyo.Set(initialize=destinations_list) +model.commodities = pyo.Set(initialize=commodities_list) +# ... (define parameters, variables, constraints, objective as per modeling stage) + +# solve with status / termination checks +solver = pyo.SolverFactory('glpk') +solver.options['tmlim'] = 30 +results = solver.solve(model, tee=False) + +result_payload = {} +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in (TerminationCondition.optimal, TerminationCondition.feasible)): + result_payload['status'] = str(results.solver.termination_condition) + result_payload['objective_value'] = pyo.value(model.obj) + allocations = [] + for s in model.sources: + for d in model.destinations: + for c in model.commodities: + val = pyo.value(model.x[s, d, c]) + if val > 1e-6: + allocations.append({'source': s, 'dest': d, 'commodity': c, 'value': val}) + result_payload['allocations'] = allocations +else: + result_payload['status'] = 'FAILED' + result_payload['solver_status'] = str(results.solver.status) + result_payload['termination_condition'] = str(results.solver.termination_condition) +``` + +### Common Pitfalls +- Assuming the solution is automatically loaded into the model; always check if `load_solutions` behavior is as expected. +- Not handling the `feasible` termination condition, which still provides a valid but potentially suboptimal solution. +- Setting solver options incorrectly for the chosen solver (e.g., using `'seconds'` for GLPK instead of `'tmlim'`). diff --git a/skills/optskills/skill_library/multicommodityflowoptimization.md b/skills/optskills/skill_library/multicommodityflowoptimization.md new file mode 100644 index 0000000..38cdf4c --- /dev/null +++ b/skills/optskills/skill_library/multicommodityflowoptimization.md @@ -0,0 +1,213 @@ +--- +name: MultiCommodityFlowOptimization +description: | + Model multi-commodity flow problems with shared arc capacities and solve them using linear programming, with robust status checking and solution validation. +--- + +# Workflow 1 (Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a multi-commodity flow network using Pyomo's declarative modeling. Define separate sets for origins, destinations, and commodities, with three-dimensional flow variables. Use linear equality constraints for supply and demand, and linear inequality constraints for shared arc capacities. + +### Step 1 - Define Sets and Data Structure +- Define three index sets: `ORIGINS`, `DESTINATIONS`, `COMMODITIES`. +- Organize parameters as dictionaries with tuple keys: `supply[(i,k)]`, `demand[(j,k)]`, `cost[(i,j,k)]`, `capacity[(i,j)]`. + +### Step 2 - Declare Decision Variables +- Create a continuous, non-negative variable `model.x[i, j, k]` representing the flow quantity of commodity `k` from origin `i` to destination `j`. +- Use `domain=pyo.NonNegativeReals`. + +### Step 3 - Formulate Supply and Demand Constraints +- For each origin `i` and commodity `k`: `sum(model.x[i, j, k] for j in DESTINATIONS) == supply[i, k]`. +- For each destination `j` and commodity `k`: `sum(model.x[i, j, k] for i in ORIGINS) == demand[j, k]`. + +### Step 4 - Formulate Arc Capacity Constraints +- For each origin-destination pair `(i, j)`: `sum(model.x[i, j, k] for k in COMMODITIES) <= capacity[i, j]`. + +### Step 5 - Define Linear Cost Objective +- Minimize total cost: `model.obj = pyo.Objective(expr=sum(cost[i, j, k] * model.x[i, j, k] for i, j, k), sense=pyo.minimize)`. + +### Formulation Template +```json +{ + "sets": ["ORIGINS", "DESTINATIONS", "COMMODITIES"], + "parameters": [ + "supply[ORIGINS, COMMODITIES]", + "demand[DESTINATIONS, COMMODITIES]", + "cost[ORIGINS, DESTINATIONS, COMMODITIES]", + "capacity[ORIGINS, DESTINATIONS]" + ], + "decision_variables": ["x[ORIGINS, DESTINATIONS, COMMODITIES] >= 0"], + "objective": { + "sense": "min", + "expression": "sum(cost[i,j,k] * x[i,j,k])" + }, + "constraints": [ + "supply_con[i,k]: sum_j x[i,j,k] == supply[i,k]", + "demand_con[j,k]: sum_i x[i,j,k] == demand[j,k]", + "capacity_con[i,j]: sum_k x[i,j,k] <= capacity[i,j]" + ] +} +``` + +### Common Pitfalls +- Assuming total supply-demand balance guarantees feasibility; arc capacity constraints can still cause infeasibility. +- Not verifying that parameter dictionaries are correctly indexed, leading to `KeyError` during constraint construction. +- Using overly complex constraint rules that obscure the linear structure; keep rules simple and direct. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using an open-source LP solver (HiGHS or CBC). Configure solver options for performance and determinism, then rigorously check solver status and termination condition before extracting results. Implement post-solution verification. + +### Step 1 - Instantiate and Configure Solver +- Create solver: `solver = pyo.SolverFactory("highs")` (or `"cbc"`). +- Set options: `solver.options["time_limit"] = 30`, `solver.options["mip_rel_gap"] = 0.0`. + +### Step 2 - Solve and Check Status +- Execute: `results = solver.solve(model, tee=False)`. +- Check `results.solver.status == SolverStatus.ok`. +- Check `results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}`. + +### Step 3 - Extract and Validate Solution +- Extract objective value: `obj_val = float(pyo.value(model.obj))`. +- Optionally, iterate over variables to collect non-zero flows (`if pyo.value(model.x[i,j,k]) > 1e-6`). +- Programmatically verify all constraints by recomputing sums and comparing to limits with a tolerance (e.g., `1e-6`). + +### Step 4 - Output Structured Results +- Output the objective value in a parseable format: `print(f"RESULT:{obj_val}")`. +- For detailed output, package results (status, objective, non-zero flows) into a JSON string. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Build model from formulation (sets, variables, constraints, objective as defined above) +model = pyo.ConcreteModel() +# ... model construction code ... + +# Solve with status / termination checks +solver = pyo.SolverFactory("highs") +solver.options["time_limit"] = 30 +results = solver.solve(model, tee=False) + +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}): + obj_val = float(pyo.value(model.obj)) + print(f"RESULT:{obj_val}") + # Optional verification and detailed output +else: + print("ERROR:Solver failed or no feasible solution found.") +``` + +### Common Pitfalls +- Not checking both solver status and termination condition, leading to errors when extracting values from infeasible or failed solves. +- Setting solver options that conflict with the solver's environment (e.g., `threads` when already initialized). +- Interpreting solver errors generically; examine specific error messages for diagnostics like infeasibility. + +# Workflow 2 (OR-Tools with GLOP) + +## Modeling stage + +### Strategy Overview +Model the multi-commodity flow problem using OR-Tools' linear solver wrapper. Build the model imperatively by creating variables and adding constraints directly. Use nested loops for systematic construction, leveraging OR-Tools' efficient coefficient setting. + +### Step 1 - Initialize Solver and Data Structures +- Create solver: `solver = pywraplp.Solver.CreateSolver('GLOP')`. +- Organize data as nested dictionaries or lists: `supply[i][k]`, `demand[j][k]`, `cost[i][j][k]`, `capacity[i][j]`. + +### Step 2 - Create Flow Variables +- Use nested loops over origins, destinations, commodities to create variables: `x[i][j][k] = solver.NumVar(0, solver.infinity(), f'x_{i}_{j}_{k}')`. + +### Step 3 - Add Supply and Demand Constraints +- For each origin `i` and commodity `k`: `solver.Add(sum(x[i][j][k] for j in destinations) == supply[i][k])`. +- For each destination `j` and commodity `k`: `solver.Add(sum(x[i][j][k] for i in origins) == demand[j][k])`. + +### Step 4 - Add Arc Capacity Constraints +- For each origin-destination pair `(i, j)`: `solver.Add(sum(x[i][j][k] for k in commodities) <= capacity[i][j])`. + +### Step 5 - Set Linear Cost Objective +- Create objective: `objective = solver.Objective()`. +- In nested loops, set coefficients: `objective.SetCoefficient(x[i][j][k], cost[i][j][k])`. +- Call `objective.SetMinimization()`. + +### Formulation Template +```json +{ + "sets": ["origins", "destinations", "commodities"], + "parameters": [ + "supply[origins][commodities]", + "demand[destinations][commodities]", + "cost[origins][destinations][commodities]", + "capacity[origins][destinations]" + ], + "decision_variables": ["x[origins][destinations][commodities] >= 0"], + "objective": { + "sense": "min", + "expression": "sum(cost[i][j][k] * x[i][j][k])" + }, + "constraints": [ + "supply: for each i,k: sum_j x[i][j][k] == supply[i][k]", + "demand: for each j,k: sum_i x[i][j][k] == demand[j][k]", + "capacity: for each i,j: sum_k x[i][j][k] <= capacity[i][j]" + ] +} +``` + +### Common Pitfalls +- Using `solver.infinity()` for variable upper bounds when capacities are already constrained; it's acceptable but less explicit. +- Building constraints inefficiently with repeated `solver.Add()` inside deep loops; pre-aggregate expressions where possible. +- Not using descriptive variable names, making debugging difficult for larger instances. + +## Solving stage + +### Strategy Overview +Solve the model using OR-Tools' GLOP solver for linear programming. After solving, verify solution feasibility by checking all constraints against the variable values. Output results in a structured format suitable for automation. + +### Step 1 - Execute Solve +- Call `solver.Solve()`. +- Check status: `status = solver.Solve()`. + +### Step 2 - Validate Solution Status +- Verify `status == pywraplp.Solver.OPTIMAL` (or `FEASIBLE` for non-optimal but feasible solutions). +- If not optimal/feasible, report infeasibility and avoid extracting values. + +### Step 3 - Extract and Verify Solution +- Extract objective value: `obj_val = objective.Value()`. +- For verification, compute actual flows: `flow_val = x[i][j][k].solution_value()`. +- Recalculate supply usage, demand satisfaction, and capacity usage, comparing to limits with a tolerance (e.g., `1e-6`). + +### Step 4 - Output Results +- Output the objective value: `print(f"RESULT:{obj_val}")`. +- Optionally, output a JSON payload with non-zero flows and verification details. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# Build model from formulation +solver = pywraplp.Solver.CreateSolver('GLOP') +# ... variable and constraint creation ... + +# Solve with status / termination checks +status = solver.Solve() +if status in (pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE): + obj_val = solver.Objective().Value() + print(f"RESULT:{obj_val}") + # Optional verification + for i in origins: + for k in commodities: + used = sum(x[i][j][k].solution_value() for j in destinations) + if abs(used - supply[i][k]) > 1e-6: + print(f"Warning: Supply violation for ({i},{k})") +else: + print("ERROR:No optimal or feasible solution found.") +``` + +### Common Pitfalls +- Assuming `solver.Solve()` returns only `OPTIMAL`; also check for `FEASIBLE`. +- Not performing post-solution verification, potentially accepting numerically invalid solutions. +- Using the solver's default parameters for large problems; consider setting time limits if needed. diff --git a/skills/optskills/skill_library/multicommoditynetworkflow.md b/skills/optskills/skill_library/multicommoditynetworkflow.md new file mode 100644 index 0000000..9f71951 --- /dev/null +++ b/skills/optskills/skill_library/multicommoditynetworkflow.md @@ -0,0 +1,276 @@ +--- +name: MultiCommodityNetworkFlow +description: | + Model and solve multi-commodity flow problems with shared arc capacities and commodity-specific limits, minimizing linear transportation cost. +--- + +# Workflow 1 (Pyomo with Commercial/Open-Source MILP Solver) + +## Modeling stage + +### Strategy Overview +Use Pyomo's abstract modeling syntax to define a multi-commodity flow problem with three-index variables, separating model logic from data. Designed for solvers like Gurobi, CPLEX, or HiGHS that accept Pyomo models. + +### Step 1 - Define Sets and Parameters +- Define sets `nodes`, `commodities`, and directed `arcs` (as tuples `(i, j)` where `i != j`). For undirected edges, create two directed arcs `(i, j)` and `(j, i)`. +- Define parameters: + - `unit_cost[arcs]`: linear cost per unit flow. + - `arc_capacity[arcs]`: total capacity shared by all commodities. + - `commodity_cap[arcs, commodities]`: upper bound for each commodity on each arc. + - `net_demand[nodes, commodities]`: net demand per node and commodity (negative for supply, positive for demand, zero for transshipment). Ensure sum over all nodes equals zero for each commodity. + +### Step 2 - Create Decision Variables +- Create a non-negative continuous variable `flow[i, j, p]` for each arc `(i, j)` and commodity `p`. +- Embed the per-commodity upper bound directly in the variable declaration using `bounds=(0, commodity_cap[i, j, p])`. This avoids adding separate commodity-capacity constraints. + +### Step 3 - Formulate Constraints +- **Flow Conservation**: For each node `n` and commodity `p`: `sum(flow[n, j, p] for j) - sum(flow[i, n, p] for i) = net_demand[n, p]`. +- **Total Arc Capacity**: For each arc `(i, j)`: `sum(flow[i, j, p] for p in commodities) <= arc_capacity[i, j]`. + +### Step 4 - Define Objective +- Define a linear objective to minimize total cost: `sum(unit_cost[i, j] * flow[i, j, p] for all arcs and commodities)`. + +### Formulation Template +```json +{ + "sets": [ + "nodes", + "commodities", + "arcs (dimen=2)" + ], + "parameters": [ + "unit_cost[arcs]", + "arc_capacity[arcs]", + "commodity_cap[arcs, commodities]", + "net_demand[nodes, commodities]" + ], + "decision_variables": [ + "flow[arcs, commodities] with bounds [0, commodity_cap]" + ], + "objective": { + "sense": "min", + "expression": "sum(unit_cost[i,j] * flow[i,j,p] for (i,j) in arcs for p in commodities)" + }, + "constraints": [ + "flow_conservation[n, p]: sum(flow[n,j,p] for j) - sum(flow[i,n,p] for i) = net_demand[n, p]", + "total_capacity[i, j]: sum(flow[i,j,p] for p in commodities) <= arc_capacity[i, j]" + ] +} +``` + +### Common Pitfalls +- Forgetting to exclude self-loops when generating the `arcs` set. +- Mismatching the sign convention for `net_demand` in the flow conservation rule, causing infeasibility. +- Defining `commodity_cap` as a scalar instead of a parameter indexed by arc and commodity, which incorrectly applies the same limit everywhere. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a configured solver instance, carefully check the solution status, and load results. Includes validation of the solution against problem constraints. + +### Step 1 - Configure and Execute Solver +- Instantiate the solver factory (e.g., `SolverFactory("highs")`). +- Set solver options for deterministic performance: `time_limit=[TIME_LIMIT]`, `mip_rel_gap` (or `MIPGap`) to 0.0 for LP, `threads`, and `seed`. +- Call `solve(model, ...)` with `tee=False` for quiet operation or `tee=True` for logs. + +### Step 2 - Check Status and Load Solution +- Check the solver status (`SolverStatus.ok`) and termination condition (`TerminationCondition.optimal` or `.feasible`). +- If using a solver like HiGHS, use `load_solutions=False` and then `model.solutions.load_from(results)` to avoid loading errors. +- If the solve failed (status not ok or termination condition not optimal/feasible), output a structured JSON error message and do not attempt to extract solution values. + +### Step 3 - Validate and Report Solution +- Compute the objective value from the model variable to verify against the solver's reported value. +- Programmatically check flow conservation and total capacity constraints with a small tolerance (e.g., `1e-6`). +- Print only non-zero flows for clarity and output the final result in a parseable format (e.g., `RESULT:{objective_value}`). + +### Code Usage +```python +import pyomo.environ as pyo +import json + +def solve_multi_commodity_flow(model): + # Configure solver + solver = pyo.SolverFactory("highs") + solver.options["time_limit"] = [TIME_LIMIT] + solver.options["mip_rel_gap"] = 0.0 + solver.options["threads"] = 4 + solver.options["seed"] = 42 + + # Solve with controlled solution loading + results = solver.solve(model, tee=False, load_solutions=False) + + # Check status and load + if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in {pyo.TerminationCondition.optimal, + pyo.TerminationCondition.feasible}): + model.solutions.load_from(results) + # Validate key constraints + tolerance = 1e-6 + # ... (add validation checks here) + # Output result + print(f"RESULT:{pyo.value(model.obj)}") + # Optional: print non-zero flows + for idx in model.flow: + if pyo.value(model.flow[idx]) > tolerance: + print(f" flow{idx} = {pyo.value(model.flow[idx])}") + else: + error_info = { + "status": "failed", + "reason": "infeasible_or_error", + "solver_status": str(results.solver.status), + "termination_condition": str(results.solver.termination_condition) + } + print(f"RESULT_JSON:{json.dumps(error_info)}") +``` + +### Common Pitfalls +- Assuming the solution is loaded automatically; some solvers require explicit `load_solutions=False` and manual loading. +- Not checking both `solver.status` and `termination_condition`, leading to misinterpretation of suboptimal or feasible solutions. +- Forgetting to set `mip_rel_gap=0.0` for linear problems, causing unnecessary early stopping. +- Attempting to extract or print solution values when the solve failed; always guard with a status check first. + +# Workflow 2 (Google OR-Tools Linear Solver) + +## Modeling stage + +### Strategy Overview +Use Google OR-Tools' linear solver API (`pywraplp`) to construct the model imperatively. Suitable for direct integration and offers fine-grained control over variable and constraint creation. + +### Step 1 - Initialize Solver and Create Variables +- Create a solver instance (e.g., `GLOP` for LP, `CBC` for MIP). +- For each arc `(i, j)` and commodity `p`, create a continuous variable with lower bound 0 and upper bound set to `commodity_cap[i, j, p]`. + +### Step 2 - Add Flow Conservation Constraints +- For each node `n` and commodity `p`, create a constraint: `sum(outgoing_flow) - sum(incoming_flow) = net_demand[n, p]`. +- Set coefficients carefully: `+1` for outgoing flows, `-1` for incoming flows. + +### Step 3 - Add Total Arc Capacity Constraints +- For each arc `(i, j)`, create a constraint: `sum(flow[i, j, p] for all p) <= arc_capacity[i, j]`. + +### Step 4 - Define Objective +- Set the objective to minimize `sum(unit_cost[i, j] * flow[i, j, p] for all arcs and commodities)`. + +### Formulation Template +```json +{ + "sets": [ + "nodes", + "commodities", + "arcs (dimen=2)" + ], + "parameters": [ + "unit_cost[arcs]", + "arc_capacity[arcs]", + "commodity_cap[arcs, commodities]", + "net_demand[nodes, commodities]" + ], + "decision_variables": [ + "flow[arcs, commodities] with bounds [0, commodity_cap]" + ], + "objective": { + "sense": "min", + "expression": "sum(unit_cost[i,j] * flow[i,j,p] for (i,j) in arcs for p in commodities)" + }, + "constraints": [ + "flow_conservation[n, p]: sum(flow[n,j,p] for j) - sum(flow[i,n,p] for i) = net_demand[n, p]", + "total_capacity[i, j]: sum(flow[i,j,p] for p in commodities) <= arc_capacity[i, j]" + ] +} +``` + +### Common Pitfalls +- Incorrectly mapping data structures (e.g., capacities stored per neighbor list) to the `(i, j, p)` indexing, requiring explicit lookup logic. +- Setting variable upper bounds to `infinity` instead of the commodity capacity, missing an entire layer of constraints. +- Adding the commodity capacity as a separate constraint instead of a variable bound, which is less efficient. + +## Solving stage + +### Strategy Overview +Solve using OR-Tools' native `Solve()` method, check the result status, extract the solution values, and perform post-solution validation. + +### Step 1 - Solve and Check Status +- Call `solver.Solve()`. +- Check the result status: `pywraplp.Solver.OPTIMAL` or `FEASIBLE` indicates a successful solve. Any other status indicates failure; do not extract solution values. + +### Step 2 - Extract Solution +- If optimal or feasible, retrieve the objective value via `solver.Objective().Value()`. +- Iterate through all flow variables, extracting their solution values with `var.solution_value()`. + +### Step 3 - Validate and Report +- Programmatically verify flow conservation and total capacity constraints using the extracted solution values. +- Filter and print only non-zero flows (above a small tolerance) for interpretability. +- Output the total cost in a consistent format. + +### Code Usage +```python +from ortools.linear_solver import pywraplp +import json + +def solve_with_ortools(node_data, commodity_data, arc_data): + # Initialize solver + solver = pywraplp.Solver.CreateSolver('GLOP') # Use 'CBC' for MIP + if not solver: + raise RuntimeError("Solver not available.") + + # Create variables and store in a dictionary + flow_vars = {} + for (i, j) in arc_data['arcs']: + for p in commodity_data['commodities']: + ub = commodity_data['commodity_cap'].get((i, j, p), solver.infinity()) + flow_vars[(i, j, p)] = solver.NumVar(0.0, ub, f'flow_{i}_{j}_{p}') + + # Add flow conservation constraints + for n in node_data['nodes']: + for p in commodity_data['commodities']: + demand = node_data['net_demand'].get((n, p), 0.0) + constraint = solver.Constraint(demand, demand) + # Outgoing flows: coefficient +1 + for (i, j, k), var in flow_vars.items(): + if i == n and k == p: + constraint.SetCoefficient(var, 1.0) + if j == n and k == p: + constraint.SetCoefficient(var, -1.0) + + # Add total arc capacity constraints + for (i, j) in arc_data['arcs']: + cap = arc_data['arc_capacity'].get((i, j), solver.infinity()) + cap_constraint = solver.Constraint(-solver.infinity(), cap) + for p in commodity_data['commodities']: + var = flow_vars.get((i, j, p)) + if var is not None: + cap_constraint.SetCoefficient(var, 1.0) + + # Set objective + objective = solver.Objective() + for (i, j, p), var in flow_vars.items(): + cost = arc_data['unit_cost'].get((i, j), 0.0) + objective.SetCoefficient(var, cost) + objective.SetMinimization() + + # Solve + status = solver.Solve() + + # Process result + if status in [pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE]: + total_cost = solver.Objective().Value() + # Validation checks can be added here + print(f"RESULT:{total_cost}") + tolerance = 1e-6 + for idx, var in flow_vars.items(): + val = var.solution_value() + if val > tolerance: + print(f" flow{idx} = {val}") + else: + error_info = { + "status": "failed", + "reason": "infeasible_or_error", + "solver_status": status + } + print(f"RESULT_JSON:{json.dumps(error_info)}") +``` + +### Common Pitfalls +- Confusing `OPTIMAL` and `FEASIBLE` statuses; both may be acceptable depending on the stopping criteria. +- Not using variable bounds for commodity capacities, which reduces solver performance. +- Failing to set the coefficient sign correctly in flow conservation constraints, leading to incorrect material balance. +- Attempting to extract solution values when the solve status is not `OPTIMAL` or `FEASIBLE`; always guard with a status check first. diff --git a/skills/optskills/skill_library/multiindexflowassignmentlp.md b/skills/optskills/skill_library/multiindexflowassignmentlp.md new file mode 100644 index 0000000..53617fb --- /dev/null +++ b/skills/optskills/skill_library/multiindexflowassignmentlp.md @@ -0,0 +1,217 @@ +--- +name: MultiIndexFlowAssignmentLP +description: | + Model and solve multi-index flow assignment problems with linear profit maximization and exact demand satisfaction using continuous variables, implemented via either direct solver APIs or algebraic modeling frameworks. +--- + +# Workflow 1 (Direct Solver API - OR-Tools GLOP) + +## Modeling stage + +### Strategy Overview +This workflow uses a direct solver API (OR-Tools) to construct the linear program without an intermediate algebraic modeling language. It is efficient for straightforward LP formulations and provides fine-grained control over constraint building. + +### Step 1 - Define Multi-Index Variables +- Create a non-negative continuous decision variable for each combination of source, destination, and product type. +- Use a descriptive naming convention (e.g., `x_src_dst_prod`) for debugging clarity. +- Store variables in a dictionary keyed by their indices for easy access during coefficient setting. + +### Step 2 - Enforce Demand Satisfaction Constraints +- For each unique (destination, product) pair, create a single linear equality constraint. +- Set the constraint's lower and upper bounds to the exact demand value. +- Within the constraint, sum contributions from all source variables for that specific pair by setting their coefficients to 1. + +### Step 3 - Formulate Linear Profit Objective +- Create a maximization objective. +- For each variable, add a term equal to the variable multiplied by its corresponding profit coefficient. +- Use nested loops over all index dimensions to systematically set all objective coefficients. + +### Formulation Template +```json +{ + "sets": [ + "sources", + "destinations", + "products" + ], + "parameters": [ + "profit[source, destination, product]", + "demand[destination, product]" + ], + "decision_variables": [ + "x[source, destination, product] >= 0" + ], + "objective": { + "sense": "max", + "expression": "sum(profit[s,d,p] * x[s,d,p] for all s,d,p)" + }, + "constraints": [ + "demand_satisfaction[d,p]: sum(x[s,d,p] for all s) == demand[d,p] for all d,p" + ] +} +``` + +### Common Pitfalls +- Forgetting to set the coefficient for every variable within a constraint, leading to incorrect sums. +- Using loose tolerances when checking solution feasibility; always verify constraints with a small epsilon (e.g., 1e-6). +- Creating variables or constraints inside deeply nested loops inefficiently; prefer pre-initializing data structures. + +## Solving stage + +### Strategy Overview +Solve the constructed model using the GLOP linear programming solver. Focus on robust status checking, solution verification, and extracting a clean, sparse solution representation. + +### Step 1 - Configure and Execute Solver +- Instantiate the GLOP solver. +- Invoke the solve method and capture the result status. + +### Step 2 - Validate Solution Status +- Check for an `OPTIMAL` or `FEASIBLE` status. Treat both as successful solves for practical purposes. +- If the status is not acceptable, output a structured error message (e.g., JSON) with the solver status for debugging. + +### Step 3 - Extract and Verify Solution +- Retrieve the objective value. +- Iterate through all decision variables, collecting those with a value greater than a defined tolerance (e.g., 1e-6). +- Programmatically verify that the extracted solution satisfies all demand constraints by recalculating sums. + +### Code Usage +```python +# build model from formulation +from ortools.linear_solver import pywraplp + +solver = pywraplp.Solver.CreateSolver('GLOP') +# ... (variable and constraint creation as per modeling stage) + +# solve with status / termination checks +status = solver.Solve() + +if status in [solver.OPTIMAL, solver.FEASIBLE]: + objective_value = solver.Objective().Value() + solution = {} + tolerance = 1e-6 + # Extract non-zero flows + for var_name, var in variables_dict.items(): + val = var.solution_value() + if val > tolerance: + solution[var_name] = val + # Verification loop (pseudo-code) + # for each (d,p): assert abs(sum(solution values) - demand[d,p]) < tolerance +else: + # Output structured error + error_info = {'status': status, 'message': 'Solver did not find a solution.'} + print(f"RESULT_JSON:{error_info}") +``` + +### Common Pitfalls +- Assuming an `OPTIMAL` status guarantees exact constraint satisfaction; always perform numerical verification. +- Extracting all variable values for large problems, which can clutter output; filter by tolerance. +- Not handling the case where the solver might return `FEASIBLE` but not `OPTIMAL`. + +# Workflow 2 (Algebraic Modeling - Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +This workflow uses Pyomo, an algebraic modeling language, to declaratively define sets, parameters, variables, and constraints. It separates the problem formulation from the solver interface, improving readability and maintainability for complex models. + +### Step 1 - Declare Model Sets and Parameters +- Define Pyomo `Set` objects for each index dimension (e.g., sources, destinations, products). +- Define `Param` objects for profit and demand data, initialized from nested dictionaries keyed by tuples. + +### Step 2 - Define Decision Variables +- Create a `Var` object indexed over the Cartesian product of the defined sets. +- Specify the domain as `pyo.NonNegativeReals` for continuous, non-negative flows. + +### Step 3 - Construct Objective and Constraints +- Define the objective as a `pyo.Objective` using a summation expression over all indices. +- Create demand satisfaction constraints using a `pyo.Constraint` rule that, for each (destination, product) pair, sums the appropriate variables and enforces equality with the demand parameter. + +### Formulation Template +```json +{ + "sets": [ + "sources", + "destinations", + "products" + ], + "parameters": [ + "profit[source, destination, product]", + "demand[destination, product]" + ], + "decision_variables": [ + "x[source, destination, product] in NonNegativeReals" + ], + "objective": { + "sense": "max", + "expression": "sum(profit[s,d,p] * x[s,d,p] for all s,d,p)" + }, + "constraints": [ + "demand_con[d,p]: sum(x[s,d,p] for all s) == demand[d,p] for all d,p" + ] +} +``` + +### Common Pitfalls +- Incorrectly initializing multi-dimensional parameters; ensure the dictionary keys match the index order of the Pyomo `Param` declaration. +- Defining constraints inside loops instead of using Pyomo's indexed `Constraint` construct, which is less efficient and harder to debug. +- Confusing the order of indices in variable definitions, leading to mismatched coefficients in the objective and constraints. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a high-performance LP solver like HiGHS or CBC. Implement a robust pattern for checking solver status, handling solution loading, and performing post-solution verification. + +### Step 1 - Configure Solver and Solve +- Create a solver instance (e.g., `SolverFactory('highs')` or `'cbc'`). +- Set appropriate options such as time limit (`seconds`) and optimality gap tolerance (`ratio`). +- Execute the solve command, optionally with `tee=True` for debugging output. + +### Step 2 - Check Termination Status +- Verify that `results.solver.status` is `SolverStatus.ok`. +- Check that `results.solver.termination_condition` is either `TerminationCondition.optimal` or `TerminationCondition.feasible`. +- If status checks fail, output a structured error before attempting to load the solution. + +### Step 3 - Extract and Verify Solution +- Load the solution into the model instance. +- Retrieve the objective value via `pyo.value(model.obj)`. +- Iterate through the indexed variable object, collecting values above a tolerance into a sparse solution dictionary. +- Implement a verification loop to confirm demand constraints are satisfied numerically. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +model = pyo.ConcreteModel() +# ... (set, parameter, variable, objective, constraint creation as per modeling stage) + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') # or 'cbc' +solver_options = {'seconds': 30, 'ratio': 0.0} +results = solver.solve(model, options=solver_options, tee=False) + +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in [TerminationCondition.optimal, + TerminationCondition.feasible]): + # Solution loading is typically automatic for common solvers + objective_value = pyo.value(model.obj) + solution = {} + tolerance = 1e-6 + for index in model.x.index_set(): + val = pyo.value(model.x[index]) + if val > tolerance: + solution[f'x[{index}]'] = val + # Add verification logic here +else: + error_info = { + 'solver_status': str(results.solver.status), + 'termination_condition': str(results.solver.termination_condition) + } + print(f"RESULT_JSON:{error_info}") +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition` before extracting results. +- Forgetting that some solvers require explicit solution loading (`load_solutions=False`/`model.solutions.load_from(results)`). +- Setting overly restrictive solver options (like `ratio=0.0`) on very large problems, potentially causing long solve times. diff --git a/skills/optskills/skill_library/multiperiodproductioninventorylp.md b/skills/optskills/skill_library/multiperiodproductioninventorylp.md new file mode 100644 index 0000000..1db62c1 --- /dev/null +++ b/skills/optskills/skill_library/multiperiodproductioninventorylp.md @@ -0,0 +1,218 @@ +--- +name: MultiPeriodProductionInventoryLP +description: | + Model and solve multi-period production-inventory problems with resource capacity constraints and sales limits using linear programming, with workflows for both direct solver APIs and algebraic modeling languages. +--- + +# Workflow 1 (Direct Solver API - OR-Tools) + +## Modeling stage + +### Strategy Overview +This workflow uses a direct solver API (OR-Tools) to construct the model imperatively. Variables and constraints are created one by one, which offers fine-grained control and is well-suited for prototyping or embedding within larger applications. + +### Step 1 - Define Core Sets and Parameters +- Define sets for `periods`, `products`, and `machines` as lists or ranges. +- Create dictionaries for parameters: `profit_per_unit[p]`, `holding_cost`, `machine_time_required[m][p]`, `machine_capacity[m][t]`, `max_sales[p][t]`, and `target_inventory[p]`. + +### Step 2 - Create Decision Variables +- Create three dictionaries of non-negative continuous variables: `production[t][p]`, `inventory[t][p]`, and `sales[t][p]`. +- Optionally, embed simple bounds like `max_sales` directly in the variable creation for efficiency. + +### Step 3 - Formulate Inventory Balance Constraints +- For the initial period (`t=0`), add constraints: `production[0][p] == sales[0][p] + inventory[0][p]`. +- For subsequent periods (`t>0`), add constraints: `inventory[t-1][p] + production[t][p] == sales[t][p] + inventory[t][p]`. + +### Step 4 - Add Resource and Sales Constraints +- For each machine `m` and period `t`, add capacity constraints: `sum(machine_time_required[m][p] * production[t][p] for p in products) <= machine_capacity[m][t]`. +- If not embedded as variable bounds, add sales limit constraints: `sales[t][p] <= max_sales[p][t]`. + +### Step 5 - Set Terminal Conditions and Objective +- Add terminal inventory constraints: `inventory[final_period][p] == target_inventory[p]`. +- Formulate the objective to maximize total profit: `sum(profit_per_unit[p] * sales[t][p] - holding_cost * inventory[t][p] for t in periods for p in products)`. + +### Formulation Template +```json +{ + "sets": ["periods", "products", "machines"], + "parameters": [ + "profit_per_unit[product]", + "holding_cost", + "machine_time_required[machine][product]", + "machine_capacity[machine][period]", + "max_sales[product][period]", + "target_inventory[product]" + ], + "decision_variables": [ + "production[period][product] >= 0", + "inventory[period][product] >= 0", + "sales[period][product] >= 0" + ], + "objective": { + "sense": "max", + "expression": "sum(profit_per_unit[p] * sales[t][p] - holding_cost * inventory[t][p])" + }, + "constraints": [ + "initial_balance: production[0][p] == sales[0][p] + inventory[0][p]", + "balance: inventory[t-1][p] + production[t][p] == sales[t][p] + inventory[t][p] for t>0", + "machine_capacity: sum(machine_time_required[m][p] * production[t][p]) <= machine_capacity[m][t]", + "sales_limit: sales[t][p] <= max_sales[p][t]", + "terminal_inventory: inventory[final_period][p] == target_inventory[p]" + ] +} +``` + +### Common Pitfalls +- Forgetting to handle the initial period (`t=0`) separately in the inventory balance, leading to an index error for `t-1`. +- Defining sales limits as separate constraints when they could be more efficiently set as upper bounds during variable creation. +- Not verifying that all parameter dictionaries are fully populated for all indices, which can cause silent constraint omissions. + +## Solving stage + +### Strategy Overview +Solve the model using OR-Tools' linear solver wrapper. Focus on setting practical limits, robust status checking, and systematic solution extraction for validation and reporting. + +### Step 1 - Initialize Solver and Set Limits +- Create a solver instance: `solver = pywraplp.Solver.CreateSolver('GLOP')` for LP or `'CBC'` for MIP. +- Set a time limit to prevent hanging: `solver.SetTimeLimit(30000)` (milliseconds). + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve()`. +- Check for optimal or feasible status: `if status in (solver.OPTIMAL, solver.FEASIBLE):`. + +### Step 3 - Extract and Validate Solution +- Extract variable values using `.solution_value()` and store in structured dictionaries. +- Programmatically verify key constraints (e.g., inventory balance, capacity usage) against the extracted values to catch potential solver inconsistencies. + +### Step 4 - Report Results +- Print the objective value in a standard format: `print(f'RESULT:{solver.Objective().Value()}')`. +- Optionally, output detailed plans for production, inventory, and sales, summarizing only positive values for clarity. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('GLOP') +# ... (variable and constraint creation code) +solver.SetTimeLimit(30000) + +# solve with status / termination checks +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + # Extract solution + obj_val = solver.Objective().Value() + print(f'RESULT:{obj_val}') + # ... extract and validate variable values +else: + print('Solver failed to find a solution.') +``` + +### Common Pitfalls +- Assuming `solver.Solve()` returns only `OPTIMAL`; always also accept `FEASIBLE` for time-limited runs. +- Not setting a time limit, which can cause the process to hang on large or poorly formulated models. +- Extracting variable values without checking the solver status first, leading to errors. + +# Workflow 2 (Algebraic Modeling Language - Pyomo) + +## Modeling stage + +### Strategy Overview +This workflow uses an algebraic modeling language (Pyomo) to declare the model abstractly using sets, parameters, and rules. This approach enhances readability, maintainability, and is closer to the mathematical formulation. + +### Step 1 - Declare Abstract Sets and Parameters +- Use `pyo.Set()` to define abstract sets for `periods`, `products`, and `machines`. +- Use `pyo.Param()` within indexed sets to declare all necessary parameters (profit, costs, capacities, limits). + +### Step 2 - Define Decision Variables with Rules +- Define `pyo.Var()` for `production`, `inventory`, and `sales`, indexed over `periods` and `products`, with domain `pyo.NonNegativeReals`. +- Use `bounds` rule or parameter to set variable bounds (e.g., inventory capacity) where appropriate. + +### Step 3 - Construct Constraints with Rule Functions +- Define a rule function for inventory balance that uses conditional logic (`if t == 0`) or `Constraint.Skip` to handle the initial period. +- Define separate rule functions for machine capacity and sales limits, iterating over the respective sets. + +### Step 4 - Enforce Terminal Conditions and Objective +- Add a terminal inventory constraint as a simple equality rule indexed by products for the final period. +- Define the objective using a `pyo.Objective` rule that sums profit minus holding costs across all indices. + +### Formulation Template +```json +{ + "sets": ["periods", "products", "machines"], + "parameters": [ + "profit_per_unit[product]", + "holding_cost", + "machine_time_required[machine, product]", + "machine_capacity[machine, period]", + "max_sales[product, period]", + "target_inventory[product]" + ], + "decision_variables": [ + "production[period, product] in NonNegativeReals", + "inventory[period, product] in NonNegativeReals", + "sales[period, product] in NonNegativeReals" + ], + "objective": { + "sense": "maximize", + "expression": "sum(profit_per_unit[p] * sales[t,p] - holding_cost * inventory[t,p])" + }, + "constraints": [ + "balance_rule(t, p): inventory[t-1,p] + production[t,p] == sales[t,p] + inventory[t,p], with special case for t=0", + "machine_cap_rule(m, t): sum(machine_time_required[m,p] * production[t,p]) <= machine_capacity[m,t]", + "sales_limit_rule(t, p): sales[t,p] <= max_sales[p,t]", + "terminal_inv_rule(p): inventory[final_period,p] == target_inventory[p]" + ] +} +``` + +### Common Pitfalls +- Using 1-based indexing in rule logic while Python uses 0-based indexing for list-derived sets, causing off-by-one errors. +- Forgetting to handle the `t=0` case in the balance rule, resulting in an attempt to access `inventory[-1, p]`. +- Declaring parameters without initializing all required indices, which leads to runtime errors when the rule is constructed. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using an external solver interface (e.g., HiGHS, CBC). Configure solver options, implement robust solution loading, and perform post-solution verification. + +### Step 1 - Configure and Execute Solver +- Create a solver instance: `solver = pyo.SolverFactory('highs')`. +- Set solver options: `solver.options['time_limit'] = 30`, `solver.options['threads'] = 4`. + +### Step 2 - Solve with Robust Status Handling +- Solve with `load_solutions=False`: `results = solver.solve(model, load_solutions=False, tee=False)`. +- Check the solver status and termination condition before loading the solution. + +### Step 3 - Load Solution and Verify +- If the status is acceptable (`optimal` or `feasible`), load the solution: `model.solutions.load_from(results)`. +- Programmatically verify constraint satisfaction by iterating through constraints and comparing the left-hand side and right-hand side values. + +### Step 4 - Analyze and Report +- Extract the objective value: `pyo.value(model.objective)`. +- Print the result in the standard format and optionally output detailed plans. +- Calculate and report utilizations (e.g., machine usage vs. capacity) to identify bottlenecks. + +### Code Usage +```python +# build model from formulation +model = pyo.ConcreteModel() +# ... (set, parameter, variable, constraint, objective definition) + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = 30 +results = solver.solve(model, load_solutions=False) + +from pyomo.opt import SolverStatus, TerminationCondition +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in (TerminationCondition.optimal, TerminationCondition.feasible)): + model.solutions.load_from(results) + print(f'RESULT:{pyo.value(model.objective)}') + # ... perform verification and detailed reporting +else: + print('Solver failed to find a solution.') +``` + +### Common Pitfalls +- Loading the solution without checking the termination condition, which may load an infeasible or suboptimal point. +- Not using `load_solutions=False`, which can cause confusion if the solver fails but the model retains a previous solution. +- Omitting post-solution verification, missing potential numerical issues or formulation errors. diff --git a/skills/optskills/skill_library/permutation_flow_shop_scheduling.md b/skills/optskills/skill_library/permutation_flow_shop_scheduling.md new file mode 100644 index 0000000..c10acd0 --- /dev/null +++ b/skills/optskills/skill_library/permutation_flow_shop_scheduling.md @@ -0,0 +1,193 @@ +--- +name: Permutation Flow Shop Scheduling +description: | + Model and solve permutation flow shop problems with makespan minimization using either CP-SAT with interval variables or MILP with position-based formulations. + +--- +# Workflow 1 (CP-SAT with Interval Variables) + +## Modeling stage + +### Strategy Overview +Model the problem using Constraint Programming (CP) concepts, specifically interval variables to represent job processing on each machine. This approach directly captures temporal relationships and uses native CP-SAT constraints for sequencing and resource capacity. + +### Step 1 - Define Core Variables +- Create an interval variable for each job-machine pair, defined by its start time, processing duration, and end time. +- Define a single makespan variable, constrained to be greater than or equal to the completion time of every job on the last machine. +- Set a global time horizon as an upper bound, typically the sum of all processing times. + +### Step 2 - Enforce Machine and Job Constraints +- Add a `NoOverlap` constraint on the interval variables for each machine to ensure only one job is processed at a time. +- For each job, add precedence constraints: the end time on machine `m` must be less than or equal to the start time on machine `m+1`. +- Link the makespan variable to the end times on the final machine. + +### Formulation Template +```json +{ + "sets": [ + "Jobs", + "Machines" + ], + "parameters": [ + "processing_time[j][m]" + ], + "decision_variables": [ + "interval[j][m] (start, size, end)", + "makespan" + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + "NoOverlap([interval[j][m] for j in Jobs]) for each m in Machines", + "end_before_start(interval[j][m], interval[j][m+1]) for each j in Jobs, m in Machines except last", + "makespan >= end(interval[j][last_machine]) for each j in Jobs" + ] +} +``` + +### Common Pitfalls +- Using an insufficiently large time horizon, which can make the model infeasible. Use the sum of all processing times. +- Forgetting to enforce the permutation rule (same job order on all machines). The `NoOverlap` constraint on each machine's intervals implicitly creates a sequence, but the job order must be consistent. This is enforced by using the same interval variable list per job across precedence constraints. + +## Solving stage + +### Strategy Overview +Use OR-Tools CP-SAT solver, configured for scheduling problems. The solver natively handles interval and sequence constraints. After solving, extract the job sequence by sorting based on start times. + +### Step 1 - Configure and Run Solver +- Instantiate the `CpSolver` and set key parameters: `max_time_in_seconds` for a time limit, `num_search_workers` for parallelism, and `random_seed` for reproducibility. +- For optimality proofs, set `relative_gap_limit = 0.0`. + +### Step 2 - Extract and Validate Solution +- Check the solver status (`OPTIMAL` or `FEASIBLE`). +- Extract the start time values for each job on the first machine. +- Infer the job permutation by sorting jobs based on these start times. +- Perform a forward pass calculation using the extracted sequence and the standard flow shop recursion to verify the makespan and schedule consistency. + +### Code Usage +```python +# build model from formulation +model = cp_model.CpModel() +# ... (create interval variables, add constraints, set objective) +solver = cp_model.CpSolver() +# set parameters +solver.parameters.max_time_in_seconds = [TIME_LIMIT] +solver.parameters.num_search_workers = [NUM_WORKERS] +solver.parameters.random_seed = [SEED] +# solve with status / termination checks +status = solver.Solve(model) +if status in [cp_model.OPTIMAL, cp_model.FEASIBLE]: + # Extract sequence + starts_on_m0 = {j: solver.Value(start_var[j][0]) for j in jobs} + sequence = sorted(jobs, key=lambda j: starts_on_m0[j]) + # Verify via forward pass + verified_makespan = forward_pass_calc(sequence, processing_times) +else: + print("No solution found.") +``` + +### Common Pitfalls +- Not checking solver status before extracting solution values, leading to runtime errors. +- Assuming the solver returns an optimal solution within the time limit; always handle feasible but non-optimal results. +- Incorrectly inferring sequence from unsorted start times that may have identical values; ensure a deterministic tie-breaker. + +# Workflow 2 (MILP with Position-Based Formulation) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) using binary assignment variables to link jobs to sequence positions. Continuous variables track completion times per position and machine, enabling a linear representation of the flow shop precedence rules. + +### Step 1 - Define Assignment and Completion Variables +- Create binary variable `x[j, p]` = 1 if job `j` is assigned to sequence position `p`. +- Create continuous variable `C[p, m]` for the completion time of the job in position `p` on machine `m`. + +### Step 2 - Enforce Permutation and Sequencing Logic +- Add constraints so each job is assigned to exactly one position and each position gets exactly one job. +- For the first position and machine, set the completion time based on the assigned job's processing time. +- For subsequent positions and machines, enforce the flow shop relationship using two linear constraints: `C[p, m] >= C[p-1, m] + processing_time` and `C[p, m] >= C[p, m-1] + processing_time`, where `processing_time` is summed over jobs using the `x[j, p]` variables. + +### Step 3 - Define the Objective +- Define the makespan as the completion time of the last position on the last machine: `C[last_position, last_machine]`. +- Set the objective to minimize this makespan. + +### Formulation Template +```json +{ + "sets": [ + "Jobs", + "Positions", + "Machines" + ], + "parameters": [ + "processing_time[j][m]" + ], + "decision_variables": [ + "x[j][p] ∈ {0,1}", + "C[p][m] ≥ 0" + ], + "objective": { + "sense": "min", + "expression": "C[last_position][last_machine]" + }, + "constraints": [ + "sum(x[j][p] for p in Positions) == 1 for each j in Jobs", + "sum(x[j][p] for j in Jobs) == 1 for each p in Positions", + "C[1][1] == sum(processing_time[j][1] * x[j][1] for j in Jobs)", + "C[p][m] >= C[p-1][m] + sum(processing_time[j][m] * x[j][p] for j in Jobs) for p>1, all m", + "C[p][m] >= C[p][m-1] + sum(processing_time[j][m] * x[j][p] for j in Jobs) for m>1, all p" + ] +} +``` + +### Common Pitfalls +- Using an insufficiently large "Big M" when linearizing disjunctive constraints is not needed in this formulation; the position-based model avoids this. +- Incorrectly indexing the initialization constraint for the first position and machine. +- Not ensuring the `Positions` set has the same cardinality as the `Jobs` set. + +## Solving stage + +### Strategy Overview +Use a MILP solver (e.g., Gurobi, HiGHS) via a modeling framework like Pyomo. Configure for optimality or time-limited search, extract the assignment variables to build the sequence, and validate the schedule. + +### Step 1 - Configure and Execute Solver +- Instantiate the solver (e.g., `SolverFactory('[SOLVER_NAME]')`). +- Set parameters: `TimeLimit`, `MIPGap` (to 0.0 for optimality), `Threads`, and `Seed` for reproducibility. + +### Step 2 - Process Solution and Verify +- Check the solver termination condition (`optimal`, `feasible`, or `timeLimit`). +- For each position `p`, find the job `j` where `x[j, p] > 0.5` to construct the sequence. +- Extract the makespan value from `C[last_position, last_machine]`. +- Optionally, verify the makespan by performing a forward pass calculation with the extracted sequence. + +### Code Usage +```python +# build model from formulation +model = pyo.ConcreteModel() +# ... (define sets, params, variables, constraints, objective) +solver = pyo.SolverFactory('[SOLVER_NAME]') +# solve with status / termination checks +results = solver.solve(model, options={'TimeLimit': [TIME_LIMIT], 'MIPGap': 0.0}) +if results.solver.termination_condition == pyo.TerminationCondition.optimal: + # Extract sequence + sequence = [] + for p in model.Positions: + for j in model.Jobs: + if pyo.value(model.x[j, p]) > 0.5: + sequence.append(j) + break + makespan = pyo.value(model.C[len(model.Jobs), len(model.Machines)]) + # Optional verification + verified_makespan = forward_pass_calc(sequence, processing_times) +elif results.solver.termination_condition == pyo.TerminationCondition.feasible: + print("Feasible solution found, may not be optimal.") +else: + print("Solver did not find a feasible solution.") +``` + +### Common Pitfalls +- Not verifying the solver status is `ok` before checking termination condition. +- Using a floating-point tolerance (e.g., 0.5) that is too tight for the solver's integrality tolerance when reading binary variables. +- Forgetting that Pyomo solution values are accessed via `pyo.value(var)` or `var.value`. diff --git a/skills/optskills/skill_library/permutation_flow_shop_scheduling_2.md b/skills/optskills/skill_library/permutation_flow_shop_scheduling_2.md new file mode 100644 index 0000000..ef7fc19 --- /dev/null +++ b/skills/optskills/skill_library/permutation_flow_shop_scheduling_2.md @@ -0,0 +1,232 @@ +--- +name: Permutation Flow Shop Scheduling +description: | + Model and solve permutation flow shop problems with makespan minimization using either complete enumeration for small instances or MILP/CP-SAT formulations for larger ones, ensuring the same job order across all machines. +--- + +# Workflow 1 (Exact Enumeration & Dynamic Programming) + +## Modeling stage + +### Strategy Overview +For small problem instances (typically ≤ 6 jobs), the optimal permutation can be found by evaluating all possible job sequences. The modeling focuses on efficiently calculating the makespan for any given sequence using dynamic programming, which directly enforces the flow shop precedence and machine capacity constraints inherent to the permutation structure. + +### Step 1 - Define Problem Data +- Identify the set of jobs and machines, along with the processing time for each job on each machine. +- Confirm the problem is a permutation flow shop: all jobs must be processed in the same order on every machine. + +### Step 2 - Define Makespan Calculation Logic +- Implement a function that, given a job sequence, computes the completion time for each job on each machine using the standard flow shop recurrence. +- This recurrence inherently respects the precedence chain (job must finish on machine m-1 before starting on m) and machine non-overlap (only one job processed at a time per machine). + +### Formulation Template +```json +{ + "sets": [ + {"name": "jobs", "description": "Set of all jobs to be scheduled."}, + {"name": "machines", "description": "Set of machines in the flow line, ordered by processing sequence."}, + {"name": "positions", "description": "Set of sequence positions, equal in size to the jobs set."} + ], + "parameters": [ + {"name": "processing_time", "set": ["jobs", "machines"], "description": "Time required to process job j on machine m."} + ], + "decision_variables": [ + {"name": "sequence", "type": "permutation", "set": ["jobs"], "description": "An ordered list of jobs defining the processing order on all machines."} + ], + "objective": { + "sense": "min", + "expression": "makespan", + "description": "Minimize the total completion time of the last job on the last machine." + }, + "constraints": [ + {"name": "makespan_definition", "description": "The makespan is the maximum completion time on the last machine, derived from the dynamic programming calculation."} + ] +} +``` + +### Common Pitfalls +- Assuming the recurrence works for any shop configuration; it is valid only for pure flow shops where all jobs visit all machines in the same order. +- Not accounting for zero-indexing in implementation, leading to off-by-one errors in the completion time matrix. +- Forgetting that the dynamic programming table `completion[i][m]` represents the completion time of the job in sequence position `i` on machine `m`. + +## Solving stage + +### Strategy Overview +The solving stage involves generating all possible job permutations, evaluating the makespan for each using the dynamic programming function, and selecting the sequence with the minimal makespan. This guarantees optimality for exhaustively searchable instances. + +### Step 1 - Enumerate Permutations +- Use `itertools.permutations` to generate all possible sequences of jobs. +- For each permutation, calculate its makespan using the predefined DP function. + +### Step 2 - Evaluate and Select Optimal Sequence +- Track the best makespan and the corresponding sequence during enumeration. +- After evaluating all permutations, the best sequence is the optimal schedule. + +### Step 3 - Extract Detailed Schedule +- Using the optimal sequence, re-run the DP calculation to populate the full `completion[i][m]` matrix. +- Derive start times for each operation as `start[i][m] = completion[i][m] - processing_time[sequence[i]][m]`. + +### Code Usage +```python +import itertools + +def calculate_makespan(seq, processing_times): + """Calculate makespan for a given job sequence in a permutation flow shop.""" + n_jobs = len(seq) + n_machines = len(processing_times[0]) + completion = [[0] * n_machines for _ in range(n_jobs)] + for i, job in enumerate(seq): + for m in range(n_machines): + if i == 0 and m == 0: + completion[i][m] = processing_times[job][m] + elif i == 0: + completion[i][m] = completion[i][m-1] + processing_times[job][m] + elif m == 0: + completion[i][m] = completion[i-1][m] + processing_times[job][m] + else: + completion[i][m] = max(completion[i-1][m], completion[i][m-1]) + processing_times[job][m] + return completion[-1][-1] + +# Main solving loop +best_makespan = float('inf') +best_sequence = None +for perm in itertools.permutations(jobs): + makespan = calculate_makespan(perm, processing_times) + if makespan < best_makespan: + best_makespan = makespan + best_sequence = perm +# best_sequence and best_makespan now hold the optimal solution +``` + +### Common Pitfalls +- Attempting full enumeration for n! > ~10000 permutations without considering runtime explosion. +- Not verifying the DP calculation with a manual example for a small instance. +- Storing all permutations and makespans in memory instead of tracking only the best during iteration. + +# Workflow 2 (MILP with Position-Based Assignment) + +## Modeling stage + +### Strategy Overview +For larger instances where enumeration is impractical, model the problem as a Mixed-Integer Linear Program (MILP). Use binary assignment variables to assign jobs to sequence positions, and continuous variables to track completion times. This formulation explicitly captures the precedence and disjunctive constraints while maintaining the permutation flow shop structure. + +### Step 1 - Define Assignment Variables +- Create binary variable `x[j, k]` which equals 1 if job `j` is assigned to sequence position `k`. +- Add constraints so each job is assigned to exactly one position and each position receives exactly one job. + +### Step 2 - Define Completion Time Variables +- Create continuous variable `C[k, m]` representing the completion time of the job in position `k` on machine `m`. +- Link completion times across machines and positions using big-M or direct precedence constraints based on the assignment. + +### Step 3 - Model Machine Precedence and Capacity +- For precedence (job flow): `C[k, m] >= C[k, m-1] + sum( x[j, k] * processing_time[j, m] for j in jobs )` for all k, m>0. +- For machine capacity (non-overlap): `C[k, m] >= C[k-1, m] + sum( x[j, k] * processing_time[j, m] for j in jobs )` for all k>0, m. + +### Step 4 - Define Makespan Objective +- Define makespan variable `makespan` with constraints `makespan >= C[k, last_machine]` for all positions `k`. +- Set objective to minimize `makespan`. + +### Formulation Template +```json +{ + "sets": [ + {"name": "jobs", "description": "Set of all jobs."}, + {"name": "machines", "description": "Ordered set of machines."}, + {"name": "positions", "description": "Set of sequence positions, indexed 0..n-1."} + ], + "parameters": [ + {"name": "processing_time", "set": ["jobs", "machines"], "description": "Processing time p_{j,m}."}, + {"name": "M", "description": "Large enough constant for big-M constraints, e.g., sum of all processing times."} + ], + "decision_variables": [ + {"name": "x", "type": "binary", "set": ["jobs", "positions"], "description": "1 if job j is assigned to position k."}, + {"name": "C", "type": "continuous", "set": ["positions", "machines"], "description": "Completion time at position k on machine m."}, + {"name": "makespan", "type": "continuous", "description": "Maximum completion time."} + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + {"name": "assign_each_job_once", "expression": "sum(x[j, k] for k in positions) == 1 for all j in jobs"}, + {"name": "fill_each_position", "expression": "sum(x[j, k] for j in jobs) == 1 for all k in positions"}, + {"name": "flow_precedence", "expression": "C[k, m] >= C[k, m-1] + sum(x[j, k] * processing_time[j, m] for j in jobs) for all k in positions, m in machines where m>0"}, + {"name": "machine_disjunctive", "expression": "C[k, m] >= C[k-1, m] + sum(x[j, k] * processing_time[j, m] for j in jobs) for all k in positions where k>0, m in machines"}, + {"name": "makespan_definition", "expression": "makespan >= C[k, last_machine] for all k in positions"}, + {"name": "nonnegative_C", "expression": "C[k, m] >= 0 for all k, m"} + ] +} +``` + +### Common Pitfalls +- Using an insufficiently large big-M value, weakening the LP relaxation and slowing convergence. +- Incorrectly indexing the `C` variable bounds, leading to infeasibility or incorrect schedules. +- Forgetting to enforce non-negativity for completion time variables. + +## Solving stage + +### Strategy Overview +Implement the MILP model using a modeling language (e.g., Pyomo) and solve it with a capable MIP solver (e.g., Gurobi, HiGHS). Configure the solver for deterministic performance and optimality, and post-process the solution to extract the job sequence and detailed schedule. + +### Step 1 - Model Instantiation +- Define sets, parameters, and variables as per the formulation. +- Add all constraints and the objective to the model. + +### Step 2 - Solver Configuration +- Set a reasonable time limit (e.g., `TimeLimit=30`). +- Set optimality tolerance to zero (`MIPGap=0.0`) for an exact solution. +- Fix the random seed and number of threads for reproducibility (e.g., `Seed=42`, `Threads=4`). + +### Step 3 - Solve and Check Status +- Invoke the solver and capture its termination status. +- Check if the solution is optimal or feasible. Handle timeouts or infeasibility with appropriate messages. + +### Step 4 - Extract and Validate Solution +- Retrieve the values of `x[j, k]` to construct the optimal job sequence. +- Compute the makespan from the `C` variables or the objective value. +- As a sanity check, recalculate the makespan for the extracted sequence using the DP function from Workflow 1 to verify consistency. + +### Code Usage +```python +import pyomo.environ as pyo + +# Create model +model = pyo.ConcreteModel() +# Define sets +model.J = pyo.Set(initialize=jobs) +model.K = pyo.Set(initialize=positions) +model.M = pyo.Set(initialize=machines) +# Define parameters +model.p = pyo.Param(model.J, model.M, initialize=processing_time) +# Define variables +model.x = pyo.Var(model.J, model.K, domain=pyo.Binary) +model.C = pyo.Var(model.K, model.M, domain=pyo.NonNegativeReals) +model.makespan = pyo.Var(domain=pyo.NonNegativeReals) +# Add constraints (examples) +def assign_each_job_rule(model, j): + return sum(model.x[j, k] for k in model.K) == 1 +model.assign_job = pyo.Constraint(model.J, rule=assign_each_job_rule) +# ... Add other constraints ... +# Define objective +model.obj = pyo.Objective(expr=model.makespan, sense=pyo.minimize) +# Solve +solver = pyo.SolverFactory('gurobi') +solver.options['TimeLimit'] = 30 +solver.options['MIPGap'] = 0.0 +results = solver.solve(model, tee=True) +# Check status and extract solution +if pyo.check_optimal_termination(results): + # Extract sequence + optimal_sequence = [] + for k in model.K: + for j in model.J: + if pyo.value(model.x[j, k]) > 0.5: + optimal_sequence.append(j) + break + makespan_val = pyo.value(model.makespan) +``` + +### Common Pitfalls +- Not checking solver termination status, leading to errors when trying to extract values from an unsolved model. +- Using default solver parameters which may be non-deterministic or may not prove optimality. +- Misinterpreting the `x` variable matrix; ensure the extracted sequence respects the position ordering. diff --git a/skills/optskills/skill_library/quadratic_assignment_problem_solver.md b/skills/optskills/skill_library/quadratic_assignment_problem_solver.md new file mode 100644 index 0000000..3040c08 --- /dev/null +++ b/skills/optskills/skill_library/quadratic_assignment_problem_solver.md @@ -0,0 +1,251 @@ +--- +name: Quadratic Assignment Problem Solver +description: | + Models and solves one-to-one assignment problems with quadratic interaction costs using either MILP linearization or direct permutation enumeration. + +--- +# Workflow 1 (MILP Linearization with OR-Tools CP-SAT) + +## Modeling stage + +### Strategy Overview +Transform the quadratic assignment objective into a linear form by introducing auxiliary binary variables for each product of assignment variables. This enables the use of MILP solvers like OR-Tools CP-SAT for small to medium instances. + +### Step 1 - Define Sets and Parameters +- Create index sets for items (e.g., `items = range(N)`) and slots (e.g., `slots = range(N)`). +- Define flow matrix `flow[i][k]` and distance matrix `dist[j][l]` as dictionaries or 2D lists keyed by index pairs. +- **Verify problem symmetry**: Check if flow and distance matrices are symmetric (flow[i][k] = flow[k][i], dist[j][l] = dist[l][j]) to confirm the problem structure and potentially optimize computation. + +### Step 2 - Declare Binary Assignment Variables +- Create binary variable `x[i][j]` for each item `i` and slot `j`, indicating assignment. +- Use `model.NewBoolVar(f'x_{i}_{j}')` in OR-Tools CP-SAT. + +### Step 3 - Enforce One-to-One Constraints +- Add constraint: `sum(x[i][j] for j in slots) == 1` for each item `i`. +- Add constraint: `sum(x[i][j] for i in items) == 1` for each slot `j`. + +### Step 4 - Linearize Quadratic Objective +- For each pair of items `(i, k)` with non-zero flow and each pair of slots `(j, l)` with non-zero distance, create auxiliary binary variable `z[i][k][j][l]` representing `x[i][j] * x[k][l]`. +- Add linearization constraints: + - `z[i][k][j][l] <= x[i][j]` + - `z[i][k][j][l] <= x[k][l]` + - `z[i][k][j][l] >= x[i][j] + x[k][l] - 1` +- Only create `z` variables for quadruples where `flow[i][k] > 0` and `dist[j][l] > 0` to reduce model size. + +### Step 5 - Build Objective +- Minimize `sum(flow[i][k] * dist[j][l] * z[i][k][j][l] for all relevant quadruples)`. +- **Scale to integers**: Convert floating-point flow/distance values to integers before passing to CP-SAT (requires integer coefficients). + +### Formulation Template +```json +{ + "sets": ["I: items", "J: slots"], + "parameters": ["flow[i][k]: interaction flow between items i and k", "dist[j][l]: distance between slots j and l"], + "decision_variables": [ + "x[i][j] ∈ {0,1}: assignment of item i to slot j", + "z[i][k][j][l] ∈ {0,1}: product x[i][j] * x[k][l] for non-zero flow/distance pairs" + ], + "objective": { + "sense": "min", + "expression": "sum_{i,k} sum_{j,l} flow[i][k] * dist[j][l] * z[i][k][j][l]" + }, + "constraints": [ + "sum_j x[i][j] == 1, ∀i ∈ I", + "sum_i x[i][j] == 1, ∀j ∈ J", + "z[i][k][j][l] <= x[i][j], ∀ relevant quadruples", + "z[i][k][j][l] <= x[k][l], ∀ relevant quadruples", + "z[i][k][j][l] >= x[i][j] + x[k][l] - 1, ∀ relevant quadruples" + ] +} +``` + +### Common Pitfalls +- Creating `z` variables for all quadruples (O(N^4)) even when flow or distance is zero, causing unnecessary model bloat. +- Forgetting to enforce both directions of the one-to-one constraint (items-to-slots and slots-to-items). +- Using floating-point flow/distance values without scaling to integers for CP-SAT. + +## Solving stage + +### Strategy Overview +Use OR-Tools CP-SAT solver with parallel search and time limits. Extract results with proper status checking and output in structured JSON format. + +### Step 1 - Configure Solver +- Create `CpSolver()` instance. +- Set parameters: `max_time_in_seconds=[TIME_LIMIT]`, `num_search_workers=8`, `random_seed=42`, `relative_gap_limit=0.0`. + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve(model)`. +- Check if status is `cp_model.OPTIMAL` or `cp_model.FEASIBLE`. + +### Step 3 - Extract Results +- Retrieve objective value via `solver.ObjectiveValue()`. +- Extract assignment by iterating over `x` variables and checking `solver.Value(x[i][j]) == 1`. +- Build assignment dictionary mapping item index to slot index. + +### Step 4 - Output and Validate +- Output JSON with keys: `status`, `objective_value`, `assignment`. +- **Include verification step**: Recompute cost for the best assignment using the original quadratic formula to ensure correctness. + +### Code Usage +```python +from ortools.sat.python import cp_model + +def solve_qap_cpsat(items, slots, flow, dist, time_limit=30.0): + model = cp_model.CpModel() + N = len(items) + x = {} + for i in items: + for j in slots: + x[i, j] = model.NewBoolVar(f'x_{i}_{j}') + + # One-to-one constraints + for i in items: + model.Add(sum(x[i, j] for j in slots) == 1) + for j in slots: + model.Add(sum(x[i, j] for i in items) == 1) + + # Linearization + z = {} + for i in items: + for k in items: + if flow[i][k] == 0: + continue + for j in slots: + for l in slots: + if dist[j][l] == 0: + continue + z[i, k, j, l] = model.NewBoolVar(f'z_{i}_{k}_{j}_{l}') + model.Add(z[i, k, j, l] <= x[i, j]) + model.Add(z[i, k, j, l] <= x[k, l]) + model.Add(z[i, k, j, l] >= x[i, j] + x[k, l] - 1) + + # Objective + obj_expr = sum(flow[i][k] * dist[j][l] * z[i, k, j, l] + for (i, k, j, l) in z) + model.Minimize(obj_expr) + + # Solve + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = time_limit + solver.parameters.num_search_workers = 8 + solver.parameters.random_seed = 42 + solver.parameters.relative_gap_limit = 0.0 + status = solver.Solve(model) + + if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + assignment = {i: next(j for j in slots if solver.Value(x[i, j]) == 1) for i in items} + return { + "status": "OPTIMAL" if status == cp_model.OPTIMAL else "FEASIBLE", + "objective_value": solver.ObjectiveValue(), + "assignment": assignment + } + else: + return {"status": "FAILURE", "solver_status": status} +``` + +### Common Pitfalls +- Not scaling floating-point parameters to integers before passing to CP-SAT (requires integer coefficients). +- Ignoring the `FEASIBLE` status when optimality is not guaranteed within time limit. +- Forgetting to set `random_seed` for reproducibility across runs. + +# Workflow 2 (Direct Permutation Enumeration) + +## Modeling stage + +### Strategy Overview +For small instances (N ≤ 8), enumerate all possible assignments as permutations of slots. This avoids solver dependencies and provides exact optimal solutions through brute-force search. + +### Step 1 - Define Sets and Parameters +- Create list of items `items = list(range(N))` and slots `slots = list(range(N))`. +- Define flow matrix `flow[i][k]` and distance matrix `dist[j][l]` as 2D lists or NumPy arrays. +- **Verify problem symmetry**: Check if flow and distance matrices are symmetric (flow[i][k] = flow[k][i], dist[j][l] = dist[l][j]) to confirm the problem structure and enable computational optimizations. +- **Note small problem size**: For N ≤ 8, complete enumeration is feasible (N! ≤ 40320 permutations). For N ≤ 10, enumeration may still be practical depending on hardware. + +### Step 2 - Model as Permutation Problem +- Recognize that a one-to-one assignment is equivalent to a permutation `π` where item `i` is assigned to slot `π[i]`. +- The objective becomes: `sum_{i,k} flow[i][k] * dist[π[i]][π[k]]`. +- **Leverage symmetry for optimization**: Since matrices are symmetric, compute only upper triangle (i < k) and double the result. + +### Step 3 - No Explicit Variables or Constraints +- The permutation structure inherently satisfies one-to-one constraints. +- No decision variables or constraints are needed; the search space is all permutations of `N` elements. + +### Formulation Template +```json +{ + "sets": ["I: items (0..N-1)", "J: slots (0..N-1)"], + "parameters": ["flow[i][k]: interaction flow between items i and k", "dist[j][l]: distance between slots j and l"], + "decision_variables": ["π: permutation mapping item index to slot index"], + "objective": { + "sense": "min", + "expression": "sum_{i,k} flow[i][k] * dist[π[i]][π[k]]" + }, + "constraints": ["π is a bijection from I to J (implicitly satisfied by permutation structure)"] +} +``` + +### Common Pitfalls +- Assuming this approach scales beyond N=8 (factorial growth makes it infeasible). +- Forgetting to handle symmetric flow/distance matrices to avoid double-counting in objective computation. +- Not using early pruning or symmetry breaking for larger instances within the limit. + +## Solving stage + +### Strategy Overview +Use `itertools.permutations` to generate all possible assignments, compute the objective for each, and track the minimum. This provides a simple, exact solution for small instances. + +### Step 1 - Generate Permutations +- Use `itertools.permutations(range(N))` to iterate over all slot assignments for items. + +### Step 2 - Compute Objective for Each Permutation +- For each permutation `perm`, compute cost as `sum(flow[i][k] * dist[perm[i]][perm[k]] for i in range(N) for k in range(N))`. +- **Optimize computation**: If matrices are symmetric, compute only upper triangle (i < k) and double the result. Include diagonal terms (i=k) separately. + +### Step 3 - Track Best Solution +- Initialize `best_cost = float('inf')` and `best_perm = None`. +- Update when a lower cost is found. + +### Step 4 - Output Results +- Return JSON with status `"OPTIMAL"`, objective value, and assignment dictionary mapping item index to slot index. +- **Include verification step**: Recompute cost for the best assignment using the original quadratic formula to ensure correctness. +- **Confirm uniqueness (optional)**: After finding optimal cost, enumerate all permutations again to check if multiple optimal solutions exist. + +### Code Usage +```python +import itertools +import json + +def solve_qap_enumeration(items, slots, flow, dist): + N = len(items) + best_cost = float('inf') + best_perm = None + + for perm in itertools.permutations(range(N)): + cost = 0 + # Leverage symmetry: compute only upper triangle and double + for i in range(N): + for k in range(i, N): # i <= k + if i == k: + cost += flow[i][i] * dist[perm[i]][perm[i]] + else: + cost += 2 * flow[i][k] * dist[perm[i]][perm[k]] + if cost < best_cost: + best_cost = cost + best_perm = perm + + assignment = {i: best_perm[i] for i in range(N)} + return { + "status": "OPTIMAL", + "objective_value": best_cost, + "assignment": assignment + } +``` + +### Common Pitfalls +- Using `itertools.permutations` for N > 8 (will cause memory/time explosion). +- Not exploiting symmetry (e.g., if flow or distance matrices are symmetric, compute only upper triangle and double). +- Forgetting to convert permutation indices to match slot indexing convention. + +## Verification and Output (Both Workflows) +- After obtaining a solution, recompute the objective value using the assignment and the original quadratic formula to verify correctness. +- Present the final answer as a numeric value (e.g., `[OBJECTIVE_VALUE]`) for integration into automated pipelines. diff --git a/skills/optskills/skill_library/quadratic_portfolio_optimization.md b/skills/optskills/skill_library/quadratic_portfolio_optimization.md new file mode 100644 index 0000000..5b62d65 --- /dev/null +++ b/skills/optskills/skill_library/quadratic_portfolio_optimization.md @@ -0,0 +1,234 @@ +--- +name: Quadratic Portfolio Optimization +description: | + Formulate and solve portfolio variance minimization as a quadratic program with linear constraints, handling incomplete covariance data and verifying solution feasibility. + +--- + +# Workflow 1 (Pyomo with Commercial QP Solver) + +## Modeling stage + +### Strategy Overview +Formulate the portfolio allocation problem as a convex quadratic program (QP) using Pyomo's algebraic modeling. The objective is to minimize portfolio variance (a quadratic function of weights and covariance matrix) subject to linear budget, return target, and diversification constraints. + +### Step 1 - Define Sets and Parameters +- Define an indexed set for assets. +- Store expected returns as a parameter indexed by the asset set. +- Store the covariance matrix as a symmetric parameter indexed by asset pairs. +- Define scalar parameters for the minimum required return and maximum allowed weight per asset. + +### Step 2 - Declare Decision Variables +- Declare continuous, non-negative variables for portfolio weights. +- Enforce upper bounds per asset directly in the variable declaration (e.g., `bounds=(0, max_weight)`). + +### Step 3 - Formulate Quadratic Objective +- Construct the portfolio variance objective as the double summation: `sum(weight[i] * covariance[i,j] * weight[j] for i in assets for j in assets)`. +- Set the objective sense to minimize. + +### Step 4 - Implement Linear Constraints +- Add a linear equality constraint enforcing the sum of weights equals one (budget constraint). +- Add a linear inequality constraint ensuring the weighted sum of expected returns meets or exceeds the target return. + +### Formulation Template +```json +{ + "sets": ["assets"], + "parameters": ["expected_returns[assets]", "covariance[assets, assets]", "min_return", "max_weight"], + "decision_variables": ["weight[assets] (continuous, >=0, <=max_weight)"], + "objective": { + "sense": "min", + "expression": "sum_{i in assets} sum_{j in assets} weight[i] * covariance[i,j] * weight[j]" + }, + "constraints": [ + "budget: sum_{i in assets} weight[i] == 1", + "return_target: sum_{i in assets} expected_returns[i] * weight[i] >= min_return" + ] +} +``` + +### Common Pitfalls +- Assuming missing covariance data is zero without verifying matrix positive definiteness. +- Using inconsistent indexing (e.g., 0-based vs 1-based) between sets and parameter dictionaries. +- Redundantly defining non-negativity via both variable domain (`NonNegativeReals`) and explicit lower bound. + +## Solving stage + +### Strategy Overview +Solve the QP using a commercial solver (e.g., Gurobi) via Pyomo's `SolverFactory`. Configure solver options for reproducibility and performance, then rigorously check solution status and validate constraint satisfaction. + +### Step 1 - Configure and Execute Solver +- Instantiate the solver: `SolverFactory("gurobi")`. +- Set options: `TimeLimit`, `MIPGap=0.0` (for continuous QP optimality), `Threads`, and `Seed` for deterministic results. +- Solve the model with `tee=False` to suppress verbose output. + +### Step 2 - Check Solver Status +- Verify `results.solver.status == SolverStatus.ok`. +- Verify `results.solver.termination_condition` is `optimal` or `feasible`. +- If status is not acceptable, output a structured JSON error payload instead of the objective value. + +### Step 3 - Extract and Validate Solution +- Extract optimal weights: `[pyo.value(model.weight[i]) for i in model.assets]`. +- Compute the achieved portfolio return from optimal weights and expected returns. +- Validate all constraints: sum of weights ≈ 1, return meets target, and weights respect bounds (within a small tolerance, e.g., 1e-6). + +### Step 4 - Output Results +- Print the optimal portfolio variance with a prefix for parsing: `print(f"RESULT:{portfolio_variance:.6f}")`. +- Optionally, output weights and validation metrics for debugging. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition +import json + +# ... model building steps ... + +solver = pyo.SolverFactory('gurobi') +solver.options['TimeLimit'] = 30 +solver.options['MIPGap'] = 0.0 +solver.options['Threads'] = 4 +solver.options['Seed'] = 42 +results = solver.solve(model, tee=False) + +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + portfolio_variance = float(pyo.value(model.obj)) + weights = [pyo.value(model.weight[i]) for i in model.assets] + portfolio_return = sum(expected_returns[i] * weights[i] for i in model.assets) + # Validate constraints here + print(f"RESULT:{portfolio_variance:.6f}") +else: + payload = { + "status": "failed", + "reason": "infeasible_or_error", + "solver_status": str(status), + "termination_condition": str(term) + } + print(f"RESULT_JSON:{json.dumps(payload)}") +``` + +### Common Pitfalls +- Trusting a solver's `ok` status without checking the termination condition. +- Not validating constraint satisfaction post-solution, especially for the return target. +- Using solver-specific options (like `threads`) that may conflict with the execution environment. + +# Workflow 2 (Pyomo with Open-Source NLP/QP Solver) + +## Modeling stage + +### Strategy Overview +Formulate the same QP structure but target open-source solvers like IPOPT or HiGHS. Emphasize handling of incomplete covariance data by constructing a reasonable, positive definite matrix, and include a pre-solve feasibility check. + +### Step 1 - Handle Incomplete Covariance Data +- If the covariance matrix is partially specified, construct a complete, positive definite matrix. +- Set known variance/covariance values. +- For unspecified diagonal entries (variances), use estimated values (e.g., average of known variances). +- For unspecified off-diagonal entries, assume a moderate correlation (e.g., 0.3) and compute covariance as `corr * sqrt(var_i * var_j)`. +- Ensure matrix symmetry and positive definiteness via eigenvalue adjustment if necessary. + +### Step 2 - Perform Feasibility Analysis +- Before building the full QP, check if the return target is achievable given the weight bounds. +- Solve a simple linear feasibility problem or compute the maximum possible return by allocating `max_weight` to the highest-return assets. + +### Step 3 - Build Pyomo Model +- Similar to Workflow 1: define sets, parameters, variables with bounds, quadratic objective, and linear constraints. +- Provide sensible variable initialization (e.g., equal weights) to aid solver convergence. + +### Formulation Template +```json +{ + "sets": ["assets"], + "parameters": ["expected_returns[assets]", "covariance[assets, assets]", "min_return", "max_weight"], + "decision_variables": ["weight[assets] (continuous, >=0, <=max_weight)"], + "objective": { + "sense": "min", + "expression": "sum_{i in assets} sum_{j in assets} weight[i] * covariance[i,j] * weight[j]" + }, + "constraints": [ + "budget: sum_{i in assets} weight[i] == 1", + "return_target: sum_{i in assets} expected_returns[i] * weight[i] >= min_return" + ] +} +``` + +### Common Pitfalls +- Creating synthetic covariance matrices without documenting assumptions or checking positive definiteness. +- Skipping the feasibility check and wasting time solving an infeasible QP. +- Using nested loops for the quadratic objective instead of Pyomo's efficient expression building. + +## Solving stage + +### Strategy Overview +Solve using an open-source solver (IPOPT for general NLP, HiGHS for QP). Configure tolerances for precision, implement robust status checking, and include post-solution validation and optional sensitivity analysis. + +### Step 1 - Select and Configure Solver +- For general QP: `SolverFactory("ipopt")`. Set options: `tol=1e-8`, `acceptable_tol=1e-6`, `max_iter=1000`, `print_level=0`. +- For convex QP: `SolverFactory("highs")`. Set options: `time_limit=30`, `presolve="on"`. +- Avoid setting options that may cause conflicts (e.g., `threads` in HiGHS if the environment is already configured). + +### Step 2 - Solve and Check Status +- Solve the model. +- Check `SolverStatus.ok` and accept termination conditions `optimal`, `locallyOptimal`, or `feasible`. + +### Step 3 - Validate and Analyze Solution +- Extract weights and compute portfolio return and variance. +- Verify all constraints are satisfied within tolerance. +- Optionally, perform a local perturbation test to confirm optimality. + +### Step 4 - Output Structured Results +- Output a JSON payload containing status, objective value, weights, portfolio return, and constraint satisfaction flags. +- Ensure all values are JSON-serializable (convert numpy types to Python floats/ints). + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition +import json + +# ... model building steps, including covariance matrix construction ... + +solver = pyo.SolverFactory('ipopt') # or 'highs' +solver.options['tol'] = 1e-8 +solver.options['max_iter'] = 1000 +solver.options['print_level'] = 0 +results = solver.solve(model, tee=False) + +status = results.solver.status +term = results.solver.termination_condition +ok_terms = {TerminationCondition.optimal, TerminationCondition.locallyOptimal, TerminationCondition.feasible} + +if status == SolverStatus.ok and term in ok_terms: + weights = [float(pyo.value(model.weight[i])) for i in model.assets] + portfolio_variance = float(pyo.value(model.obj)) + portfolio_return = sum(expected_returns[i] * weights[i] for i in model.assets) + + # Constraint validation + budget_sat = abs(sum(weights) - 1.0) < 1e-6 + return_sat = portfolio_return >= min_return - 1e-6 + bounds_sat = all(0.0 <= w <= max_weight + 1e-6 for w in weights) + + payload = { + "status": "success", + "portfolio_variance": portfolio_variance, + "portfolio_return": portfolio_return, + "weights": weights, + "constraints_satisfied": { + "budget": bool(budget_sat), + "return_target": bool(return_sat), + "weight_bounds": bool(bounds_sat) + } + } + print(f"RESULT_JSON:{json.dumps(payload)}") + print(f"RESULT:{portfolio_variance:.6f}") +else: + payload = {"status": "failed", "solver_status": str(status), "termination_condition": str(term)} + print(f"RESULT_JSON:{json.dumps(payload)}") +``` + +### Common Pitfalls +- Not converting numpy types to Python types before JSON serialization. +- Setting overly restrictive solver options that cause convergence failures. +- Accepting solutions without verifying the return target constraint is actually met. diff --git a/skills/optskills/skill_library/rectangularassignmentskill.md b/skills/optskills/skill_library/rectangularassignmentskill.md new file mode 100644 index 0000000..0087784 --- /dev/null +++ b/skills/optskills/skill_library/rectangularassignmentskill.md @@ -0,0 +1,200 @@ +--- +name: RectangularAssignmentSkill +description: | + Solves unbalanced assignment problems where one set must be fully matched to a subset of another set, minimizing total cost via binary linear programming or network flow. +--- + +# Workflow 1 (MIP Solver with Explicit Formulation) + +## Modeling stage + +### Strategy Overview +Model the problem as a Binary Integer Program (BIP) using explicit sets, binary decision variables for each potential assignment, and linear constraints enforcing one-to-one and cardinality rules. + +### Step 1 - Problem Recognition and Set Definition +- Identify the two disjoint sets: Set A (elements requiring full assignment) and Set B (elements with limited capacity). +- Define the dimensions: `m = |A|`, `n = |B|`, with `m <= n` for a feasible matching. +- Prepare the `m x n` cost matrix `c[i][j]` for assigning element `i` in A to element `j` in B. + +### Step 2 - Variable and Constraint Formulation +- Create binary decision variable `x[i][j]` for each `(i, j)` pair. +- Add constraint: `sum_{j in B} x[i][j] == 1` for each `i in A`. This ensures each element in A is assigned exactly once. +- Add constraint: `sum_{i in A} x[i][j] <= 1` for each `j in B`. This ensures each element in B is assigned at most once. +- Define the objective: Minimize `sum_{i in A} sum_{j in B} c[i][j] * x[i][j]`. + +### Formulation Template +```json +{ + "sets": ["A", "B"], + "parameters": ["c[i][j]"], + "decision_variables": ["x[i][j] ∈ {0,1}"], + "objective": { + "sense": "min", + "expression": "sum_{i in A} sum_{j in B} c[i][j] * x[i][j]" + }, + "constraints": [ + "sum_{j in B} x[i][j] == 1, for all i in A", + "sum_{i in A} x[i][j] <= 1, for all j in B" + ] +} +``` + +### Common Pitfalls +- Assuming square cost matrices; always check dimensions `m` and `n`. +- Forgetting to enforce `x[i][j]` as binary, which is essential for a matching. +- Misindexing the cost matrix when sets are defined as lists or ranges. + +## Solving stage + +### Strategy Overview +Solve the formulated BIP using a Mixed-Integer Programming (MIP) solver via a modeling library (e.g., OR-Tools, Pyomo). The focus is on a direct, declarative approach that separates model building from solver execution. + +### Step 1 - Solver and Model Setup +- Instantiate a solver object capable of handling binary variables (e.g., SCIP, CBC, HiGHS). +- Create the model container and define the index sets `A` and `B` as Python ranges or lists. +- Add binary variables `x[i][j]` to the model for all `(i, j)` pairs. + +### Step 2 - Constraint and Objective Implementation +- Use loops to add the `sum == 1` constraints for each `i` in A. +- Use loops to add the `sum <= 1` constraints for each `j` in B. +- Build the linear objective expression using the cost matrix and the variables. + +### Step 3 - Solve and Solution Extraction +- Invoke the solver with appropriate parameters (e.g., time limit, optimality gap). +- Check the solver status for `OPTIMAL` or `FEASIBLE`. +- Extract the solution by iterating over variables `x[i][j]` and collecting pairs where the solution value is > 0.5. +- Compute the achieved total cost from the objective value or by summing costs of assigned pairs. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('SCIP') +x = {} +for i in range(m): + for j in range(n): + x[i, j] = solver.BoolVar(f'x[{i},{j}]') + +# Constraints: Each i assigned exactly once +for i in range(m): + solver.Add(sum(x[i, j] for j in range(n)) == 1) +# Constraints: Each j assigned at most once +for j in range(n): + solver.Add(sum(x[i, j] for i in range(m)) <= 1) + +# Objective +objective = solver.Sum(c[i][j] * x[i, j] for i in range(m) for j in range(n)) +solver.Minimize(objective) + +# solve with status / termination checks +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + assignments = [(i, j) for i in range(m) for j in range(n) if x[i, j].solution_value() > 0.5] + total_cost = solver.Objective().Value() +``` + +### Common Pitfalls +- Not verifying the solver status before extracting the solution, leading to errors. +- Using a loose optimality gap when an exact solution is required. +- Inefficiently building large models in loops; consider vectorized construction if supported. + +# Workflow 2 (Network Flow Solver) + +## Modeling stage + +### Strategy Overview +Reformulate the assignment problem as a minimum-cost flow problem on a bipartite network. This leverages efficient network simplex algorithms and provides a natural representation of the matching constraints through flow conservation and arc capacities. + +### Step 1 - Network Structure Definition +- Define nodes: a source node is not strictly needed. Have supply nodes for set A (supply = 1), transshipment nodes for set B (supply = 0), and a sink node (supply = -|A|). +- Define arcs: from each node `i` in A to each node `j` in B, with capacity 1 and unit cost `c[i][j]`. +- Define arcs: from each node `j` in B to the sink node, with capacity 1 and zero cost. + +### Step 2 - Flow Formulation +- The flow on arc `(i, j)` represents the assignment variable `x[i][j]`. +- Flow conservation at node `i` in A ensures its unit supply is sent out. +- Flow conservation at node `j` in B ensures at most one unit can pass through to the sink, enforcing the `<= 1` constraint. +- The objective is to minimize the total cost of flow on arcs from A to B. + +### Formulation Template +```json +{ + "sets": ["A_nodes", "B_nodes"], + "parameters": ["c[i][j]", "supply[A_i] = 1", "supply[B_j] = 0", "supply[sink] = -|A|"], + "decision_variables": ["flow[i][j] ∈ {0,1}"], + "objective": { + "sense": "min", + "expression": "sum_{i in A} sum_{j in B} c[i][j] * flow[i][j]" + }, + "constraints": [ + "Flow conservation at each node.", + "Capacity on each arc from A to B: flow[i][j] <= 1", + "Capacity on each arc from B to sink: flow[j][sink] <= 1" + ] +} +``` + +### Common Pitfalls +- Incorrectly setting node supplies, leading to infeasibility. +- Forgetting to add arcs from B nodes to the sink, which are necessary to absorb the flow. +- Misinterpreting the solver's flow output, which may be fractional if not using an integer flow solver. + +## Solving stage + +### Strategy Overview +Use a dedicated minimum-cost flow solver (e.g., OR-Tools `SimpleMinCostFlow`) to find the optimal integral flow. This approach is often faster for pure network problems and requires less modeling overhead. + +### Step 1 - Graph Construction +- Instantiate a min-cost flow object. +- Add nodes for each element in A, each element in B, and one sink node. Set their supplies accordingly. +- For each `(i, j)` pair, add an arc from node `i` (A) to node `j` (B) with capacity 1 and cost `c[i][j]`. +- For each node `j` in B, add an arc from `j` to the sink node with capacity 1 and cost 0. + +### Step 2 - Solve and Flow Interpretation +- Call the solver's `Solve()` method. +- Check the return status (e.g., `OPTIMAL`). +- Iterate over all arcs, specifically those from A to B. Arcs with a flow value > 0.5 indicate an assignment. +- The total cost is provided by the solver. + +### Step 3 - Validation and Output +- Verify that the number of assignments equals `|A|`. +- Optionally, verify that no node in B receives more than one unit of flow. +- Output the list of assignments and the total cost. + +### Code Usage +```python +# build model from formulation +from ortools.graph import pywrapgraph +min_cost_flow = pywrapgraph.SimpleMinCostFlow() + +# Add nodes and set supplies +for i in range(m): # A nodes + min_cost_flow.AddNodeWithSupply(i, 1) +for j in range(n): # B nodes + min_cost_flow.AddNodeWithSupply(m + j, 0) +sink_id = m + n +min_cost_flow.AddNodeWithSupply(sink_id, -m) + +# Add arcs from A to B +for i in range(m): + for j in range(n): + min_cost_flow.AddArcWithCapacityAndUnitCost(i, m + j, 1, c[i][j]) +# Add arcs from B to sink +for j in range(n): + min_cost_flow.AddArcWithCapacityAndUnitCost(m + j, sink_id, 1, 0) + +# solve with status / termination checks +if min_cost_flow.Solve() == min_cost_flow.OPTIMAL: + total_cost = min_cost_flow.OptimalCost() + assignments = [] + for arc in range(min_cost_flow.NumArcs()): + if min_cost_flow.Tail(arc) < m and min_cost_flow.Head(arc) < (m + n): + if min_cost_flow.Flow(arc) > 0: + i = min_cost_flow.Tail(arc) + j = min_cost_flow.Head(arc) - m + assignments.append((i, j)) +``` + +### Common Pitfalls +- Assuming the solver returns integer flows; while `SimpleMinCostFlow` does for this structure, always check. +- Not mapping arc indices back to the original `(i, j)` indices correctly when extracting the solution. +- Ignoring the solver status, which could be `INFEASIBLE` or `UNBALANCED`. diff --git a/skills/optskills/skill_library/resource_constrained_assignment_problem_solver.md b/skills/optskills/skill_library/resource_constrained_assignment_problem_solver.md new file mode 100644 index 0000000..74ff11c --- /dev/null +++ b/skills/optskills/skill_library/resource_constrained_assignment_problem_solver.md @@ -0,0 +1,282 @@ +--- +name: Resource-Constrained Assignment Problem Solver +description: | + Models and solves assignment problems where tasks must be assigned to resources with capacity limits, minimizing total cost using either CP-SAT or MILP solvers. +--- + +# Workflow 1 (CP-SAT Solver) + +## Modeling stage + +### Strategy Overview +Use Google OR-Tools CP-SAT solver to model the assignment problem with binary decision variables, exactly-one constraints per task, resource capacity constraints, and a linear cost objective. This approach is well-suited for discrete optimization with binary variables and provides strong performance for moderate-sized instances. + +### Step 1 - Define Sets and Parameters +- Define the set of resources and tasks as lists or ranges. +- Prepare dictionaries for resource capacities, task-resource requirements, and task-resource costs, indexed by resource and task identifiers. + +### Step 2 - Create Binary Decision Variables +- For each (resource, task) pair, create a `NewBoolVar` indicating whether the task is assigned to that resource. +- Use a nested dictionary structure: `x[resource][task] = model.NewBoolVar(f"x_{resource}_{task}")`. + +### Step 3 - Add Each Task Assigned Exactly Once Constraint +- For each task, add a constraint that the sum of assignment variables across all resources equals 1. +- This ensures every task is allocated to exactly one resource and no task is split or left unassigned. + +### Step 4 - Add Resource Capacity Constraints +- For each resource, add a constraint that the sum of (requirement * assignment variable) over all tasks is less than or equal to the resource's capacity. +- This ensures the total load on each resource does not exceed its available capacity. + +### Step 5 - Define and Minimize Objective +- Compute total cost as the sum of (cost * assignment variable) over all resource-task pairs. +- Set the objective to minimize this total cost using `model.Minimize(total_cost)`. + +### Formulation Template +```json +{ + "sets": ["resources", "tasks"], + "parameters": ["capacity[resource]", "requirement[resource][task]", "cost[resource][task]"], + "decision_variables": ["x[resource][task] ∈ {0,1}"], + "objective": { + "sense": "min", + "expression": "sum(cost[resource][task] * x[resource][task] for resource in resources for task in tasks)" + }, + "constraints": [ + "sum(x[resource][task] for resource in resources) == 1 for each task", + "sum(requirement[resource][task] * x[resource][task] for task in tasks) <= capacity[resource] for each resource" + ] +} +``` + +### Common Pitfalls +- Forgetting to create variables for all resource-task pairs, leading to missing assignments. +- Using integer variables instead of boolean variables, which can slow down solving. +- Not ensuring requirement and cost data are indexed consistently with variable indices. + +## Solving stage + +### Strategy Overview +Configure the CP-SAT solver with appropriate parameters for reliability and performance, then solve the model. Parse the solver status and extract the assignment solution, outputting results in a structured JSON format for downstream consumption. + +### Step 1 - Configure Solver Parameters +- Create a `CpSolver` instance and set key parameters: `max_time_in_seconds` for time limit, `num_search_workers` for parallelism, `random_seed` for reproducibility, and `relative_gap_limit` for optimality tolerance. + +### Step 2 - Solve the Model +- Call `solver.Solve(model)` and capture the status code. + +### Step 3 - Handle Solver Status +- Check if status is `OPTIMAL` or `FEASIBLE` before reading results. +- For optimal/feasible solutions, extract the assignment by iterating over tasks and finding the resource where the variable value is 1. +- For infeasible cases, output a failure payload with the solver status code. + +### Step 4 - Verify Solution Feasibility (Recommended) +- After extracting the assignment, compute resource usage per resource as `sum(requirement[(r, t)] for t assigned to r)` and verify it does not exceed `capacity[r]`. +- Compute total cost manually as `sum(cost[(r, t)] for each assignment)` and confirm it matches the solver's objective value. This guards against silent data indexing errors. + +### Step 5 - Output Structured Results +- Build a JSON payload containing status, objective value, and assignment mapping. +- Print the payload with the prefix `RESULT_JSON:` for consistent parsing. + +### Code Usage +```python +import json +from ortools.sat.python import cp_model + +def solve_assignment(): + # Define data: resources, tasks, capacities, requirements, costs + resources = [...] # e.g., ["R1", "R2"] + tasks = [...] # e.g., ["T1", "T2", "T3"] + capacity = {"R1": 10, "R2": 15} + requirement = {("R1", "T1"): 3, ("R1", "T2"): 5, ...} + cost = {("R1", "T1"): 100, ("R1", "T2"): 150, ...} + + model = cp_model.CpModel() + x = {r: {t: model.NewBoolVar(f"x_{r}_{t}") for t in tasks} for r in resources} + + # Each task assigned exactly once + for t in tasks: + model.Add(sum(x[r][t] for r in resources) == 1) + + # Resource capacity constraints + for r in resources: + model.Add(sum(requirement[(r, t)] * x[r][t] for t in tasks) <= capacity[r]) + + # Minimize total cost + total_cost = sum(cost[(r, t)] * x[r][t] for r in resources for t in tasks) + model.Minimize(total_cost) + + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = 30 + solver.parameters.num_search_workers = 8 + solver.parameters.random_seed = 42 + solver.parameters.relative_gap_limit = 0.0 + + status = solver.Solve(model) + + if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + assignment = {t: next(r for r in resources if solver.Value(x[r][t]) == 1) for t in tasks} + # Optional: verify feasibility + # for r in resources: + # usage = sum(requirement[(r, t)] for t in tasks if assignment[t] == r) + # assert usage <= capacity[r], f"Capacity violated for {r}" + payload = { + "status": "optimal" if status == cp_model.OPTIMAL else "feasible", + "objective": float(solver.ObjectiveValue()), + "assignment": assignment + } + print(f"RESULT_JSON:{json.dumps(payload)}") + else: + payload = {"status": "failed", "solver_status": int(status)} + print(f"RESULT_JSON:{json.dumps(payload)}") + +if __name__ == "__main__": + solve_assignment() +``` + +### Common Pitfalls +- Not checking for `FEASIBLE` status in addition to `OPTIMAL`, missing valid solutions when optimality is not proven. +- Using `solver.Value()` on variables that were not part of the solution, causing runtime errors. +- Forgetting to convert objective value to float for JSON serialization. + +# Workflow 2 (Pyomo MILP Solver) + +## Modeling stage + +### Strategy Overview +Use Pyomo with a MILP solver (e.g., CBC or GLPK) to model the assignment problem. Define sets and parameters explicitly, create binary decision variables, enforce assignment and capacity constraints, and minimize total cost. This approach provides flexibility for extensions and access to a wide range of open-source and commercial solvers. + +### Step 1 - Define Sets and Parameters +- Create a `ConcreteModel` and define `Set` objects for tasks and resources. +- Store input data (costs, requirements, capacities) as `Param` objects indexed by the appropriate sets, using dictionaries for initialization. + +### Step 2 - Create Binary Decision Variables +- Define a binary `Var` indexed by (resource, task) pairs using `within=pyo.Binary`. +- Use a nested indexing approach: `model.x = pyo.Var(model.resources, model.tasks, within=pyo.Binary)`. + +### Step 3 - Add Each Task Assigned Exactly Once Constraint +- For each task, add a constraint that the sum of assignment variables across all resources equals 1. +- Use a constraint rule with a lambda or function that iterates over tasks. + +### Step 4 - Add Resource Capacity Constraints +- For each resource, add a constraint that the sum of (requirement * assignment variable) over all tasks is less than or equal to the resource's capacity. +- Ensure the requirement parameter is indexed consistently with the variable indices. + +### Step 5 - Define and Minimize Objective +- Compute total cost as the sum of (cost * assignment variable) over all resource-task pairs. +- Set the objective with `sense=pyo.minimize`. + +### Formulation Template +```json +{ + "sets": ["resources", "tasks"], + "parameters": ["capacity[resource]", "requirement[resource][task]", "cost[resource][task]"], + "decision_variables": ["x[resource][task] ∈ {0,1}"], + "objective": { + "sense": "min", + "expression": "sum(cost[resource][task] * x[resource][task] for resource in resources for task in tasks)" + }, + "constraints": [ + "sum(x[resource][task] for resource in resources) == 1 for each task", + "sum(requirement[resource][task] * x[resource][task] for task in tasks) <= capacity[resource] for each resource" + ] +} +``` + +### Common Pitfalls +- Using `pyo.Set(initialize=...)` with lists that contain duplicate entries, causing indexing errors. +- Forgetting to use `mutable=True` for parameters that may need to be updated after model creation. +- Defining constraints with incorrect indexing, leading to missing or extra constraints. + +## Solving stage + +### Strategy Overview +Configure a MILP solver (CBC or GLPK) with appropriate options, solve the model, and check solver status before extracting results. Parse the solution to build an assignment mapping and output structured JSON results. + +### Step 1 - Configure Solver +- Create a solver instance using `pyo.SolverFactory("cbc")` or `pyo.SolverFactory("glpk")`. +- Set solver options: time limit (`"seconds"` or `"tmlim"`), MIP gap tolerance (`"ratio"` or `"mipgap"`), and other relevant parameters. + +### Step 2 - Solve the Model +- Call `solver.solve(model, tee=False)` and capture the results object. + +### Step 3 - Check Solver Status +- Verify `results.solver.status == SolverStatus.ok` and `results.solver.termination_condition` is `TerminationCondition.optimal` or `TerminationCondition.feasible`. +- Only proceed to extract variable values if the status is acceptable. + +### Step 4 - Extract Assignment and Output Results +- Iterate over all (resource, task) pairs and check if `pyo.value(model.x[resource, task]) > 0.5` to determine assignments. +- Build a JSON payload with status, objective value, and assignment mapping. +- Print the payload with the prefix `RESULT_JSON:` for consistent parsing. + +### Code Usage +```python +import json +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +def solve_assignment(): + # Define data + resources = ["R1", "R2"] + tasks = ["T1", "T2", "T3"] + capacity = {"R1": 10, "R2": 15} + requirement = {("R1", "T1"): 3, ("R1", "T2"): 5, ("R1", "T3"): 4, + ("R2", "T1"): 2, ("R2", "T2"): 6, ("R2", "T3"): 3} + cost = {("R1", "T1"): 100, ("R1", "T2"): 150, ("R1", "T3"): 120, + ("R2", "T1"): 110, ("R2", "T2"): 140, ("R2", "T3"): 130} + + model = pyo.ConcreteModel() + model.resources = pyo.Set(initialize=resources) + model.tasks = pyo.Set(initialize=tasks) + model.capacity = pyo.Param(model.resources, initialize=capacity, mutable=True) + model.requirement = pyo.Param(model.resources, model.tasks, initialize=requirement, mutable=True) + model.cost = pyo.Param(model.resources, model.tasks, initialize=cost, mutable=True) + model.x = pyo.Var(model.resources, model.tasks, within=pyo.Binary) + + # Each task assigned exactly once + def assign_rule(m, t): + return sum(m.x[r, t] for r in m.resources) == 1 + model.assign_constraint = pyo.Constraint(model.tasks, rule=assign_rule) + + # Resource capacity constraints + def capacity_rule(m, r): + return sum(m.requirement[r, t] * m.x[r, t] for t in m.tasks) <= m.capacity[r] + model.capacity_constraint = pyo.Constraint(model.resources, rule=capacity_rule) + + # Objective: minimize total cost + def obj_rule(m): + return sum(m.cost[r, t] * m.x[r, t] for r in m.resources for t in m.tasks) + model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + + # Solve + solver = pyo.SolverFactory("cbc") + solver.options["seconds"] = 30 + solver.options["ratio"] = 0.0 + results = solver.solve(model, tee=False) + + # Check status + if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in (TerminationCondition.optimal, TerminationCondition.feasible)): + assignment = {} + for t in tasks: + for r in resources: + if pyo.value(model.x[r, t]) > 0.5: + assignment[t] = r + break + payload = { + "status": "optimal" if results.solver.termination_condition == TerminationCondition.optimal else "feasible", + "objective": float(pyo.value(model.obj)), + "assignment": assignment + } + print(f"RESULT_JSON:{json.dumps(payload)}") + else: + payload = {"status": "failed", "solver_status": str(results.solver.status)} + print(f"RESULT_JSON:{json.dumps(payload)}") + +if __name__ == "__main__": + solve_assignment() +``` + +### Common Pitfalls +- Not checking `SolverStatus.ok` before accessing termination condition, leading to attribute errors. +- Using `pyo.value()` on the objective before solving, which returns the initial expression rather than the solved value. +- Forgetting to break out of the inner loop when finding an assignment, potentially assigning the same task to multiple resources. diff --git a/skills/optskills/skill_library/rooted_tree_optimization_with_subtree_size_constraints.md b/skills/optskills/skill_library/rooted_tree_optimization_with_subtree_size_constraints.md new file mode 100644 index 0000000..14d00a9 --- /dev/null +++ b/skills/optskills/skill_library/rooted_tree_optimization_with_subtree_size_constraints.md @@ -0,0 +1,334 @@ +--- +name: Rooted Tree Optimization with Subtree Size Constraints +description: | + Models and solves minimum-cost rooted tree problems with connectivity and subtree size limits using single-commodity flow formulations, with two solver backends (CBC and GLPK). +--- + +# Workflow 1 (CBC Flow Formulation) + +## Modeling stage + +### Strategy Overview +Use a directed single-commodity flow formulation to enforce tree structure, connectivity, and subtree size constraints. Binary variables represent parent-child relationships, while continuous flow variables track subtree sizes and ensure connectivity. + +### Step 1 - Define Directed Edge Variables +- Create binary variables `y[i,j]` for each ordered pair `(i,j)` with `i != j`, indicating a directed edge from node `i` to node `j`. +- For undirected graphs, define variables for both directions to allow flexible orientation. + +### Step 2 - Enforce Tree Structure via In-Degree +- For each non-root node `j`, require exactly one incoming edge: `sum(y[i,j] for i != j) == 1`. +- For the root node, set all incoming edges to zero: `sum(y[i,root] for i != root) == 0`. + +### Step 3 - Add Flow Variables for Connectivity +- Introduce continuous flow variables `f[i,j]` on each directed edge, bounded between `0` and `N-1` (where `N` is total nodes). +- Set root outflow: `sum(f[root,j] for j != root) == N-1`. +- For each non-root node `j`, enforce flow conservation: `sum(f[i,j] for i != j) - sum(f[j,k] for k != j) == 1`. + +### Step 4 - Link Flow to Binary Variables +- Use Big-M constraints: `f[i,j] <= (N-1) * y[i,j]` for all directed edges, ensuring flow only exists on selected arcs. + +### Step 5 - Apply Subtree Size Constraints +- For each node `j`, the flow on edge `(root,j)` equals the size of the subtree rooted at `j`. +- Constrain: `f[root,j] <= max_subtree_size` for all `j != root`. + +### Step 6 - Define Objective +- Minimize total edge cost: `sum(cost[i,j] * (y[i,j] + y[j,i]) for i < j)` to avoid double-counting undirected edges. + +### Formulation Template +```json +{ + "sets": ["NODES", "ROOT"], + "parameters": ["cost[NODES,NODES]", "max_subtree_size"], + "decision_variables": [ + "y[NODES,NODES] binary", + "f[NODES,NODES] continuous >= 0" + ], + "objective": { + "sense": "min", + "expression": "sum_{i 0.5`. +- Verify exactly `N-1` edges are selected. +- Compute subtree sizes from `f[ROOT,j].value` and validate against `max_subtree_size`. +- Reconstruct the tree by following parent-child relationships from `y` variables. + +### Step 4 - Output Results +- For successful solves: `print(f"RESULT:{float(pyo.value(model.obj))}")`. +- For failures: `print(f"RESULT_JSON:{json.dumps({'status': 'failed', 'message': '...'})}")`. + +### Code Usage +```python +import pyomo.environ as pyo +import json + +def build_and_solve(N, cost, root=0, max_subtree=4): + model = pyo.ConcreteModel() + model.NODES = pyo.RangeSet(0, N-1) + model.ROOT = root + + # Decision variables + model.y = pyo.Var(model.NODES, model.NODES, within=pyo.Binary) + model.f = pyo.Var(model.NODES, model.NODES, within=pyo.NonNegativeReals, bounds=(0, N-1)) + + # Constraints + def in_degree_rule(m, j): + if j == root: + return sum(m.y[i,j] for i in m.NODES if i != j) == 0 + return sum(m.y[i,j] for i in m.NODES if i != j) == 1 + model.in_degree = pyo.Constraint(model.NODES, rule=in_degree_rule) + + def root_flow_rule(m): + return sum(m.f[root,j] for j in m.NODES if j != root) == N-1 + model.root_flow = pyo.Constraint(rule=root_flow_rule) + + def flow_balance_rule(m, j): + if j == root: + return pyo.Constraint.Skip + inflow = sum(m.f[i,j] for i in m.NODES if i != j) + outflow = sum(m.f[j,k] for k in m.NODES if k != j) + return inflow - outflow == 1 + model.flow_balance = pyo.Constraint(model.NODES, rule=flow_balance_rule) + + def big_m_rule(m, i, j): + return m.f[i,j] <= (N-1) * m.y[i,j] + model.big_m = pyo.Constraint(model.NODES, model.NODES, rule=big_m_rule) + + def subtree_limit_rule(m, j): + if j == root: + return pyo.Constraint.Skip + return m.f[root,j] <= max_subtree + model.subtree_limit = pyo.Constraint(model.NODES, rule=subtree_limit_rule) + + # Objective + def obj_rule(m): + total = 0 + for i in m.NODES: + for j in m.NODES: + if i < j: + total += cost[i][j] * (m.y[i,j] + m.y[j,i]) + return total + model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + + # Solve + solver = pyo.SolverFactory('cbc') + solver.options["seconds"] = 120 + solver.options["ratio"] = 0.0 + result = solver.solve(model, tee=False) + + # Check status + if result.solver.status != pyo.SolverStatus.ok or \ + (result.solver.termination_condition != pyo.TerminationCondition.optimal and \ + result.solver.termination_condition != pyo.TerminationCondition.feasible): + print(f"RESULT_JSON:{json.dumps({'status': 'failed', 'message': 'No feasible solution'})}") + return + + # Extract edges + edges = [(i,j) for i in model.NODES for j in model.NODES if i != j and pyo.value(model.y[i,j]) > 0.5] + # Verification + assert len(edges) == N-1, f"Expected {N-1} edges, got {len(edges)}" + for j in model.NODES: + if j != root: + size = pyo.value(model.f[root,j]) + assert size <= max_subtree, f"Subtree at {j} has size {size} > {max_subtree}" + + print(f"RESULT:{float(pyo.value(model.obj))}") + return edges +``` + +### Common Pitfalls +- Not checking termination condition for feasible (non-optimal) solutions, which may still be acceptable. +- Using `solution_value()` instead of `.value` in Pyomo, causing attribute errors. +- Forgetting to convert cost matrix to a list of lists or dictionary before passing to the model. + +# Workflow 2 (GLPK Flow Formulation) + +## Modeling stage + +### Strategy Overview +Implement the same single-commodity flow formulation using GLPK solver with direct solver options. This workflow emphasizes tighter Big-M tuning and manual post-processing verification. + +### Step 1 - Define Directed Binary Variables +- Create binary variables `x[i,j]` for all ordered pairs `(i,j)` with `i != j`, representing directed edges away from the root. + +### Step 2 - Enforce Rooted Arborescence +- For each non-root node `j`: `sum(x[i,j] for i != j) == 1`. +- For root node: `sum(x[i,root] for i != root) == 0`. + +### Step 3 - Add Continuous Flow Variables +- Define flow variables `f[i,j]` with bounds `[0, N-1]`. +- Root sends `N-1` units: `sum(f[root,j] for j != root) == N-1`. +- Each non-root node consumes 1 unit: `sum(f[i,j] for i != j) - sum(f[j,k] for k != j) == 1`. + +### Step 4 - Couple Flow with Binary Variables +- Use tight Big-M: `f[i,j] <= (N-1) * x[i,j]` for all directed edges. + +### Step 5 - Subtree Size Constraints +- For each node `j != root`: `f[root,j] <= max_subtree_size`. + +### Step 6 - Objective Function +- Minimize `sum(cost[i,j] * (x[i,j] + x[j,i]) for i < j)`. + +### Formulation Template +```json +{ + "sets": ["NODES", "ROOT"], + "parameters": ["cost[NODES,NODES]", "max_subtree_size"], + "decision_variables": [ + "x[NODES,NODES] binary", + "f[NODES,NODES] continuous >= 0" + ], + "objective": { + "sense": "min", + "expression": "sum_{i 0.5`. +- Verify exactly `N-1` edges form a connected tree. +- Validate each root-child subtree size using `f[root,j].value <= max_subtree_size`. + +### Step 4 - Output with Verification +- Print objective value for successful solves. +- Optionally print selected edges and subtree sizes for debugging. + +### Code Usage +```python +import pyomo.environ as pyo +import json + +def solve_with_glpk(N, cost, root=0, max_subtree=4): + model = pyo.ConcreteModel() + model.NODES = pyo.RangeSet(0, N-1) + model.ROOT = root + + # Variables + model.x = pyo.Var(model.NODES, model.NODES, within=pyo.Binary) + model.f = pyo.Var(model.NODES, model.NODES, within=pyo.NonNegativeReals, bounds=(0, N-1)) + + # Constraints + def in_degree_rule(m, j): + if j == root: + return sum(m.x[i,j] for i in m.NODES if i != j) == 0 + return sum(m.x[i,j] for i in m.NODES if i != j) == 1 + model.in_degree = pyo.Constraint(model.NODES, rule=in_degree_rule) + + def root_flow_rule(m): + return sum(m.f[root,j] for j in m.NODES if j != root) == N-1 + model.root_flow = pyo.Constraint(rule=root_flow_rule) + + def flow_balance_rule(m, j): + if j == root: + return pyo.Constraint.Skip + inflow = sum(m.f[i,j] for i in m.NODES if i != j) + outflow = sum(m.f[j,k] for k in m.NODES if k != j) + return inflow - outflow == 1 + model.flow_balance = pyo.Constraint(model.NODES, rule=flow_balance_rule) + + def big_m_rule(m, i, j): + return m.f[i,j] <= (N-1) * m.x[i,j] + model.big_m = pyo.Constraint(model.NODES, model.NODES, rule=big_m_rule) + + def subtree_limit_rule(m, j): + if j == root: + return pyo.Constraint.Skip + return m.f[root,j] <= max_subtree + model.subtree_limit = pyo.Constraint(model.NODES, rule=subtree_limit_rule) + + # Objective + def obj_rule(m): + total = 0 + for i in m.NODES: + for j in m.NODES: + if i < j: + total += cost[i][j] * (m.x[i,j] + m.x[j,i]) + return total + model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + + # Solve with GLPK + solver = pyo.SolverFactory('glpk') + solver.options["tmlim"] = 120 + solver.options["mipgap"] = 0.0 + solver.options["presolve"] = 1 + result = solver.solve(model, tee=False) + + # Status check + if result.solver.status != pyo.SolverStatus.ok or \ + (result.solver.termination_condition != pyo.TerminationCondition.optimal and \ + result.solver.termination_condition != pyo.TerminationCondition.feasible): + print(f"RESULT_JSON:{json.dumps({'status': 'failed', 'message': 'No feasible solution'})}") + return + + # Extract and verify + edges = [(i,j) for i in model.NODES for j in model.NODES if i != j and pyo.value(model.x[i,j]) > 0.5] + assert len(edges) == N-1, f"Expected {N-1} edges, got {len(edges)}" + + # Verify subtree sizes + for j in model.NODES: + if j != root: + size = pyo.value(model.f[root,j]) + assert size <= max_subtree, f"Subtree at {j} has size {size} > {max_subtree}" + + print(f"RESULT:{float(pyo.value(model.obj))}") + return edges +``` + +### Common Pitfalls +- GLPK may return `feasible` without `optimal` for some instances; always accept feasible solutions if optimal is not required. +- Presolve may eliminate variables; verify that all expected constraints are still present after presolve. +- GLPK's `mipgap` option expects a fraction (e.g., 0.05 for 5% gap), not a percentage. diff --git a/skills/optskills/skill_library/rural_postman_problem_rpp_solver.md b/skills/optskills/skill_library/rural_postman_problem_rpp_solver.md new file mode 100644 index 0000000..9368503 --- /dev/null +++ b/skills/optskills/skill_library/rural_postman_problem_rpp_solver.md @@ -0,0 +1,298 @@ +--- +name: Rural Postman Problem (RPP) Solver +description: | + Model and solve the Rural Postman Problem (RPP) to find a minimum-cost closed tour covering a set of required edges, using either a direct MIP formulation with connectivity cuts or a matching-based reduction to an Eulerian graph. + +--- +# Workflow 1 (Direct MIP with Cut-Set Constraints) + +## Modeling stage + +### Strategy Overview +Model the RPP as a Mixed-Integer Program (MIP) using integer edge traversal variables. Enforce mandatory edge coverage, Eulerian degree conditions, and connectivity via explicit cut-set constraints to guarantee a single closed tour. + +### Step 1 - Define Core Variables and Objective +- Define non-negative integer variables `x[edge]` representing the number of times each undirected edge is traversed. +- Formulate the objective to minimize total traversal cost: `min sum(cost[edge] * x[edge] for edge in all_edges)`. + +### Step 2 - Enforce Mandatory Edge Coverage +- For each edge in the required set `R`, add a constraint `x[edge] >= 1`. + +### Step 3 - Impose Eulerian Degree Conditions +- For each node `i`, introduce an auxiliary non-negative integer variable `k[i]`. +- Add a constraint that the sum of `x` for all edges incident to node `i` equals `2 * k[i]`. This ensures even degree at each node, a necessary condition for an Eulerian circuit. + +### Step 4 - Ensure Connectivity via Subtour Elimination +- To prevent disconnected cycles, add cut-set constraints for every proper subset `S` of nodes that does not contain the designated depot node. +- For each such subset `S`, add the constraint: `sum(x[edge] for edge in the cut(S)) >= 2`. This forces at least two connections between `S` and its complement, ensuring a single connected tour. +- **Prerequisite Check**: Before solving, verify the depot node is incident to at least one required edge or is connected to the required subgraph via the underlying network. If the depot is isolated, the problem may be infeasible or produce a disconnected solution. +- **Scalability Warning**: Pre-generating all cut-set constraints for large graphs (|NODES| > [SMALL_INSTANCE_THRESHOLD]) leads to an exponential number of constraints; use lazy constraint callbacks for scalability. + +### Formulation Template +```json +{ + "sets": [ + "NODES", + "EDGES (undirected pairs from NODES)", + "REQUIRED_EDGES (subset of EDGES)" + ], + "parameters": [ + "cost[edge in EDGES] (non-negative)", + "depot (node in NODES)" + ], + "decision_variables": [ + "x[edge in EDGES] (non-negative integer)", + "k[node in NODES] (non-negative integer)" + ], + "objective": { + "sense": "min", + "expression": "sum(cost[edge] * x[edge] for edge in EDGES)" + }, + "constraints": [ + "coverage[edge in REQUIRED_EDGES]: x[edge] >= 1", + "even_degree[node in NODES]: sum(x[(i,j)] for (i,j) in EDGES if i==node or j==node) == 2 * k[node]", + "connectivity[S in proper_subsets(NODES \\ {depot})]: sum(x[(i,j)] for (i,j) in EDGES if (i in S) != (j in S)) >= 2" + ] +} +``` + +### Common Pitfalls +- Using binary variables (`x[edge] in {0,1}`) incorrectly limits the model to traversing each edge at most once, which may render the problem infeasible if required edges are not connected. +- Forgetting to exclude the depot from the subsets `S` in the cut-set constraints, which can lead to redundant or incorrect constraints. +- **Depot Isolation**: The standard cut-set constraints only ensure connectivity among non-depot nodes. A solution may satisfy all constraints yet leave the depot disconnected (degree 0). Always verify depot connectivity post-solution. +- **Suboptimality Risk**: For small instances where the required subgraph is connected, the direct MIP formulation may be computationally heavy and prone to finding suboptimal solutions. Consider using Workflow 2 (Matching-Based Reduction) in such cases. + +## Solving stage + +### Strategy Overview +Solve the MIP using a standard solver like SCIP or CBC. After obtaining a solution, verify its feasibility and construct the explicit Eulerian tour. + +### Step 1 - Configure and Solve the MIP +- Instantiate the solver (e.g., `pywraplp.Solver.CreateSolver("SCIP")`). +- Build the model using the formulation, adding all pre-computed cut-set constraints for small instances. +- Set solver parameters such as time limit, relative MIP gap, and number of threads. +- Call `solver.Solve()` and check the status for optimality or feasibility. + +### Step 2 - Verify Solution Correctness +- Extract the solution values for `x[edge]`. +- Programmatically verify all constraints: check `x[edge] >= 1` for required edges, verify even degree at each node, and confirm the graph defined by edges with `x[edge] > 0` is connected (e.g., using a BFS/DFS). +- **Critical Depot Check**: Explicitly verify the depot node has positive degree in the solution (`sum(x[edge] for edges incident to depot) > 0`). If depot degree is zero, the solution is invalid (disconnected depot). This check must be performed even if all other constraints are satisfied. +- Log any constraint violations for debugging. + +### Step 3 - Construct the Eulerian Tour +- Build a multigraph representation where each edge `(i,j)` appears `x[edge].solution_value()` times. +- Apply Hierholzer's algorithm to find an Eulerian circuit starting and ending at the depot. +- Output the sequence of nodes representing the tour. + +### Code Usage +```python +import math +from ortools.linear_solver import pywraplp + +# 1. Initialize solver +solver = pywraplp.Solver.CreateSolver('SCIP') +if not solver: + raise Exception('Solver not available.') + +# 2. Define variables (example for a small graph) +x = {} +for i in NODES: + for j in NODES: + if i < j: + x[(i, j)] = solver.IntVar(0, solver.infinity(), f'x_{i}_{j}') + +# 3. Set objective +objective = solver.Objective() +for (i, j), var in x.items(): + objective.SetCoefficient(var, COST_MATRIX[i][j]) +objective.SetMinimization() + +# 4. Add mandatory coverage constraints +for (i, j) in REQUIRED_EDGES: + solver.Add(x[(min(i, j), max(i, j))] >= 1) + +# 5. Add even-degree constraints +k_vars = {} +for i in NODES: + k_vars[i] = solver.IntVar(0, solver.infinity(), f'k_{i}') + degree_expr = sum(x.get((min(i, adj), max(i, adj)), 0) for adj in NODES if adj != i) + solver.Add(degree_expr == 2 * k_vars[i]) + +# 6. Add connectivity constraints (for small N, pre-generate all cuts) +# ... (implementation for generating proper subsets S) +# for S in all_proper_subsets: +# cut_expr = 0 +# for (i, j) in x: +# if (i in S) != (j in S): +# cut_expr += x[(i, j)] +# solver.Add(cut_expr >= 2) + +# 7. Solve +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + print(f'Objective value = {objective.Value()}') + # Extract solution and build tour + solution_x = {edge: var.solution_value() for edge, var in x.items()} + # --- Critical Depot Connectivity Check --- + depot_degree = sum(solution_x.get((min(DEPOT, adj), max(DEPOT, adj)), 0) for adj in NODES if adj != DEPOT) + if depot_degree == 0: + raise Exception('Solution invalid: depot is disconnected (degree 0).') + # ... verification and tour construction code +else: + print('No solution found.') +``` + +### Common Pitfalls +- Not checking the solver status for `FEASIBLE` in addition to `OPTIMAL`, which may miss valid but suboptimal solutions. +- Attempting to generate all cut-set constraints for more than ~[SMALL_INSTANCE_THRESHOLD] nodes, causing memory issues; implement a callback or use an alternative formulation. +- Assuming the solver's solution is automatically connected; always run a connectivity check post-solve. +- **Missing Depot Verification**: Accepting a solution where the depot has degree 0 violates the closed tour requirement. Always explicitly check depot degree. +- **Suboptimality Risk**: The MIP solver may not find the global optimum for this problem structure. If the required subgraph is connected, consider verifying the solution cost against the matching-based reduction (Workflow 2) for optimality assurance. + +# Workflow 2 (Matching-Based Reduction) + +## Modeling stage + +### Strategy Overview +Reduce the RPP to a Minimum-Weight Perfect Matching (MWPM) problem. First construct an Eulerian multigraph by adding shortest paths between odd-degree nodes in the required subgraph, then find an Eulerian circuit. The core MIP solves the matching problem. + +### Step 1 - Analyze the Required Subgraph +- Construct a graph `G_R` containing only the required edges. +- Calculate the degree of each node within `G_R`. +- Identify the set `ODD_NODES` of nodes with odd degree in `G_R`. By the Handshaking Lemma, this set has even cardinality. +- **Depot Connectivity Pre-check**: If the depot is not incident to any required edge, it will have degree 0 in `G_R` (even) and will not be in `ODD_NODES`. The standard matching formulation will not connect the depot, potentially leading to a disconnected final tour. In such cases, either treat the depot as an odd node artificially (by adding a dummy required edge of zero cost to a neighbor) or use Workflow 1. + +### Step 2 - Compute Shortest Path Distances +- Compute all-pairs shortest path distances `dist[i][j]` on the original complete graph (or the underlying network) using a suitable algorithm (e.g., Floyd-Warshall). +- **Symmetry Verification**: Ensure the distance matrix is symmetric (`dist[i][j] == dist[j][i]`). If input data is asymmetric, correct it (e.g., by taking the minimum or enforcing symmetry) to guarantee a valid matching cost matrix. + +### Step 3 - Model Minimum-Weight Perfect Matching +- Define binary variables `z[i,j]` for each pair `(i,j)` in `ODD_NODES` where `i < j`, indicating that a shortest path between `i` and `j` is added to the tour. +- Add constraints so that each odd node is matched exactly once: for each node `i` in `ODD_NODES`, `sum(z[i,j] for j != i) == 1`. +- The objective is to minimize the total cost of added paths: `min sum(dist[i][j] * z[i,j] for i,j in ODD_NODES, i 0.5: + # Retrieve shortest path edges (precomputed) + path_edges = get_shortest_path_edges(i, j) + multigraph.add_edges_from(path_edges) + # --- Critical Depot Connectivity Check --- + if DEPOT not in multigraph or multigraph.degree(DEPOT) == 0: + raise Exception('Solution invalid: depot is disconnected. Consider using Direct MIP workflow.') + # --- Graph Connectivity Check --- + if not nx.is_connected(multigraph.to_undirected()): + raise Exception('Multigraph is disconnected. Matching solution invalid.') + # Generate Eulerian circuit + if nx.is_eulerian(multigraph): + tour = list(nx.eulerian_circuit(multigraph, source=DEPOT)) + total_cost = sum(COST_MATRIX[i][j] for i, j in multigraph.edges()) + print(f'Tour: {tour}') + print(f'Total cost: {total_cost}') diff --git a/skills/optskills/skill_library/schedule_separation_with_makespan_minimization.md b/skills/optskills/skill_library/schedule_separation_with_makespan_minimization.md new file mode 100644 index 0000000..ede997c --- /dev/null +++ b/skills/optskills/skill_library/schedule_separation_with_makespan_minimization.md @@ -0,0 +1,198 @@ +--- +name: Schedule Separation with Makespan Minimization +description: | + Model and solve scheduling problems with pairwise separation constraints and a makespan objective using integer programming techniques. + +--- +# Workflow 1 (CP-SAT with Absolute Value Constraints) + +## Modeling stage + +### Strategy Overview +This workflow uses Google's OR-Tools CP-SAT solver, which natively supports integer variables and constraints. It directly models absolute value constraints for pairwise separations, avoiding the need for big-M linearization and binary variables, leading to a concise and efficient formulation. + +### Step 1 - Define Core Variables +- Define an integer decision variable for the start time of each entity (e.g., `t[i]`). Use `model.NewIntVar(lb, ub, name)` with appropriate lower and upper bounds. +- Define an auxiliary integer variable for the makespan (`makespan`). This variable will represent the maximum completion time. + +### Step 2 - Link Makespan to Schedule +- For each entity `i`, add a constraint: `makespan >= t[i]`. This ensures the makespan variable correctly captures the latest start time. + +### Step 3 - Enforce Pairwise Separations +- For each required separation `(u, v, d)`, enforce `|t[u] - t[v]| >= d`. +- Use the CP-SAT pattern `model.AddAbsEquality(abs_diff, diff)` to handle the absolute value, then constrain `abs_diff >= d`. +- Define `diff` as `t[u] - t[v]` and `abs_diff` as a non-negative integer variable. + +### Step 4 - Set Objective +- Set the objective to minimize the makespan variable: `model.Minimize(makespan)`. + +### Formulation Template +```json +{ + "sets": [ + {"name": "entities", "description": "Set of all entities to be scheduled."}, + {"name": "dependencies", "description": "Set of pairwise separation requirements."} + ], + "parameters": [ + {"name": "separation_required", "set": "dependencies", "description": "Minimum required separation distance d for pair (u,v)."} + ], + "decision_variables": [ + {"name": "start_time", "set": "entities", "type": "integer", "bounds": "[lower_bound, upper_bound]"}, + {"name": "makespan", "type": "integer", "bounds": "[lower_bound, upper_bound]"} + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + "makespan >= start_time[i] for all i in entities", + "|start_time[u] - start_time[v]| >= separation_required[k] for all k in dependencies" + ] +} +``` + +### Common Pitfalls +- Forgetting to define an upper bound for the `makespan` and `start_time` variables, which is required by CP-SAT. +- Using an insufficiently large range when creating the `abs_diff` variable, which can lead to constraint violations being missed. +- Not verifying the solution against the original separation constraints, especially when bounds are tight. + +## Solving stage + +### Strategy Overview +Configure the CP-SAT solver for a balance of speed and proof of optimality. Use parallel search and a time limit, then carefully extract and verify the solution. + +### Step 1 - Configure Solver +- Instantiate the solver: `solver = cp_model.CpSolver()`. +- Set practical parameters: `solver.parameters.max_time_in_seconds`, `solver.parameters.num_search_workers`, and `solver.parameters.random_seed` for reproducibility. +- For optimality, set `solver.parameters.relative_gap_limit = 0.0`. + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve(model)`. +- Check the status against `cp_model.OPTIMAL`, `cp_model.FEASIBLE`, or `cp_model.INFEASIBLE`. Handle each case appropriately (e.g., extract solution only if feasible). + +### Step 3 - Extract and Verify Solution +- If a solution exists, retrieve variable values using `solver.Value(var)`. +- Programmatically verify all separation constraints by computing `abs(start_time[u] - start_time[v])` and comparing against the required distance `d`. This catches potential modeling or solver issues. + +### Step 4 - Report Results +- Output the solver status, the optimal or best-found makespan, and the schedule of start times. +- For programmatic use, structure the output (e.g., as a JSON object containing status, objective value, and variable assignments). + +### Code Usage +```python +# build model from formulation +model = cp_model.CpModel() +# ... define variables, constraints, objective +# solve with status / termination checks +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30 +solver.parameters.num_search_workers = 8 +status = solver.Solve(model) +if status in [cp_model.OPTIMAL, cp_model.FEASIBLE]: + makespan_value = solver.Value(makespan) + schedule = {i: solver.Value(t[i]) for i in entities} + # ... verification and output +else: + # handle infeasible or unknown status +``` + +### Common Pitfalls +- Assuming `FEASIBLE` status means optimal. Check for `OPTIMAL` if a guaranteed optimum is required. +- Not using `solver.Value()` on variables before the solver is reset or the model goes out of scope. +- Ignoring the verification step, which is crucial for validating that the solver's solution satisfies all absolute difference constraints. + +# Workflow 2 (MILP with Big-M Disjunctive Constraints) + +## Modeling stage + +### Strategy Overview +This workflow formulates the problem as a Mixed-Integer Linear Program (MILP), suitable for solvers like Gurobi, CPLEX, or HiGHS. It uses a big-M method to linearize the disjunctive nature of pairwise separation constraints (`|a-b|>=d`), providing explicit control over the linear relaxation. + +### Step 1 - Define Variables and Makespan +- Define non-negative integer variables for entity start times. +- Define a makespan variable and add constraints `makespan >= start_time[i]` for all `i`. + +### Step 2 - Linearize Pairwise Separation Constraints +- For each separation requirement `(u, v, d)`, the constraint `|t_u - t_v| >= d` is equivalent to `t_u - t_v >= d OR t_v - t_u >= d`. +- Introduce a binary variable `y_k` for each dependency to model this disjunction. +- Use a big-M constant to formulate the linear constraints: `t_u - t_v >= d - M * (1 - y_k)` and `t_v - t_u >= d - M * y_k`. + +### Step 3 - Set Objective +- Set the objective to minimize the makespan variable. + +### Formulation Template +```json +{ + "sets": [ + {"name": "entities", "description": "Set of all entities to be scheduled."}, + {"name": "dependencies", "description": "Set of pairwise separation requirements."} + ], + "parameters": [ + {"name": "separation_required", "set": "dependencies", "description": "Minimum required separation distance d for pair (u,v)."}, + {"name": "big_M", "description": "A sufficiently large constant to deactivate constraints."} + ], + "decision_variables": [ + {"name": "start_time", "set": "entities", "type": "integer", "bounds": "[0, None]"}, + {"name": "makespan", "type": "integer", "bounds": "[0, None]"}, + {"name": "disjunction_selector", "set": "dependencies", "type": "binary"} + ], + "objective": { + "sense": "min", + "expression": "makespan" + }, + "constraints": [ + "makespan >= start_time[i] for all i in entities", + "start_time[u] - start_time[v] >= separation_required[k] - big_M * (1 - disjunction_selector[k]) for all k in dependencies", + "start_time[v] - start_time[u] >= separation_required[k] - big_M * disjunction_selector[k] for all k in dependencies" + ] +} +``` + +### Common Pitfalls +- Choosing a `big_M` value that is too small, which can cut off valid solutions. +- Choosing a `big_M` value that is excessively large, which can lead to numerical instability and slow convergence. +- Not declaring start time variables as integer, leading to a relaxed LP that may not satisfy integral separation requirements. + +## Solving stage + +### Strategy Overview +Configure a MILP solver with emphasis on finding and proving optimality. Use techniques like solving with an objective bound to verify optimality. + +### Step 1 - Configure and Solve +- Instantiate the solver (e.g., `solver = pyo.SolverFactory('gurobi')`). +- Set key parameters: `time_limit`, `mipgap` (to 0.0 for optimality), `threads`, and `seed` for reproducibility. +- Solve the model and capture the termination condition. + +### Step 2 - Check Solution Status +- Check the solver status (e.g., `ok`, `optimal`, `feasible`, `infeasible`) and the model termination condition (e.g., `optimal`, `maxTimeLimit`, `infeasible`). +- Only extract variable values if a feasible solution is reported. + +### Step 3 - Verify Optimality via Bound Testing +- To confirm a candidate makespan `M*` is optimal, add a constraint `makespan <= M* - 1` and resolve. +- If the modified model is infeasible, then `M*` is optimal. This provides a simple proof of optimality. + +### Step 4 - Extract and Verify Solution +- Extract the makespan and start time values. +- Perform post-solution verification by checking all separation constraints directly to ensure the big-M formulation behaved correctly. + +### Code Usage +```python +# build model from formulation +model = pyo.ConcreteModel() +# ... define sets, parameters, variables, constraints, objective +# solve with status / termination checks +solver = pyo.SolverFactory('appsi_highs') +solver.options['time_limit'] = 30 +solver.options['threads'] = 4 +results = solver.solve(model, tee=False) +if results.solver.termination_condition == pyo.TerminationCondition.optimal: + makespan_value = pyo.value(model.makespan) + schedule = {i: pyo.value(model.start_time[i]) for i in model.entities} + # ... verification and output +# ... optimality bound test +``` + +### Common Pitfalls +- Confusing solver status (did it run?) with model termination condition (what did it find?). +- Not setting an appropriate `mipgap` or `time_limit`, leading to excessively long runs or accepting suboptimal solutions. +- Forgetting to remove the optimality-testing constraint (`makespan <= M* - 1`) before proceeding or re-solving the original model. diff --git a/skills/optskills/skill_library/schedulewithprecedenceanddeviations.md b/skills/optskills/skill_library/schedulewithprecedenceanddeviations.md new file mode 100644 index 0000000..4b3c4fc --- /dev/null +++ b/skills/optskills/skill_library/schedulewithprecedenceanddeviations.md @@ -0,0 +1,248 @@ +--- +name: ScheduleWithPrecedenceAndDeviations +description: | + Model and solve scheduling problems with precedence decisions, time windows, and piecewise-linear deviation penalties using MILP formulations and modern solvers. +--- + +# Workflow 1 (CP-SAT with Explicit Big-M) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program (MILP) suitable for constraint programming/satisfiability solvers like OR-Tools CP-SAT. Use integer time variables, binary precedence variables, and linear constraints with a conservative Big-M to enforce disjunctive separation. + +### Step 1 - Define Core Variables +- Create an integer variable `t[i]` for each entity `i`, bounded by its earliest and latest time. +- Create a binary variable `x[i,j]` for each unordered pair `(i, j)` where `i < j`, representing that `i` precedes `j`. +- Create non-negative continuous or integer variables `e[i]` and `l[i]` to capture early and late deviations from a target time. + +### Step 2 - Enforce Time Windows and Deviation Definitions +- Add constraints `earliest[i] <= t[i] <= latest[i]` for each entity `i`. +- Define early deviation: `e[i] >= target[i] - t[i]`. Since `e[i]` is non-negative, it captures positive earliness. +- Define late deviation: `l[i] >= t[i] - target[i]`. Since `l[i]` is non-negative, it captures positive lateness. + +### Step 3 - Model Precedence and Separation +- For each unordered pair `(i, j)`, enforce mutual exclusivity: `x[i,j] + x[j,i] = 1`. +- For each ordered pair, add a Big-M constraint to enforce separation if that order is chosen: `t[j] >= t[i] + sep[i,j] - M * (1 - x[i,j])`. Choose `M` as a sufficiently large constant (e.g., `max_time_range + max_separation`). + +### Step 4 - Formulate Objective +- Minimize the weighted sum of deviations: `min sum( early_penalty[i] * e[i] + late_penalty[i] * l[i] )`. + +### Formulation Template +```json +{ + "sets": [ + "Entities", + "UnorderedPairs" + ], + "parameters": [ + {"name": "earliest", "domain": "Entities", "type": "float"}, + {"name": "latest", "domain": "Entities", "type": "float"}, + {"name": "target", "domain": "Entities", "type": "float"}, + {"name": "early_penalty", "domain": "Entities", "type": "float"}, + {"name": "late_penalty", "domain": "Entities", "type": "float"}, + {"name": "sep", "domain": "UnorderedPairs", "type": "float"}, + {"name": "M", "type": "float"} + ], + "decision_variables": [ + {"name": "t", "domain": "Entities", "type": "integer", "bounds": "[earliest[i], latest[i]]"}, + {"name": "x", "domain": "UnorderedPairs", "type": "binary"}, + {"name": "e", "domain": "Entities", "type": "continuous", "bounds": "[0, INF]"}, + {"name": "l", "domain": "Entities", "type": "continuous", "bounds": "[0, INF]"} + ], + "objective": { + "sense": "min", + "expression": "sum( early_penalty[i] * e[i] + late_penalty[i] * l[i] for i in Entities )" + }, + "constraints": [ + {"name": "deviation_early", "formula": "e[i] >= target[i] - t[i]", "domain": "Entities"}, + {"name": "deviation_late", "formula": "l[i] >= t[i] - target[i]", "domain": "Entities"}, + {"name": "mutual_exclusion", "formula": "x[i,j] + x[j,i] == 1", "domain": "UnorderedPairs"}, + {"name": "separation_if_precedes", "formula": "t[j] >= t[i] + sep[i,j] - M * (1 - x[i,j])", "domain": "UnorderedPairs"} + ] +} +``` + +### Common Pitfalls +- Using an excessively large Big-M value, which weakens the linear relaxation and slows solving. +- Forgetting to enforce mutual exclusivity of precedence variables, leading to infeasible or incorrect orders. +- Modeling deviations with `abs(target[i] - t[i])` directly, which is non-linear; always use separate non-negative variables. + +## Solving stage + +### Strategy Overview +Solve the MILP using OR-Tools CP-SAT solver. Configure search parameters for performance and reproducibility, extract and verify the solution, and handle solver statuses appropriately. + +### Step 1 - Configure Solver and Build Model +- Instantiate a CP-SAT model. +- Create variables using `NewIntVar` for times (within integer bounds) and `NewBoolVar` for binaries. +- Add all constraints using `model.Add()` with linear expressions. + +### Step 2 - Set Solver Parameters and Solve +- Configure the solver: set a time limit, number of parallel workers, random seed, and optionally a relative gap tolerance. +- Call the solver and capture the status. + +### Step 3 - Extract and Verify Solution +- Check if the status is `OPTIMAL` or `FEASIBLE`. +- Extract variable values using `solver.Value(var)`. +- Programmatically verify all hard constraints (time windows, separation) and recalculate the objective from primal values to ensure consistency. + +### Step 4 - Output Structured Results +- Return a dictionary or JSON containing the status, objective value, schedule times, deviations, and the derived precedence order. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# Build model +model = cp_model.CpModel() +# ... create variables and add constraints as per modeling stage ... + +# Solve +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 +status = solver.Solve(model) + +# Check status and extract solution +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + solution = {} + solution['status'] = 'OPTIMAL' if status == cp_model.OPTIMAL else 'FEASIBLE' + solution['objective'] = solver.ObjectiveValue() + solution['times'] = {i: solver.Value(t_var[i]) for i in entities} + # ... extract other variables ... + # Verification logic here + return solution +else: + return {'status': 'INFEASIBLE_OR_UNBOUNDED', 'objective': None} +``` + +### Common Pitfalls +- Not checking for both `OPTIMAL` and `FEASIBLE` statuses, potentially discarding good feasible solutions. +- Assuming variable values exist when the status is not feasible, leading to runtime errors. +- Omitting verification, which can mask modeling errors in constraint definitions. + +# Workflow 2 (Pyomo with Tight Big-M and MILP Solver) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a MILP using an algebraic modeling language (Pyomo). Use continuous time variables, a tight, pair-specific Big-M derived from time windows, and separate deviation constraints. This formulation is solved by general-purpose MILP solvers (e.g., Gurobi, HiGHS). + +### Step 1 - Define Variables and Bounds +- Create a continuous variable `t[i]` for each entity `i`. Set its bounds directly to `[earliest[i], latest[i]]`. +- Create non-negative continuous variables `e[i]` and `l[i]` for deviations. +- Create a binary variable `y[i,j]` for each unordered pair `(i, j)` where `i < j`, indicating `i` precedes `j`. + +### Step 2 - Model Deviation Constraints +- For each entity `i`, add constraints: `t[i] + e[i] >= target[i]` and `t[i] - l[i] <= target[i]`. This ensures `e[i]` and `l[i]` capture the positive deviations. + +### Step 3 - Enforce Disjunctive Separation with Tight Big-M +- For each unordered pair `(i, j)` with `i < j`, enforce mutual exclusivity: `y[i,j] + y[j,i] = 1`. +- Calculate a tight Big-M for each pair: `M_ij = latest[i] - earliest[j] + sep[i,j]`. +- Add the separation constraint: `t[j] >= t[i] + sep[i,j] - M_ij * (1 - y[i,j])`. + +### Step 4 - Define Linear Objective +- Minimize the weighted deviation sum: `min sum( early_penalty[i] * e[i] + late_penalty[i] * l[i] )`. + +### Formulation Template +```json +{ + "sets": [ + "Entities", + "OrderedPairs" + ], + "parameters": [ + {"name": "earliest", "domain": "Entities", "type": "float"}, + {"name": "latest", "domain": "Entities", "type": "float"}, + {"name": "target", "domain": "Entities", "type": "float"}, + {"name": "early_penalty", "domain": "Entities", "type": "float"}, + {"name": "late_penalty", "domain": "Entities", "type": "float"}, + {"name": "sep", "domain": "OrderedPairs", "type": "float"} + ], + "decision_variables": [ + {"name": "t", "domain": "Entities", "type": "continuous", "bounds": "[earliest[i], latest[i]]"}, + {"name": "y", "domain": "OrderedPairs", "type": "binary"}, + {"name": "e", "domain": "Entities", "type": "continuous", "bounds": "[0, INF]"}, + {"name": "l", "domain": "Entities", "type": "continuous", "bounds": "[0, INF]"} + ], + "objective": { + "sense": "min", + "expression": "sum( early_penalty[i] * e[i] + late_penalty[i] * l[i] for i in Entities )" + }, + "constraints": [ + {"name": "deviation_early_def", "formula": "t[i] + e[i] >= target[i]", "domain": "Entities"}, + {"name": "deviation_late_def", "formula": "t[i] - l[i] <= target[i]", "domain": "Entities"}, + {"name": "mutual_exclusion", "formula": "y[i,j] + y[j,i] == 1", "domain": "OrderedPairs where i < j"}, + {"name": "separation", "formula": "t[j] >= t[i] + sep[i,j] - (latest[i] - earliest[j] + sep[i,j]) * (1 - y[i,j])", "domain": "OrderedPairs where i != j"} + ] +} +``` + +### Common Pitfalls +- Using a single, overly large Big-M for all pairs instead of calculating tight, pair-specific values. +- Incorrectly indexing separation parameters `sep[i,j]` when the separation is not symmetric. +- Defining deviation constraints that allow both `e[i]` and `l[i]` to be positive simultaneously, which is valid but may confuse interpretation. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a connected MILP solver (e.g., Gurobi, HiGHS, CBC). Configure solver-specific parameters for performance, handle termination conditions, and implement robust solution extraction and verification. + +### Step 1 - Instantiate Solver and Set Options +- Create a solver object via `SolverFactory`. +- Set key parameters: time limit, relative MIP gap tolerance, number of threads, and a random seed for reproducibility. + +### Step 2 - Solve and Capture Results +- Execute the solve with `tee=True` for optional verbose output. +- Capture the solver status and termination condition. + +### Step 3 - Process Solution Status +- Check if the solve was successful (`SolverStatus.ok` and `TerminationCondition.optimal` or `.feasible`). +- For successful solves, load the solution into the model instance and extract variable values. +- For failures (infeasible, unbounded, time limit), extract and report the relevant status information. + +### Step 4 - Verify and Report +- Recalculate the objective from the primal variable values to verify consistency. +- Optionally, run a constraint verification function to ensure all hard constraints are satisfied. +- Output a standardized result structure containing all relevant solution data. + +### Code Usage +```python +import pyomo.environ as pyo + +# Build model (assuming `model` is a Pyomo ConcreteModel) +# ... variable and constraint definitions as per modeling stage ... + +# Solve +solver = pyo.SolverFactory('gurobi') # or 'highs', 'cbc' +solver_options = { + 'TimeLimit': 60, + 'MIPGap': 1e-4, + 'Threads': 4, + 'Seed': 42 +} +results = solver.solve(model, options=solver_options, tee=False) + +# Process results +from pyomo.opt import SolverStatus, TerminationCondition +status = results.solver.status +termination = results.solver.termination_condition + +if status == SolverStatus.ok and termination in (TerminationCondition.optimal, TerminationCondition.feasible): + solution = {} + solution['status'] = str(termination) + solution['objective'] = pyo.value(model.obj) + solution['times'] = {i: pyo.value(model.t[i]) for i in model.A} + # ... extract other variables ... + # Verification logic here + return solution +else: + return {'status': f'{status}:{termination}', 'objective': None} +``` + +### Common Pitfalls +- Confusing Pyomo's `SolverStatus` with the solver's own termination condition; both must be checked. +- Not loading the solution into the model instance before extracting variable values, leading to `None` values. +- Ignoring time-limit stops; a feasible solution may still be available and should be extracted. diff --git a/skills/optskills/skill_library/set_cover_cost_minimization.md b/skills/optskills/skill_library/set_cover_cost_minimization.md new file mode 100644 index 0000000..9df03b3 --- /dev/null +++ b/skills/optskills/skill_library/set_cover_cost_minimization.md @@ -0,0 +1,236 @@ +--- +name: Set Cover Cost Minimization +description: | + Model and solve binary set cover problems with linear costs using either a direct matrix formulation or a sparse coverage mapping, and implement robust solving with verification. +--- + +# Workflow 1 (Direct Matrix Formulation with Pyomo) + +## Modeling stage + +### Strategy Overview +Model the set cover problem using an explicit binary coverage matrix parameter within a Pyomo `ConcreteModel`. This approach is well-suited for structured data and provides a clear mathematical representation of the coverage relationship between items and elements. + +### Step 1 - Define Index Sets +- Identify and create Pyomo `Set` objects for the collection of selectable items (e.g., packages) and the elements that require coverage (e.g., zones). +- Use these sets to index all model components, ensuring clean and scalable constraint generation. + +### Step 2 - Define Parameters +- Define a `Param` for item costs, indexed by the item set. +- Define a binary `Param` for the coverage matrix, indexed by (element, item). This parameter should be 1 if the item covers the element, and 0 otherwise. Use a rule for efficient sparse initialization. + +### Step 3 - Define Decision Variables +- Create a binary decision variable `x[i]` for each item `i`, where `x[i] = 1` indicates the item is selected. + +### Step 4 - Formulate Objective and Constraints +- Formulate the objective to minimize the total cost: `min sum(cost[i] * x[i] for i in items)`. +- For each element, create a coverage constraint: `sum(coverage_matrix[element, i] * x[i] for i in items) >= 1`. This ensures at least one covering item is selected per element. + +### Formulation Template +```json +{ + "sets": [ + "I: Set of selectable items.", + "S: Set of elements to be covered." + ], + "parameters": [ + "cost[i ∈ I]: Cost of selecting item i.", + "cover[s ∈ S, i ∈ I]: Binary parameter, 1 if item i covers element s." + ], + "decision_variables": [ + "x[i ∈ I]: Binary, 1 if item i is selected." + ], + "objective": { + "sense": "min", + "expression": "sum(cost[i] * x[i] for i in I)" + }, + "constraints": [ + "Coverage[s ∈ S]: sum(cover[s, i] * x[i] for i in I) >= 1" + ] +} +``` + +### Common Pitfalls +- Inefficiently initializing a dense coverage matrix for sparse problems, leading to memory overhead. +- Forgetting to verify that the coverage matrix correctly maps to the problem's 1-indexed or 0-indexed data. +- Defining constraints over incorrect index sets, which can silently produce an infeasible or incorrect model. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a MIP solver (e.g., Gurobi, HiGHS, CBC) with robust error handling. Implement a fallback strategy for solver failures and always verify the feasibility of the returned solution. + +### Step 1 - Configure Solver and Solve +- Instantiate the solver via `SolverFactory`. +- Set key parameters: a time limit, optimality gap (e.g., 0.0 for proven optimum), number of threads, and a random seed for reproducibility. +- Solve the model with `load_solutions=False` to first check the termination status without risking a crash on infeasibility. + +### Step 2 - Check Solver Status and Load Solution +- Check the solver termination condition. Accept statuses corresponding to `optimal` or `feasible`. +- If the status is acceptable, load the solution into the model instance. +- If the primary solver fails (e.g., `NoFeasibleSolutionError`), log the error and attempt with a secondary, more robust solver (e.g., switch from HiGHS to GLPK). + +### Step 3 - Extract and Verify Solution +- Extract selected items by filtering variables where `value(x[i]) > 0.5`. +- Calculate the objective value. +- Implement a verification loop: for each element, check if at least one selected item has `cover[element, item] == 1`. Flag any uncovered elements. + +### Step 4 - Structure Output +- Return results in a standardized dictionary or JSON format, including solver status, objective value, list of selected items, and a verification success flag. + +### Code Usage +```python +import pyomo.environ as pyo + +# Build model from formulation +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=items) +model.S = pyo.Set(initialize=elements) +model.cost = pyo.Param(model.I, initialize=cost_dict) +model.cover = pyo.Param(model.S, model.I, initialize=coverage_rule) +model.x = pyo.Var(model.I, domain=pyo.Binary) +model.obj = pyo.Objective(expr=sum(model.cost[i] * model.x[i] for i in model.I), sense=pyo.minimize) +model.coverage = pyo.Constraint(model.S, rule=lambda m, s: sum(m.cover[s, i] * m.x[i] for i in m.I) >= 1) + +# Solve with status / termination checks +solver_name = 'highs' +solver = pyo.SolverFactory(solver_name) +solver.options['time_limit'] = 30 +solver.options['mip_gap'] = 0.0 + +results = solver.solve(model, load_solutions=False, tee=False) + +status = results.solver.termination_condition +acceptable_status = {pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible} + +if status in acceptable_status: + model.solutions.load_from(results) + # Extract solution... +else: + # Fallback to another solver, e.g., 'glpk' + pass +``` + +### Common Pitfalls +- Loading solutions without checking status first, which can cause exceptions on infeasible models. +- Not implementing a solver fallback, leaving the workflow brittle to specific solver issues. +- Assuming the solver's feasibility report is correct without implementing independent verification. + +# Workflow 2 (Sparse Mapping Formulation with OR-Tools CP-SAT) + +## Modeling stage + +### Strategy Overview +Model the set cover problem using a sparse representation of coverage relationships, mapping each element to a list of items that cover it. This approach is efficient for problems where the coverage matrix is very sparse and aligns naturally with the OR-Tools CP-SAT API. + +### Step 1 - Prepare Sparse Coverage Data +- Represent coverage not as a matrix, but as a dictionary or list-of-lists: `coverage_sets[element_index] = [item_index1, item_index2, ...]`. +- Ensure all indices are converted to 0-based for use in Python/OR-Tools if the source data is 1-based. + +### Step 2 - Define Model and Variables +- Instantiate a `cp_model.CpModel()`. +- Create a list of Boolean decision variables `x[i]` for each item `i`, using `model.NewBoolVar()`. + +### Step 3 - Formulate Objective and Constraints +- Formulate the linear objective: `min sum(cost[i] * x[i] for i in items)`. In CP-SAT, use `AddLinearExpression` with the `Minimize` method. +- For each element, create a coverage constraint: `sum(x[i] for i in coverage_sets[element]) >= 1`. Use `model.Add(sum(element_vars) >= 1)`. + +### Formulation Template +```json +{ + "sets": [ + "I: Set of selectable items (0-indexed).", + "S: Set of elements to be covered (0-indexed)." + ], + "parameters": [ + "cost[i ∈ I]: Integer or linear cost of selecting item i.", + "covers[s ∈ S]: List of item indices in I that cover element s." + ], + "decision_variables": [ + "x[i ∈ I]: Boolean CP-SAT variable." + ], + "objective": { + "sense": "min", + "expression": "sum(cost[i] * x[i] for i in I)" + }, + "constraints": [ + "Coverage[s ∈ S]: sum(x[i] for i in covers[s]) >= 1" + ] +} +``` + +### Common Pitfalls +- Using floating-point costs with CP-SAT, which requires integer or linearized expressions. Scale costs to integers if necessary. +- Incorrectly mapping 1-indexed problem data to 0-indexed Python lists, leading to index errors or wrong coverage. +- Forgetting that CP-SAT's `BoolVar` is not a Python `bool` and must be evaluated with `solver.Value()`. + +## Solving stage + +### Strategy Overview +Solve the CP-SAT model with configured parameters for deterministic and efficient search. Extract the solution and perform an independent verification of coverage constraints. + +### Step 1 - Configure and Run Solver +- Instantiate `cp_model.CpSolver()`. +- Set key parameters: `max_time_in_seconds`, `num_search_workers` for parallelism, `random_seed` for reproducibility, and `relative_gap_limit` (e.g., 0.0 for optimality). +- Execute the solver with `solver.Solve(model)` and capture the status code. + +### Step 2 - Interpret Status and Extract Solution +- Check if the status is `OPTIMAL` or `FEASIBLE`. +- If acceptable, extract selected items by iterating over variables where `solver.Value(x[i]) == 1`. +- Retrieve the objective value via `solver.ObjectiveValue()`. + +### Step 3 - Verify Solution Feasibility +- Perform an independent verification: for each element, check if at least one item in its `coverage_sets` list is selected. +- This step validates the solver's result and catches any potential issues in model formulation or solution extraction. + +### Step 4 - Structure Output +- Return a structured result containing the status, objective value, list of selected indices, and a boolean flag indicating verification success. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# Build model from formulation +model = cp_model.CpModel() +n_items = len(costs) +x = [model.NewBoolVar(f"x_{i}") for i in range(n_items)] + +# Objective +objective_terms = [costs[i] * x[i] for i in range(n_items)] +model.Minimize(sum(objective_terms)) + +# Coverage constraints +for element_idx, covering_items in enumerate(coverage_sets): + element_vars = [x[i] for i in covering_items] + model.Add(sum(element_vars) >= 1) + +# Solve with status / termination checks +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30 +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 +solver.parameters.relative_gap_limit = 0.0 + +status = solver.Solve(model) + +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + selected = [i for i in range(n_items) if solver.Value(x[i]) == 1] + total_cost = solver.ObjectiveValue() + # Verification loop + all_covered = True + for element_idx, covering_items in enumerate(coverage_sets): + covered = any(solver.Value(x[i]) == 1 for i in covering_items) + if not covered: + all_covered = False + break +else: + # Handle no solution found + selected = [] + total_cost = None + all_covered = False +``` + +### Common Pitfalls +- Not setting a random seed, leading to non-reproducible results across runs. +- Misinterpreting the solver status codes (e.g., `UNKNOWN` vs. `FEASIBLE`). +- Skipping the verification step, which is crucial for validating that the extracted solution satisfies all original constraints. diff --git a/skills/optskills/skill_library/set_covering_with_logical_or_constraints.md b/skills/optskills/skill_library/set_covering_with_logical_or_constraints.md new file mode 100644 index 0000000..af21d69 --- /dev/null +++ b/skills/optskills/skill_library/set_covering_with_logical_or_constraints.md @@ -0,0 +1,234 @@ +--- +name: Set Covering with Logical OR Constraints +description: | + Model and solve binary selection problems where elements must be covered by at least one chosen set, minimizing total selection cost. +--- + +# Workflow 1 (MILP with Pyomo and Highs/CBC) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a standard Set Covering Integer Program using the Pyomo modeling language. This approach provides a declarative, solver-agnostic model that can be executed with open-source MILP solvers like HiGHS or CBC. + +### Step 1 - Define Sets and Parameters +- Identify the collection of available sets (e.g., routes, facilities) and the elements (e.g., points, tasks) that require coverage. +- Define a cost parameter for each set and a coverage mapping parameter that links each element to the list of sets that can cover it. + +### Step 2 - Declare Decision Variables +- Create a binary decision variable for each available set, where a value of 1 indicates the set is selected. + +### Step 3 - Formulate Coverage Constraints +- For each element, add a linear constraint ensuring the sum of the binary variables for its covering sets is at least 1. This enforces the logical OR condition. + +### Step 4 - Define the Objective Function +- Formulate a linear objective to minimize the total cost, defined as the sum of the cost of each selected set. + +### Formulation Template +```json +{ + "sets": [ + "S: Collection of available sets.", + "E: Collection of elements requiring coverage." + ], + "parameters": [ + "cost_s: Cost associated with selecting set s ∈ S.", + "coverage_e: List of sets s ∈ S that can cover element e ∈ E." + ], + "decision_variables": [ + "x_s ∈ {0,1}: 1 if set s is selected, 0 otherwise." + ], + "objective": { + "sense": "min", + "expression": "∑_{s ∈ S} cost_s * x_s" + }, + "constraints": [ + "Coverage: ∑_{s ∈ coverage_e} x_s ≥ 1, ∀ e ∈ E" + ] +} +``` + +### Common Pitfalls +- Forgetting to include all relevant sets in the coverage mapping for an element, leading to an infeasible model. +- Using floating-point numbers for costs when integer costs are more appropriate for exact solvers, which can cause precision issues in the solution logic. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a MILP solver, configure appropriate termination criteria, and implement robust solution extraction and verification. + +### Step 1 - Instantiate Solver and Set Options +- Select a MILP solver (e.g., `highs`, `cbc`). Configure a time limit and set the optimality gap tolerance to zero to guarantee finding the optimal solution. + +### Step 2 - Solve and Check Status +- Execute the solve command and capture the results object. Check the solver status and termination condition to confirm an optimal or feasible solution was found. + +### Step 3 - Extract and Verify Solution +- Extract the values of the binary variables using a tolerance (e.g., > 0.5). Programmatically verify that all coverage constraints are satisfied by the extracted solution. + +### Step 4 - Output Structured Results +- Package the optimal objective value and the list of selected sets into a structured format (e.g., dictionary, JSON) for downstream use. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Build model from formulation +model = pyo.ConcreteModel() +model.S = pyo.Set(initialize=sets_list) +model.E = pyo.Set(initialize=elements_list) +model.cost = pyo.Param(model.S, initialize=cost_dict) +model.x = pyo.Var(model.S, domain=pyo.Binary) + +# Objective +model.obj = pyo.Objective( + expr=sum(model.cost[s] * model.x[s] for s in model.S), + sense=pyo.minimize +) + +# Coverage constraints (using pre-defined coverage_dict) +def cover_rule(m, e): + covering_sets = coverage_dict[e] + return sum(m.x[s] for s in covering_sets) >= 1 +model.cover = pyo.Constraint(model.E, rule=cover_rule) + +# Solve with status / termination checks +solver = pyo.SolverFactory('highs') # or 'cbc' +solver.options['time_limit'] = 30 +solver.options['mip_rel_gap'] = 0.0 + +results = solver.solve(model, tee=False) + +# Check solution status +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition == TerminationCondition.optimal): + # Extract solution + selected_sets = [s for s in model.S if pyo.value(model.x[s]) > 0.5] + total_cost = pyo.value(model.obj) + # Verification + for e in model.E: + cover_sum = sum(pyo.value(model.x[s]) for s in coverage_dict[e]) + assert cover_sum >= 0.99, f"Element {e} not covered." +else: + # Handle suboptimal or infeasible result + selected_sets = None + total_cost = None +``` + +### Common Pitfalls +- Not checking both the solver status and termination condition, potentially interpreting a suboptimal or infeasible result as optimal. +- Using a naive equality check (`== 1.0`) to interpret binary variable values, which can fail due to solver floating-point precision; always use a tolerance. + +# Workflow 2 (Direct API with OR-Tools) + +## Modeling stage + +### Strategy Overview +Model the problem directly using a solver's API (e.g., OR-Tools). This imperative style offers fine-grained control and is efficient for prototyping or embedding within larger applications. + +### Step 1 - Initialize Solver and Create Variables +- Instantiate the MILP solver object. Create a dictionary of binary decision variables, one for each available set. + +### Step 2 - Add Coverage Constraints Imperatively +- For each element, create a linear constraint by summing the variables of its covering sets and setting the lower bound to 1. + +### Step 3 - Set the Linear Objective +- Define the objective function as the linear sum of each variable multiplied by its cost, and set the optimization sense to minimization. + +### Formulation Template +```json +{ + "sets": [ + "S: Collection of available sets.", + "E: Collection of elements requiring coverage." + ], + "parameters": [ + "cost_s: Cost associated with selecting set s ∈ S.", + "coverage_e: List of sets s ∈ S that can cover element e ∈ E." + ], + "decision_variables": [ + "x_s ∈ {0,1}: 1 if set s is selected, 0 otherwise." + ], + "objective": { + "sense": "min", + "expression": "∑_{s ∈ S} cost_s * x_s" + }, + "constraints": [ + "Coverage: ∑_{s ∈ coverage_e} x_s ≥ 1, ∀ e ∈ E" + ] +} +``` + +### Common Pitfalls +- Manually building large constraint expressions incorrectly, such as omitting a variable from a sum. Using list comprehensions over the coverage mapping is safer. +- Not leveraging the solver's ability to handle mandatory selections (x_s = 1) directly, which can simplify the model. + +## Solving stage + +### Strategy Overview +Solve the model using the solver's native methods, manage solver resources like time limits, and implement solution extraction with validation. + +### Step 1 - Configure Solver Settings +- Set practical limits on solving time and the number of threads to use, balancing speed and resource consumption. + +### Step 2 - Execute Solve and Interpret Status +- Call the solver's `Solve()` method. Interpret the returned status code to distinguish between optimal, feasible, and infeasible outcomes. + +### Step 3 - Extract and Validate the Solution +- Iterate through all decision variables, using a tolerance threshold to determine if they are selected. Compute the achieved objective value and verify all coverage constraints. + +### Step 4 - Handle Edge Cases +- Implement logic for cases where the solver hits the time limit (feasible but not proven optimal) or proves infeasibility, providing informative output. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# Build model from formulation +solver = pywraplp.Solver.CreateSolver('SCIP') # or 'CBC', 'SAT' +solver.SetTimeLimit(30000) # milliseconds +solver.SetNumThreads(4) + +# Create variables +x = {} +for s in sets_list: + x[s] = solver.IntVar(0, 1, f'x_{s}') + +# Add coverage constraints +for e in elements_list: + covering_vars = [x[s] for s in coverage_dict[e]] + constraint = solver.Sum(covering_vars) >= 1 + solver.Add(constraint) + +# Set objective +objective = solver.Objective() +for s in sets_list: + objective.SetCoefficient(x[s], cost_dict[s]) +objective.SetMinimization() + +# Solve with status / termination checks +status = solver.Solve() + +if status in (solver.OPTIMAL, solver.FEASIBLE): + # Extract solution + selected_sets = [s for s in sets_list if x[s].solution_value() > 0.5] + total_cost = objective.Value() + # Verification + for e in elements_list: + cover_sum = sum(x[s].solution_value() for s in coverage_dict[e]) + if cover_sum < 0.99: + raise AssertionError(f"Coverage violation for element {e}") +else: + # Handle no solution found + selected_sets = None + total_cost = None + if status == solver.INFEASIBLE: + print("Model is infeasible.") + elif status == solver.NOT_SOLVED: + print("Solver did not find a solution within limits.") +``` + +### Common Pitfalls +- Confusing the `FEASIBLE` status (found a solution, not proven optimal) with `OPTIMAL`. This can lead to reporting suboptimal solutions as optimal. +- Neglecting to set a time limit, potentially allowing the solver to run indefinitely on large or complex instances. diff --git a/skills/optskills/skill_library/single_vehicle_routing_with_pickup_and_delivery_precedence.md b/skills/optskills/skill_library/single_vehicle_routing_with_pickup_and_delivery_precedence.md new file mode 100644 index 0000000..547e996 --- /dev/null +++ b/skills/optskills/skill_library/single_vehicle_routing_with_pickup_and_delivery_precedence.md @@ -0,0 +1,339 @@ +--- +name: Single Vehicle Routing with Pickup and Delivery Precedence +description: | + Model and solve a single-vehicle routing problem that must start and end at a depot, visit all nodes exactly once, and respect precedence constraints for pickup-delivery pairs, minimizing total travel distance. +--- + +# Workflow 1 (OR-Tools Routing Model) + +## Modeling stage + +### Strategy Overview +Use OR-Tools' `RoutingModel` to formulate the problem as a Vehicle Routing Problem (VRP) with a single vehicle. The solver natively handles tour constraints and provides built-in support for pickup-delivery precedence through cumulative dimension variables. + +### Step 1 - Define Distance Matrix and Problem Parameters +- Create a 2D list `dist_matrix` where `dist_matrix[i][j]` is the travel cost from node `i` to node `j`. Ensure the matrix is symmetric if distances are undirected. +- Set `num_nodes` to the total number of locations including the depot (index 0). +- Set `num_vehicles = 1` and `depot_index = 0`. + +### Step 2 - Initialize Routing Index Manager and Model +- Instantiate `RoutingIndexManager(num_nodes, num_vehicles, depot_index)` to map between internal indices and original node IDs. +- Create `RoutingModel(manager)` to build the routing model. + +### Step 3 - Register Transit Callback and Set Arc Cost +- Define a distance callback function that returns `dist_matrix[from_node][to_node]` using `manager.IndexToNode(index)`. +- Register the callback with `routing.RegisterTransitCallback(distance_callback)`. +- Set it as the arc cost evaluator: `routing.SetArcCostEvaluatorOfAllVehicles(transit_callback_id)`. + +### Step 4 - Add Distance Dimension for Precedence Enforcement +- Add a distance dimension: `routing.AddDimension(transit_callback_id, 0, max_distance, True, "Distance")` where `max_distance` is a sufficiently large upper bound (e.g., sum of all distances). +- This dimension tracks cumulative travel distance along the route, enabling precedence constraints via cumulative variables. + +### Step 5 - Enforce Pickup-Delivery Precedence Constraints +- For each pickup-delivery pair `(pickup_node, delivery_node)`: + - Get indices: `pickup_index = manager.NodeToIndex(pickup_node)`, `delivery_index = manager.NodeToIndex(delivery_node)`. + - Ensure same vehicle: `routing.solver().Add(routing.VehicleVar(pickup_index) == routing.VehicleVar(delivery_index))`. + - Enforce precedence: `routing.solver().Add(distance_dimension.CumulVar(pickup_index) <= distance_dimension.CumulVar(delivery_index))`. + +### Step 6 - Configure Search Parameters +- Set first solution strategy: `search_params.first_solution_strategy = routing_enums_pb2.FirstSolutionStrategy.PATH_CHEAPEST_ARC`. +- Set local search metaheuristic: `search_params.local_search_metaheuristic = routing_enums_pb2.LocalSearchMetaheuristic.GUIDED_LOCAL_SEARCH`. +- Set time limit: `search_params.time_limit.seconds = [TIME_LIMIT]`. +- Set solution limit: `search_params.solution_limit = [SOLUTION_LIMIT]`. + +### Formulation Template +```json +{ + "sets": ["N: set of nodes (0 = depot, 1..n-1 = locations)", "P: set of pickup-delivery pairs (p, d)"], + "parameters": ["dist_matrix[i][j]: travel distance from node i to node j", "max_distance: upper bound for cumulative distance"], + "decision_variables": ["vehicle_route: sequence of nodes visited by the single vehicle"], + "objective": { + "sense": "min", + "expression": "sum of dist_matrix[route[i]][route[i+1]] for all consecutive nodes in route" + }, + "constraints": [ + "tour: route starts and ends at depot", + "visit_all_nodes_exactly_once: each node appears exactly once in route", + "precedence_pickup_before_delivery: for each (p, d) in P, pickup node appears before delivery node in route" + ] +} +``` + +### Common Pitfalls +- Forgetting to convert between manager indices and node IDs when defining the distance callback, causing incorrect cost evaluation. +- Setting `max_distance` too small in `AddDimension`, which may incorrectly prune feasible solutions. +- Omitting the same-vehicle constraint for pickup-delivery pairs when using multiple vehicles (not needed for single vehicle but good practice for extensibility). + +## Solving stage + +### Strategy Overview +Solve the routing model with configured parameters, extract the optimal route, validate constraints manually, and output results in a structured JSON format for downstream consumption. + +### Step 1 - Solve the Model +- Call `solution = routing.SolveWithParameters(search_params)`. +- Check if `solution` is `None`; if so, print failure JSON: `{"status": "failed", "solver_status": routing.status()}`. + +### Step 2 - Extract the Route +- Start at `index = routing.Start(0)`. +- While not `routing.IsEnd(index)`: + - Append `manager.IndexToNode(index)` to route list. + - Set `index = solution.Value(routing.NextVar(index))`. +- Append `manager.IndexToNode(index)` to include the return to depot. + +### Step 3 - Compute Total Distance +- Use `solution.ObjectiveValue()` for the solver-computed total distance. +- Optionally compute manually by summing distances between consecutive nodes in the extracted route to verify. + +### Step 4 - Validate Solution Correctness +- Verify route starts and ends at depot (node 0). +- Check that all nodes appear exactly once (set equality). +- For each pickup-delivery pair, confirm pickup index < delivery index in the route list. +- Manually compute total distance: `sum(dist_matrix[route[i]][route[i+1]] for i in range(len(route)-1))` and compare to solver's objective. + +### Step 5 - Output Structured JSON +- Build payload with keys: `"status"`, `"objective"`, `"route"`, `"total_distance_computed"`, `"solver_status"`. +- Print using `print(f"RESULT_JSON:{json.dumps(payload)}")`. + +### Code Usage +```python +from ortools.constraint_solver import routing_enums_pb2, pywrapcp +import json + +def solve_vrp_pd(dist_matrix, pickup_delivery_pairs, depot=0, time_limit=30, solution_limit=100): + num_nodes = len(dist_matrix) + manager = pywrapcp.RoutingIndexManager(num_nodes, 1, depot) + routing = pywrapcp.RoutingModel(manager) + + def distance_callback(from_index, to_index): + from_node = manager.IndexToNode(from_index) + to_node = manager.IndexToNode(to_index) + return dist_matrix[from_node][to_node] + + transit_callback_id = routing.RegisterTransitCallback(distance_callback) + routing.SetArcCostEvaluatorOfAllVehicles(transit_callback_id) + + max_distance = sum(sum(row) for row in dist_matrix) # safe upper bound + routing.AddDimension(transit_callback_id, 0, max_distance, True, "Distance") + distance_dimension = routing.GetDimensionOrDie("Distance") + + for pickup, delivery in pickup_delivery_pairs: + pickup_index = manager.NodeToIndex(pickup) + delivery_index = manager.NodeToIndex(delivery) + routing.solver().Add(routing.VehicleVar(pickup_index) == routing.VehicleVar(delivery_index)) + routing.solver().Add(distance_dimension.CumulVar(pickup_index) <= distance_dimension.CumulVar(delivery_index)) + + search_params = pywrapcp.DefaultRoutingSearchParameters() + search_params.first_solution_strategy = routing_enums_pb2.FirstSolutionStrategy.PATH_CHEAPEST_ARC + search_params.local_search_metaheuristic = routing_enums_pb2.LocalSearchMetaheuristic.GUIDED_LOCAL_SEARCH + search_params.time_limit.seconds = time_limit + search_params.solution_limit = solution_limit + + solution = routing.SolveWithParameters(search_params) + if not solution: + print(f"RESULT_JSON:{json.dumps({'status': 'failed', 'solver_status': routing.status()})}") + return + + route = [] + index = routing.Start(0) + while not routing.IsEnd(index): + route.append(manager.IndexToNode(index)) + index = solution.Value(routing.NextVar(index)) + route.append(manager.IndexToNode(index)) + + total_distance = solution.ObjectiveValue() + payload = { + "status": "success", + "objective": total_distance, + "route": route, + "total_distance_computed": total_distance, + "solver_status": routing.status() + } + print(f"RESULT_JSON:{json.dumps(payload)}") +``` + +### Common Pitfalls +- Not checking for `None` solution before extraction, causing runtime errors. +- Extracting the route incorrectly by missing the final return to depot edge. +- Assuming solver status alone indicates feasibility; always validate constraints manually after extraction. + +# Workflow 2 (CP-SAT Circuit Model) + +## Modeling stage + +### Strategy Overview +Use OR-Tools CP-SAT solver with the `AddCircuit` constraint to enforce a single Hamiltonian cycle. Introduce integer position variables to model precedence constraints, avoiding complex subtour elimination formulations. + +### Step 1 - Define Sets and Parameters +- Let `N = {0, 1, ..., n-1}` where node 0 is the depot. +- Define `dist_matrix[i][j]` for all `i, j in N, i != j`. +- Define `pickup_delivery_pairs` as list of tuples `(pickup, delivery)`. + +### Step 2 - Create Binary Arc Variables +- For each ordered pair `(i, j)` with `i != j`, create `x[i][j] = model.NewBoolVar(f'x_{i}_{j}')`. +- These variables indicate whether the tour directly travels from node `i` to node `j`. + +### Step 3 - Enforce Hamiltonian Cycle with AddCircuit +- Build a list of arcs: for each `i != j`, append `(i, j, x[i][j])` to the arcs list. +- Add `model.AddCircuit(arcs)` to enforce a single tour visiting all nodes exactly once. + +### Step 4 - Create Position Variables for Precedence +- For each node `i`, create `u[i] = model.NewIntVar(0, n-1, f'u_{i}')` representing the visit order. +- Add `model.Add(u[0] == 0)` to fix depot as first node. +- Add `model.AddAllDifferent(u)` to ensure unique positions. + +### Step 5 - Link Arc Variables to Position Variables +- For each arc `(i, j)` where `j != 0` (non-depot destination): `model.Add(u[j] == u[i] + 1).OnlyEnforceIf(x[i][j])`. +- For arcs returning to depot `(i, 0)`: `model.Add(u[i] == n - 1).OnlyEnforceIf(x[i][0])`. +- For arcs leaving depot `(0, j)`: `model.Add(u[j] == 1).OnlyEnforceIf(x[0][j])`. + +### Step 6 - Add Precedence Constraints +- For each pickup-delivery pair `(p, d)`, add `model.Add(u[p] < u[d])`. + +### Step 7 - Define Objective +- Minimize total travel distance: `model.Minimize(sum(dist_matrix[i][j] * x[i][j] for i in N for j in N if i != j))`. + +### Formulation Template +```json +{ + "sets": ["N: set of nodes (0 = depot, 1..n-1 = locations)", "A: set of ordered pairs (i, j) for i != j"], + "parameters": ["dist_matrix[i][j]: travel distance from node i to node j"], + "decision_variables": [ + "x[i][j]: binary, 1 if tour travels directly from i to j", + "u[i]: integer, position of node i in the tour (0-indexed)" + ], + "objective": { + "sense": "min", + "expression": "sum_{i,j in A} dist_matrix[i][j] * x[i][j]" + }, + "constraints": [ + "model.AddCircuit(arcs): enforces single Hamiltonian cycle", + "u[0] == 0: depot is first", + "model.AddAllDifferent(u): unique positions", + "u[j] == u[i] + 1 .OnlyEnforceIf(x[i][j]) for j != 0: arc-position consistency", + "u[i] == n - 1 .OnlyEnforceIf(x[i][0]): return to depot is last", + "u[j] == 1 .OnlyEnforceIf(x[0][j]): first move from depot", + "u[p] < u[d] for each pickup-delivery pair (p, d): precedence" + ] +} +``` + +### Common Pitfalls +- Forgetting to include arcs for all ordered pairs in `AddCircuit`, causing infeasibility. +- Adding position constraints for depot arcs incorrectly (e.g., using `u[j] == u[i] + 1` for arcs returning to depot), which over-constrains the model. +- Not using `OnlyEnforceIf` correctly, leading to constraints that apply even when the arc is not selected. + +## Solving stage + +### Strategy Overview +Configure the CP-SAT solver with appropriate parameters for optimality, extract the tour from solved arc variables, validate all constraints, and output structured results. + +### Step 1 - Configure and Run Solver +- Create `solver = cp_model.CpSolver()`. +- Set parameters: `solver.parameters.max_time_in_seconds = [TIME_LIMIT]`, `solver.parameters.num_search_workers = [NUM_WORKERS]`, `solver.parameters.random_seed = [SEED]`, `solver.parameters.relative_gap_limit = 0.0`. +- Call `status = solver.Solve(model)`. + +### Step 2 - Check Solver Status +- If `status` is not `cp_model.OPTIMAL` or `cp_model.FEASIBLE`, output failure JSON: `{"status": "failed", "reason": "no_feasible_solution", "solver_status": int(status)}`. + +### Step 3 - Extract the Tour +- Start at `current = 0`. +- Initialize `route = [0]`. +- For `step` in range `n-1`: + - Find `next_node` where `solver.Value(x[current][next_node]) == 1` and `next_node != 0`. + - Append `next_node` to route, set `current = next_node`. +- Append `0` to close the tour. + +### Step 4 - Validate Solution +- Verify route length equals `n + 1` (includes return to depot). +- Check all nodes appear exactly once (excluding final depot). +- For each pickup-delivery pair, find indices in route and confirm pickup index < delivery index. + +### Step 5 - Output Structured JSON +- Build payload with keys: `"status"`, `"objective"`, `"route"`, `"positions"`, `"solver_status"`. +- Print using `print(f"RESULT_JSON:{json.dumps(payload)}")`. + +### Code Usage +```python +from ortools.sat.python import cp_model +import json + +def solve_tsp_pd_cpsat(dist_matrix, pickup_delivery_pairs, time_limit=30, num_workers=8, seed=42): + n = len(dist_matrix) + model = cp_model.CpModel() + + # Binary arc variables + x = {} + for i in range(n): + for j in range(n): + if i != j: + x[i, j] = model.NewBoolVar(f'x_{i}_{j}') + + # Circuit constraint + arcs = [] + for i in range(n): + for j in range(n): + if i != j: + arcs.append((i, j, x[i, j])) + model.AddCircuit(arcs) + + # Position variables + u = [model.NewIntVar(0, n-1, f'u_{i}') for i in range(n)] + model.Add(u[0] == 0) + model.AddAllDifferent(u) + + # Link arcs to positions + for i in range(n): + for j in range(1, n): # non-depot destinations + if i != j: + model.Add(u[j] == u[i] + 1).OnlyEnforceIf(x[i, j]) + if i != 0: # arcs returning to depot + model.Add(u[i] == n - 1).OnlyEnforceIf(x[i, 0]) + for j in range(1, n): # arcs leaving depot + model.Add(u[j] == 1).OnlyEnforceIf(x[0, j]) + + # Precedence constraints + for pickup, delivery in pickup_delivery_pairs: + model.Add(u[pickup] < u[delivery]) + + # Objective + model.Minimize(sum(dist_matrix[i][j] * x[i, j] for i in range(n) for j in range(n) if i != j)) + + # Solve + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = time_limit + solver.parameters.num_search_workers = num_workers + solver.parameters.random_seed = seed + solver.parameters.relative_gap_limit = 0.0 + status = solver.Solve(model) + + if status not in (cp_model.OPTIMAL, cp_model.FEASIBLE): + print(f"RESULT_JSON:{json.dumps({'status': 'failed', 'reason': 'no_feasible_solution', 'solver_status': int(status)})}") + return + + # Extract route + route = [0] + current = 0 + for _ in range(n - 1): + for j in range(n): + if j != current and solver.Value(x[current, j]) == 1: + route.append(j) + current = j + break + route.append(0) + + total_distance = int(solver.ObjectiveValue()) + positions = [solver.Value(u[i]) for i in range(n)] + payload = { + "status": "success", + "objective": total_distance, + "route": route, + "positions": positions, + "solver_status": int(status) + } + print(f"RESULT_JSON:{json.dumps(payload)}") +``` + +### Common Pitfalls +- Not setting `relative_gap_limit = 0.0` when exact optimality is required, potentially returning suboptimal solutions. +- Extracting the tour incorrectly by not breaking out of the inner loop after finding the next node, leading to duplicate entries. +- Forgetting to validate that the extracted route actually forms a valid Hamiltonian cycle (e.g., missing nodes or incorrect return to depot). diff --git a/skills/optskills/skill_library/stable_matching_egalitarian_optimizer.md b/skills/optskills/skill_library/stable_matching_egalitarian_optimizer.md new file mode 100644 index 0000000..d71e969 --- /dev/null +++ b/skills/optskills/skill_library/stable_matching_egalitarian_optimizer.md @@ -0,0 +1,297 @@ +--- +name: Stable Matching Egalitarian Optimizer +description: | + Builds and solves a one-to-one stable matching problem with no blocking pairs, minimizing the egalitarian score (sum of mutual preference ranks) using either constraint programming or mixed-integer linear programming. + +--- +# Workflow 1 (CP-SAT Constraint Programming) + +## Modeling stage + +### Strategy Overview +Model the stable matching problem as a binary assignment problem with explicit no-blocking-pair constraints. Use OR-Tools CP-SAT solver which handles logical constraints and binary variables efficiently for small to medium instances (up to ~7x7). + +### Step 1 - Define Preference Rank Maps +- Convert each agent's preference list into zero-indexed rank dictionaries `rank_A[i][j]` and `rank_B[j][i]` for O(1) lookup of preference strength. +- Precompute for each pair `(i, j)` the sets `pref_A[i][j]` (partners `i` prefers over `j`) and `pref_B[i][j]` (partners `j` prefers over `i`). + +### Step 2 - Declare Binary Assignment Variables +- Define `x[i, j]` as a `cp_model.NewBoolVar` for each pair `(i, j)` where `i` is from set A and `j` from set B. +- Variable equals 1 if `i` is matched to `j`, 0 otherwise. + +### Step 3 - Enforce One-to-One Matching +- Add constraint `sum(x[i, j] for j in set_B) == 1` for each `i` in set A. +- Add constraint `sum(x[i, j] for i in set_A) == 1` for each `j` in set B. + +### Step 4 - Add No-Blocking-Pair Constraints +- For each pair `(i, j)`, add the linear inequality: + `x[i, j] + sum(x[i, jj] for jj in pref_A[i][j]) + sum(x[ii, j] for ii in pref_B[i][j]) >= 1`. +- This ensures if `i` and `j` are not matched, at least one is matched to a partner they prefer over the other. + +### Step 5 - Define Egalitarian Objective +- Minimize `sum((rank_A[i][j] + rank_B[j][i]) * x[i, j] for all i, j)`. + +### Formulation Template +```json +{ + "sets": ["A: set of agents in first group", "B: set of agents in second group"], + "parameters": ["rank_A[i][j]: zero-indexed rank of j in i's preference list", "rank_B[j][i]: zero-indexed rank of i in j's preference list"], + "decision_variables": ["x[i,j] binary: 1 if i matched to j, else 0"], + "objective": { + "sense": "min", + "expression": "sum_{i in A, j in B} (rank_A[i][j] + rank_B[j][i]) * x[i,j]" + }, + "constraints": [ + "sum_{j in B} x[i,j] == 1 for all i in A", + "sum_{i in A} x[i,j] == 1 for all j in B", + "x[i,j] + sum_{jj in pref_A[i][j]} x[i,jj] + sum_{ii in pref_B[i][j]} x[ii,j] >= 1 for all i in A, j in B" + ] +} +``` + +### Common Pitfalls +- Forgetting to zero-index ranks: preference positions must start at 0 for the objective to correctly represent egalitarian cost. +- Using `>=` instead of `==` for one-to-one constraints: each agent must be matched exactly once. +- Not precomputing `pref_A` and `pref_B` sets: computing them inside constraint generation loops causes performance degradation. + +## Solving stage + +### Strategy Overview +Use OR-Tools CP-SAT solver with parallel search and time limit. Extract the matching by iterating over all binary variables, then verify stability and compute objective value. + +### Step 1 - Initialize Solver and Configure +- Create `cp_model.CpModel()` instance. +- Set solver parameters: `max_time_in_seconds=[TIME_LIMIT]` (e.g., 30), `num_search_workers=8`, `random_seed=42`, `relative_gap_limit=0.0`. + +### Step 2 - Build Model and Solve +- Add all variables, constraints, and objective to the model. +- Call `solver.Solve(model)` and check status against `cp_model.OPTIMAL` or `cp_model.FEASIBLE`. + +### Step 3 - Extract Matching +- Iterate over all `(i, j)` pairs and collect those where `solver.Value(x[i, j]) == 1`. +- Build a dictionary `matching[i] = j` and a reverse dictionary `reverse_matching[j] = i` for verification. + +### Step 4 - Verify Stability +- For each pair `(i, j)` where `matching[i] != j`, check if `rank_A[i][j] < rank_A[i][matching[i]]` and `rank_B[j][i] < rank_B[j][reverse_matching[j]]`. +- If any such blocking pair exists, the solution is invalid; log a warning. + +### Step 5 - Compute and Output Objective +- Compute `sum(rank_A[i][matching[i]] + rank_B[matching[i]][i] for all i)`. +- On success, output `RESULT:{objective_value}`. On failure, output a JSON error payload. + +### Code Usage +```python +from ortools.sat.python import cp_model + +def solve_stable_matching(A_prefs: dict, B_prefs: dict): + n = len(A_prefs) + # Build rank maps + rank_A = {i: {j: pos for pos, j in enumerate(prefs)} for i, prefs in A_prefs.items()} + rank_B = {j: {i: pos for pos, i in enumerate(prefs)} for j, prefs in B_prefs.items()} + # Precompute preference sets for stability constraints + pref_A = {i: {j: [jj for jj in range(n) if rank_A[i][jj] < rank_A[i][j]] for j in range(n)} for i in range(n)} + pref_B = {j: {i: [ii for ii in range(n) if rank_B[j][ii] < rank_B[j][i]] for i in range(n)} for j in range(n)} + + model = cp_model.CpModel() + x = {} + for i in range(n): + for j in range(n): + x[i, j] = model.NewBoolVar(f'x_{i}_{j}') + + # One-to-one constraints + for i in range(n): + model.Add(sum(x[i, j] for j in range(n)) == 1) + for j in range(n): + model.Add(sum(x[i, j] for i in range(n)) == 1) + + # Stability constraints + for i in range(n): + for j in range(n): + model.Add(x[i, j] + + sum(x[i, jj] for jj in pref_A[i][j]) + + sum(x[ii, j] for ii in pref_B[j][i]) >= 1) + + # Objective + model.Minimize(sum((rank_A[i][j] + rank_B[j][i]) * x[i, j] for i in range(n) for j in range(n))) + + solver = cp_model.CpSolver() + solver.parameters.max_time_in_seconds = 30 + solver.parameters.num_search_workers = 8 + solver.parameters.random_seed = 42 + solver.parameters.relative_gap_limit = 0.0 + + status = solver.Solve(model) + if status == cp_model.OPTIMAL or status == cp_model.FEASIBLE: + matching = {i: j for i in range(n) for j in range(n) if solver.Value(x[i, j]) == 1} + reverse_matching = {j: i for i, j in matching.items()} + # Verify stability + stable = True + for i in range(n): + for j in range(n): + if matching[i] != j: + if (rank_A[i][j] < rank_A[i][matching[i]] and + rank_B[j][i] < rank_B[j][reverse_matching[j]]): + stable = False + break + if not stable: + break + obj_val = sum(rank_A[i][matching[i]] + rank_B[matching[i]][i] for i in range(n)) + return {"status": "optimal" if status == cp_model.OPTIMAL else "feasible", + "objective": obj_val, + "matching": matching, + "stable": stable} + else: + return {"status": "failed", "error": f"Solver status: {solver.StatusName(status)}"} +``` + +### Common Pitfalls +- Not checking solver status before extracting values: accessing `solver.Value()` on an unsolved model raises errors. +- Using `solver.Value()` on variables that are not part of the optimal solution: always verify status first. +- Forgetting to set `relative_gap_limit=0.0` for provably optimal solutions: default gap may stop early with a suboptimal result. + +# Workflow 2 (MILP with Pyomo) + +## Modeling stage + +### Strategy Overview +Formulate the stable matching problem as a mixed-integer linear program using Pyomo. Use the same binary assignment variables and no-blocking-pair constraints, and leverage a MILP solver (e.g., HiGHS, Gurobi) for the solving stage. + +### Step 1 - Build Preference Rank Maps +- Convert preference lists into zero-indexed rank dictionaries `rank_A[i][j]` and `rank_B[j][i]`. +- Precompute for each pair `(i, j)` the sets `pref_A[i][j]` and `pref_B[i][j]`. + +### Step 2 - Define Pyomo Sets and Variables +- Create `pyo.Set` objects for sets A and B with integer indices `0..n-1`. +- Declare `pyo.Var(m.A, m.B, domain=pyo.Binary)` for assignment variables `m.x[i,j]`. + +### Step 3 - Add One-to-One Matching Constraints +- Add constraint `sum(m.x[i, j] for j in m.B) == 1` for each `i` in `m.A`. +- Add constraint `sum(m.x[i, j] for i in m.A) == 1` for each `j` in `m.B`. + +### Step 4 - Add Stability Constraints +- For each pair `(i, j)`, add constraint: + `m.x[i, j] + sum(m.x[i, jj] for jj in pref_A[i][j]) + sum(m.x[ii, j] for ii in pref_B[i][j]) >= 1`. + +### Step 5 - Set Egalitarian Objective +- Minimize `sum((rank_A[i][j] + rank_B[j][i]) * m.x[i, j] for i in m.A for j in m.B)`. + +### Formulation Template +```json +{ + "sets": ["A: agents in first group (0..n-1)", "B: agents in second group (0..n-1)"], + "parameters": ["rank_A[i][j]: zero-indexed rank of j in i's list", "rank_B[j][i]: zero-indexed rank of i in j's list"], + "decision_variables": ["x[i,j] binary: 1 if i matched to j"], + "objective": { + "sense": "min", + "expression": "sum_{i in A, j in B} (rank_A[i][j] + rank_B[j][i]) * x[i,j]" + }, + "constraints": [ + "sum_{j in B} x[i,j] == 1 for all i in A", + "sum_{i in A} x[i,j] == 1 for all j in B", + "x[i,j] + sum_{jj in pref_A[i][j]} x[i,jj] + sum_{ii in pref_B[i][j]} x[ii,j] >= 1 for all i in A, j in B" + ] +} +``` + +### Common Pitfalls +- Using `pyo.Set(initialize=range(n))` without specifying `dimen`: for 1D sets this is fine, but for 2D parameter indexing ensure correct dimensionality. +- Forgetting to pass `m` as first argument in constraint rule functions: Pyomo rule functions must accept the model as first parameter. +- Not precomputing preference sets inside the rule function: computing them inside the rule causes repeated computation for each constraint evaluation. + +## Solving stage + +### Strategy Overview +Use a MILP solver (HiGHS recommended for open-source, Gurobi for commercial) via Pyomo's solver interface. Configure optimality gap and time limit, then extract and verify the solution. + +### Step 1 - Initialize Solver +- Create solver instance: `solver = pyo.SolverFactory("highs")` or `solver = pyo.SolverFactory("gurobi")`. +- Set options: `solver.options["time_limit"] = [TIME_LIMIT]` (e.g., 60), `solver.options["mip_rel_gap"] = 0.0`. + +### Step 2 - Solve Model +- Call `results = solver.solve(m, tee=True)` to see solver log. +- Check `results.solver.status == SolverStatus.ok` and `results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}`. + +### Step 3 - Extract Matching +- Iterate over all `(i, j)` pairs and collect those where `pyo.value(m.x[i, j]) > 0.5`. +- Build matching dictionary `matching[i] = j` and reverse dictionary `reverse_matching[j] = i`. + +### Step 4 - Verify Stability +- For each pair `(i, j)` where `matching[i] != j`, check if `rank_A[i][j] < rank_A[i][matching[i]]` and `rank_B[j][i] < rank_B[j][reverse_matching[j]]`. +- Print warning if any blocking pair is found. + +### Step 5 - Output Results +- Compute objective value manually from extracted matching. +- Output JSON payload with keys `status`, `objective`, `matching`, and `stable`. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +def solve_stable_matching_milp(A_prefs: dict, B_prefs: dict): + n = len(A_prefs) + # Build rank maps + rank_A = {i: {j: pos for pos, j in enumerate(prefs)} for i, prefs in A_prefs.items()} + rank_B = {j: {i: pos for pos, i in enumerate(prefs)} for j, prefs in B_prefs.items()} + # Precompute preference sets for stability constraints + pref_A = {i: {j: [jj for jj in range(n) if rank_A[i][jj] < rank_A[i][j]] for j in range(n)} for i in range(n)} + pref_B = {j: {i: [ii for ii in range(n) if rank_B[j][ii] < rank_B[j][i]] for i in range(n)} for j in range(n)} + + m = pyo.ConcreteModel() + m.A = pyo.Set(initialize=range(n)) + m.B = pyo.Set(initialize=range(n)) + m.x = pyo.Var(m.A, m.B, domain=pyo.Binary) + + # Objective + m.obj = pyo.Objective(expr=sum((rank_A[i][j] + rank_B[j][i]) * m.x[i, j] for i in m.A for j in m.B), + sense=pyo.minimize) + + # One-to-one constraints + def one_to_one_A_rule(mm, i): + return sum(mm.x[i, j] for j in mm.B) == 1 + m.con_one_A = pyo.Constraint(m.A, rule=one_to_one_A_rule) + + def one_to_one_B_rule(mm, j): + return sum(mm.x[i, j] for i in mm.A) == 1 + m.con_one_B = pyo.Constraint(m.B, rule=one_to_one_B_rule) + + # Stability constraints + def stability_rule(mm, i, j): + return (mm.x[i, j] + + sum(mm.x[i, jj] for jj in pref_A[i][j]) + + sum(mm.x[ii, j] for ii in pref_B[j][i]) >= 1) + m.con_stability = pyo.Constraint(m.A, m.B, rule=stability_rule) + + # Solve + solver = pyo.SolverFactory('highs') + solver.options['time_limit'] = 60 + solver.options['mip_rel_gap'] = 0.0 + results = solver.solve(m, tee=True) + + if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}): + matching = {i: j for i in m.A for j in m.B if pyo.value(m.x[i, j]) > 0.5} + reverse_matching = {j: i for i, j in matching.items()} + # Verify stability + stable = True + for i in range(n): + for j in range(n): + if matching[i] != j: + if (rank_A[i][j] < rank_A[i][matching[i]] and + rank_B[j][i] < rank_B[j][reverse_matching[j]]): + stable = False + break + if not stable: + break + obj_val = sum(rank_A[i][matching[i]] + rank_B[matching[i]][i] for i in range(n)) + status = "optimal" if results.solver.termination_condition == TerminationCondition.optimal else "feasible" + return {"status": status, "objective": obj_val, "matching": matching, "stable": stable} + else: + return {"status": "failed", + "error": f"Solver status: {results.solver.status}, termination: {results.solver.termination_condition}"} +``` + +### Common Pitfalls +- Not checking `results.solver.status` before accessing variable values: failed solves produce no valid solution. +- Using `pyo.value()` on variables that may not be fixed: always verify solver status first. +- Setting `mip_rel_gap` to 0.0 without a time limit: may cause extremely long solve times for larger instances; always set a time limit as a safety net. diff --git a/skills/optskills/skill_library/supplier_selection_with_minimum_quantity_and_count.md b/skills/optskills/skill_library/supplier_selection_with_minimum_quantity_and_count.md new file mode 100644 index 0000000..c98b693 --- /dev/null +++ b/skills/optskills/skill_library/supplier_selection_with_minimum_quantity_and_count.md @@ -0,0 +1,261 @@ +--- +name: Supplier Selection with Minimum Quantity and Count +description: | + Model and solve assignment problems with minimum quantity requirements if selected and minimum supplier count constraints using mixed-integer linear programming. +--- + +# Workflow 1 (Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +This workflow uses Pyomo's abstract modeling syntax to define a MILP with explicit linking constraints between continuous allocation and binary selection variables. It is designed for clarity and solver-agnostic execution, suitable for problems where data is provided via dictionaries. + +### Step 1 - Define Data Structures +- Use Python dictionaries to store all input parameters, keyed by producer and contract identifiers. +- Define `cost[(i, j)]` for unit costs, `capacity[i]` for producer limits, `demand[j]` for contract requirements, `min_quantity[i]` for minimum allocation if selected, and `min_suppliers[j]` for the required minimum number of producers per contract. + +### Step 2 - Declare Decision Variables +- Create a continuous, non-negative variable `assignment_quantity[i, j]` for the amount allocated from producer `i` to contract `j`. +- Create a binary variable `binary_assignment[i, j]` to indicate if producer `i` is selected for contract `j`. + +### Step 3 - Formulate the Objective +- Minimize the total cost: sum of `cost[i, j] * assignment_quantity[i, j]` over all producer-contract pairs. + +### Step 4 - Implement Core Constraints +- **Capacity Limit**: For each producer `i`, sum of `assignment_quantity[i, j]` over all contracts `j` must be ≤ `capacity[i]`. +- **Demand Satisfaction**: For each contract `j`, sum of `assignment_quantity[i, j]` over all producers `i` must be ≥ `demand[j]`. +- **Minimum Supplier Count**: For each contract `j`, sum of `binary_assignment[i, j]` over all producers `i` must be ≥ `min_suppliers[j]`. + +### Step 5 - Link Continuous and Binary Variables +- **Upper Bound Link**: `assignment_quantity[i, j] ≤ capacity[i] * binary_assignment[i, j]`. This forces allocation to zero if not selected. +- **Lower Bound Link**: `assignment_quantity[i, j] ≥ min_quantity[i] * binary_assignment[i, j]`. This enforces the minimum quantity if selected. + +### Formulation Template +```json +{ + "sets": [ + "PRODUCERS", + "CONTRACTS" + ], + "parameters": { + "cost": {"type": "float", "index": ["PRODUCERS", "CONTRACTS"]}, + "capacity": {"type": "float", "index": ["PRODUCERS"]}, + "demand": {"type": "float", "index": ["CONTRACTS"]}, + "min_quantity": {"type": "float", "index": ["PRODUCERS"]}, + "min_suppliers": {"type": "int", "index": ["CONTRACTS"]} + }, + "decision_variables": [ + {"name": "assignment_quantity", "type": "continuous", "bounds": [0, null], "index": ["PRODUCERS", "CONTRACTS"]}, + {"name": "binary_assignment", "type": "binary", "index": ["PRODUCERS", "CONTRACTS"]} + ], + "objective": { + "sense": "min", + "expression": "sum(cost[i,j] * assignment_quantity[i,j] for i in PRODUCERS for j in CONTRACTS)" + }, + "constraints": [ + {"name": "capacity_limit", "expression": "sum(assignment_quantity[i,j] for j in CONTRACTS) <= capacity[i] for i in PRODUCERS"}, + {"name": "demand_satisfaction", "expression": "sum(assignment_quantity[i,j] for i in PRODUCERS) >= demand[j] for j in CONTRACTS"}, + {"name": "min_supplier_count", "expression": "sum(binary_assignment[i,j] for i in PRODUCERS) >= min_suppliers[j] for j in CONTRACTS"}, + {"name": "link_upper", "expression": "assignment_quantity[i,j] <= capacity[i] * binary_assignment[i,j] for i in PRODUCERS for j in CONTRACTS"}, + {"name": "link_lower", "expression": "assignment_quantity[i,j] >= min_quantity[i] * binary_assignment[i,j] for i in PRODUCERS for j in CONTRACTS"} + ] +} +``` + +### Common Pitfalls +- Using a single, overly large "Big-M" constant for the upper bound link instead of the producer-specific `capacity[i]`, which weakens the formulation. +- Forgetting the lower bound linking constraint, which allows a selected producer to supply less than its minimum required quantity. +- Defining `min_suppliers[j]` as a float parameter instead of an integer, causing type errors in the constraint. + +## Solving stage + +### Strategy Overview +This stage focuses on solving the Pyomo model using the HiGHS or CBC solver via the `pyomo.SolverFactory` interface, with robust status checking and solution validation to ensure reliable results. + +### Step 1 - Configure and Execute Solver +- Instantiate the solver (e.g., `SolverFactory('highs')` or `SolverFactory('cbc')`). +- Set key parameters: `time_limit` for runtime control, `mip_rel_gap` for optimality tolerance, and a `seed` for reproducibility. Avoid setting `threads` if it causes conflicts. +- Call `solver.solve(model, tee=False)` to execute. + +### Step 2 - Validate Solver Status +- Check the solver status (`SolverStatus.ok`) and termination condition (`TerminationCondition.optimal` or `.feasible`). +- If the status is not OK or termination is not optimal/feasible, log the condition and handle the failure (e.g., return empty results, analyze infeasibility). + +### Step 3 - Extract and Verify Solution +- Extract the objective value: `total_cost = pyo.value(model.obj)`. +- Iterate through the decision variables to build dictionaries of allocations (`assignment_quantity[i, j]`) and selections (`binary_assignment[i, j]`), applying a small tolerance (e.g., 1e-6) to determine positivity. +- Programmatically verify key constraints: total allocated demand meets requirements, no producer exceeds capacity, and minimum supplier counts are satisfied. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# ... (model building code as per modeling stage) + +# Solve +solver = pyo.SolverFactory('highs') # or 'cbc' +solver.options['time_limit'] = 30 +solver.options['mip_rel_gap'] = 0.0001 +solver.options['seed'] = 42 + +results = solver.solve(model, tee=False) + +# Validate +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + total_cost = float(pyo.value(model.obj)) + # Extract variable values into dictionaries + allocations = {} + selections = {} + for i in model.PRODUCERS: + for j in model.CONTRACTS: + q_val = pyo.value(model.assignment_quantity[i, j]) + b_val = pyo.value(model.binary_assignment[i, j]) + if q_val > 1e-6: + allocations[(i, j)] = q_val + selections[(i, j)] = b_val + # ... (verification logic) +else: + print(f"Solver failed. Status: {status}, Termination: {term}") + # Handle failure appropriately +``` + +### Common Pitfalls +- Not checking both `SolverStatus` and `TerminationCondition`, leading to misinterpretation of suboptimal or failed solves. +- Extracting variable values without checking if the solve was successful, which may raise errors. +- Using a loose optimality gap (`mip_rel_gap`) when an exact solution is required, potentially missing the true optimum. + +# Workflow 2 (OR-Tools with SCIP/CBC Backend) + +## Modeling stage + +### Strategy Overview +This workflow uses the OR-Tools CP-SAT solver (for MIP) via its Python API, employing a more imperative, constraint-by-constraint building style. It is well-suited for deployment environments where a dedicated algebraic modeling language is not available. + +### Step 1 - Initialize Model and Create Index Mappings +- Create a `cp_model.CpModel()` instance. +- Define lists or ranges for producer and contract indices. Optionally, create dictionaries to map these indices to solver variable objects. + +### Step 2 - Create Decision Variables +- Create continuous (or integer) variables `assignment_quantity[i][j]` with bounds `[0, capacity[i]]` using `model.NewIntVar` or `model.NewNumVar`. +- Create binary variables `binary_assignment[i][j]` using `model.NewBoolVar()`. + +### Step 3 - Define the Objective +- Create a linear expression: `sum(cost[i][j] * assignment_quantity[i][j] for all i, j)`. +- Set the model to minimize this expression using `model.Minimize()`. + +### Step 4 - Add Capacity and Demand Constraints +- For each producer `i`, add a linear constraint: `sum(assignment_quantity[i][j] for j in contracts) <= capacity[i]`. +- For each contract `j`, add a linear constraint: `sum(assignment_quantity[i][j] for i in producers) >= demand[j]`. + +### Step 5 - Enforce Minimum Supplier Count +- For each contract `j`, add a linear constraint: `sum(binary_assignment[i][j] for i in producers) >= min_suppliers[j]`. + +### Step 6 - Link Variables with Conditional Constraints +- For each pair `(i, j)`, add two constraints using the `Add` method: + 1. `assignment_quantity[i][j] >= min_quantity[i] * binary_assignment[i][j]`. + 2. `assignment_quantity[i][j] <= capacity[i] * binary_assignment[i][j]`. This can be implemented by adding an implication: if `binary_assignment[i][j] == 0`, then `assignment_quantity[i][j] == 0`. + +### Formulation Template +```json +{ + "sets": [ + "producers_list", + "contracts_list" + ], + "parameters": { + "cost_matrix": {"type": "list[list[float]]"}, + "capacity_list": {"type": "list[float]"}, + "demand_list": {"type": "list[float]"}, + "min_quantity_list": {"type": "list[float]"}, + "min_suppliers_list": {"type": "list[int]"} + }, + "decision_variables": [ + {"name": "assignment_quantity", "type": "integer_or_continuous", "bounds": "variable", "index": ["i", "j"]}, + {"name": "binary_assignment", "type": "boolean", "index": ["i", "j"]} + ], + "objective": { + "sense": "min", + "expression": "LinearExpr.Sum([assignment_quantity[i][j] * cost_matrix[i][j] for i, j in all_pairs])" + }, + "constraints": [ + {"name": "capacity", "expression": "LinearExpr.Sum([assignment_quantity[i][j] for j in contracts]) <= capacity_list[i] for i in producers"}, + {"name": "demand", "expression": "LinearExpr.Sum([assignment_quantity[i][j] for i in producers]) >= demand_list[j] for j in contracts"}, + {"name": "supplier_count", "expression": "LinearExpr.Sum([binary_assignment[i][j] for i in producers]) >= min_suppliers_list[j] for j in contracts"}, + {"name": "link_lower", "expression": "assignment_quantity[i][j] >= min_quantity_list[i] * binary_assignment[i][j] for i, j in all_pairs"}, + {"name": "link_upper", "expression": "assignment_quantity[i][j] <= capacity_list[i] * binary_assignment[i][j] for i, j in all_pairs"} + ] +} +``` + +### Common Pitfalls +- Using `model.NewIntVar` for large allocation quantities, which may cause integer overflow; prefer `model.NewNumVar` for continuous values. +- Incorrectly implementing the upper bound link as a direct multiplication in OR-Tools, which requires using `AddMultiplicationEquality` or an implication constraint for exact linearization. +- Not scaling the objective coefficients (costs) appropriately, which can lead to numerical issues in the solver. + +## Solving stage + +### Strategy Overview +This stage involves solving the OR-Tools model, configuring solver parameters like time limits and threads, and implementing detailed solution verification to ensure feasibility and optimality. + +### Step 1 - Configure Solver Parameters +- Use `solver.parameters.max_time_in_seconds` to set a time limit. +- Set `solver.parameters.num_search_workers` to control parallel threads. +- Configure optimality tolerances if applicable (e.g., `solver.parameters.relative_gap_limit`). + +### Step 2 - Execute the Solver +- Create a solver instance (e.g., `cp_model.CpSolver()`). +- Call `solver.Solve(model)` to obtain a status code. + +### Step 3 - Check Solution Status +- Check if the status is `OPTIMAL` or `FEASIBLE`. Handle `INFEASIBLE` or `MODEL_INVALID` statuses with appropriate error messages. +- For `FEASIBLE` solutions, note that optimality is not guaranteed. + +### Step 4 - Extract and Validate Results +- If the solve was successful, extract the objective value using `solver.ObjectiveValue()`. +- Iterate through all variable indices, using `solver.Value(variable)` to get the solution for each `assignment_quantity` and `binary_assignment`. +- Store results in dictionaries and perform verification checks: confirm demand satisfaction, capacity adherence, minimum supplier counts, and the linking conditions. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# ... (model building code as per modeling stage) + +# Solve +solver = cp_model.CpSolver() +# Set parameters +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 4 +# solver.parameters.relative_gap_limit = 0.0001 # For optimization problems + +status = solver.Solve(model) + +# Validate and extract +if status in [cp_model.OPTIMAL, cp_model.FEASIBLE]: + total_cost = solver.ObjectiveValue() + allocations = {} + selections = {} + for i in range(num_producers): + for j in range(num_contracts): + q_val = solver.Value(assignment_quantity[i][j]) + b_val = solver.Value(binary_assignment[i][j]) + if q_val > 1e-6: + allocations[(i, j)] = q_val + selections[(i, j)] = b_val + # ... (verification logic) + print(f"Total cost: {total_cost}") +elif status == cp_model.INFEASIBLE: + print("Model is infeasible.") +else: + print(f"Solver returned status: {status}") +``` + +### Common Pitfalls +- Assuming `FEASIBLE` status implies optimality; always check for `OPTIMAL` if a proven optimum is required. +- Not using `solver.Value()` on the correct variable object, leading to extraction errors. +- Setting conflicting solver parameters (e.g., both time limit and iteration limit) without understanding precedence. diff --git a/skills/optskills/skill_library/supply_allocation_profit_maximization.md b/skills/optskills/skill_library/supply_allocation_profit_maximization.md new file mode 100644 index 0000000..56c1ddf --- /dev/null +++ b/skills/optskills/skill_library/supply_allocation_profit_maximization.md @@ -0,0 +1,282 @@ +--- +name: Supply Allocation Profit Maximization +description: | + Model and solve linear programming problems that allocate supply quantities across entities to maximize profit while satisfying demand equality and non-negativity constraints. +--- + +# Workflow 1 (OR-Tools LP Solver) + +## Modeling stage + +### Strategy Overview +Use Google OR-Tools' pywraplp module to formulate a linear program with continuous decision variables representing supply quantities. Encode profit maximization as a linear objective, demand satisfaction as equality constraints, and enforce non-negativity through variable bounds. + +### Step 1 - Define Data Structures +- Create dictionaries to store profit coefficients keyed by tuples of entity combinations (e.g., `(source, product, destination)`). +- Create dictionaries to store demand requirements keyed by tuples of product and destination (e.g., `(product, destination)`). +- Define lists of all entity identifiers (sources, products, destinations) for iteration. + +### Step 2 - Create Solver and Variables +- Instantiate solver with `pywraplp.Solver.CreateSolver("GLOP")` for continuous LP problems. +- For each combination of entities, create a decision variable using `solver.NumVar(0, solver.infinity(), name)` to represent the supply quantity. + +### Step 3 - Add Demand Constraints +- For each product-destination pair, create an equality constraint using `solver.Constraint(demand, demand)`. +- Add coefficients to the constraint by summing the appropriate decision variables across all sources. + +### Step 4 - Build Objective Function +- Create an objective object with `solver.Objective()`. +- Set each variable's coefficient using `objective.SetCoefficient(variable, profit_coefficient)`. +- Set the direction to maximization with `objective.SetMaximization()`. + +### Formulation Template +```json +{ + "sets": ["SOURCES", "PRODUCTS", "DESTINATIONS"], + "parameters": [ + {"name": "profit", "index": ["SOURCE", "PRODUCT", "DESTINATION"], "type": "float"}, + {"name": "demand", "index": ["PRODUCT", "DESTINATION"], "type": "float"} + ], + "decision_variables": [ + {"name": "x", "index": ["SOURCE", "PRODUCT", "DESTINATION"], "type": "continuous", "lower_bound": 0} + ], + "objective": { + "sense": "max", + "expression": "sum(profit[s,p,d] * x[s,p,d] for all s,p,d)" + }, + "constraints": [ + {"name": "demand_satisfaction", "index": ["PRODUCT", "DESTINATION"], "type": "equality", "expression": "sum(x[s,p,d] for all s) == demand[p,d]"} + ] +} +``` + +### Common Pitfalls +- Forgetting to set the objective direction to maximization (default is minimization). +- Using `NumVar` without specifying lower bound 0, which defaults to negative infinity. +- Mismatching variable indices when adding coefficients to constraints. + +## Solving stage + +### Strategy Overview +Solve the LP using GLOP solver, verify optimality status, extract solution values, and output results in a structured JSON format for automated parsing. + +### Step 1 - Solve and Check Status +- Call `solver.Solve()` and capture the status. +- Check if `status == pywraplp.Solver.OPTIMAL` before proceeding to read results. + +### Step 2 - Extract Solution Values +- Retrieve the objective value via `solver.Objective().Value()`. +- Iterate over all decision variables and get their values using `variable.solution_value()`. +- Round values to a reasonable precision (e.g., 4 decimal places) for output. + +### Step 3 - Verify Solution Consistency +- After extraction, verify that all demand constraints are satisfied by summing supplies per product-destination and comparing to the original demand data. +- Recompute the objective value manually from the solution to confirm consistency with the solver-reported value. + +### Step 4 - Format and Output Results +- Build a dictionary with keys "status", "objective", and "solution". +- Output the dictionary as a JSON string prefixed with `RESULT_JSON:` for reliable parsing. + +### Code Usage +```python +import pywraplp +import json + +# Data preparation +sources = ["source_1", "source_2"] +products = ["product_1", "product_2"] +destinations = ["dest_1", "dest_2"] + +profit = {("source_1", "product_1", "dest_1"): 10.0, ...} +demand = {("product_1", "dest_1"): 100.0, ...} + +# Create solver +solver = pywraplp.Solver.CreateSolver("GLOP") + +# Decision variables +x = {} +for s in sources: + for p in products: + for d in destinations: + x[(s, p, d)] = solver.NumVar(0, solver.infinity(), f"x_{s}_{p}_{d}") + +# Demand constraints +for p in products: + for d in destinations: + constraint = solver.Constraint(demand[(p, d)], demand[(p, d)]) + for s in sources: + constraint.SetCoefficient(x[(s, p, d)], 1) + +# Objective +objective = solver.Objective() +for (s, p, d), coeff in profit.items(): + objective.SetCoefficient(x[(s, p, d)], coeff) +objective.SetMaximization() + +# Solve +status = solver.Solve() + +# Extract results +if status == pywraplp.Solver.OPTIMAL: + result = { + "status": "OPTIMAL", + "objective": round(objective.Value(), 4), + "solution": {str(k): round(v.solution_value(), 4) for k, v in x.items()} + } +else: + result = {"status": "NOT_OPTIMAL", "solver_status": status} + +print(f"RESULT_JSON:{json.dumps(result)}") +``` + +### Common Pitfalls +- Not checking solver status before accessing solution values, which can cause runtime errors. +- Using `solver.infinity()` incorrectly in variable bounds (should be method call). +- Forgetting to convert dictionary keys to strings for JSON serialization. + +# Workflow 2 (Pyomo with HiGHS Solver) + +## Modeling stage + +### Strategy Overview +Use Pyomo's algebraic modeling language to define sets, parameters, decision variables, constraints, and objective in a declarative manner. Leverage Pyomo's rule-based constraint and objective definitions for clean separation of model logic. + +### Step 1 - Define Index Sets +- Create Pyomo Set objects for each entity category using `pyo.Set(initialize=list_of_items)`. +- Use descriptive set names (e.g., `model.SOURCES`, `model.PRODUCTS`, `model.DESTINATIONS`). + +### Step 2 - Declare Parameters +- Define profit and demand as Pyomo Param objects indexed over the appropriate sets. +- Initialize parameters from dictionaries using `pyo.Param(model.SOURCES, model.PRODUCTS, model.DESTINATIONS, initialize=profit_dict, within=pyo.Reals)`. + +### Step 3 - Create Decision Variables +- Define a continuous non-negative variable indexed over all sets: `pyo.Var(model.SOURCES, model.PRODUCTS, model.DESTINATIONS, domain=pyo.NonNegativeReals)`. + +### Step 4 - Formulate Constraints +- Write a constraint rule function that takes the model and index values, sums over the source index, and returns an equality expression. +- Apply the constraint over product and destination sets: `pyo.Constraint(model.PRODUCTS, model.DESTINATIONS, rule=demand_rule)`. + +### Step 5 - Build Objective +- Define the objective as a sum expression: `sum(model.profit[s,p,d] * model.x[s,p,d] for s in model.SOURCES for p in model.PRODUCTS for d in model.DESTINATIONS)`. +- Set the sense to `pyo.maximize`. + +### Formulation Template +```json +{ + "sets": ["SOURCES", "PRODUCTS", "DESTINATIONS"], + "parameters": [ + {"name": "profit", "index": ["SOURCES", "PRODUCTS", "DESTINATIONS"], "type": "float"}, + {"name": "demand", "index": ["PRODUCTS", "DESTINATIONS"], "type": "float"} + ], + "decision_variables": [ + {"name": "x", "index": ["SOURCES", "PRODUCTS", "DESTINATIONS"], "domain": "NonNegativeReals"} + ], + "objective": { + "sense": "maximize", + "expression": "sum(profit[s,p,d] * x[s,p,d] for all s,p,d)" + }, + "constraints": [ + {"name": "demand_satisfaction", "index": ["PRODUCTS", "DESTINATIONS"], "type": "equality", "expression": "sum(x[s,p,d] for all s) == demand[p,d]"} + ] +} +``` + +### Common Pitfalls +- Variable name conflicts between model object and loop variables in generator expressions (use distinct names like `mod` for model parameter in rule functions). +- Forgetting to import `pyo.environ` or using incorrect module references. +- Not specifying `within=pyo.NonNegativeReals` for parameters that should be non-negative. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the HiGHS solver, check termination conditions, extract solution values, and output results in a structured JSON format with proper error handling. + +### Step 1 - Configure and Run Solver +- Instantiate solver with `pyo.SolverFactory("highs")`. +- Set solver options like `time_limit` and `threads` for performance control. +- Call `solver.solve(model, tee=False)` and capture the results object. + +### Step 2 - Check Solver Status +- Verify `results.solver.status == SolverStatus.ok` and `results.solver.termination_condition in {TerminationCondition.optimal, TerminationCondition.feasible}`. +- For infeasible or error cases, output a failure payload with solver status and termination condition. + +### Step 3 - Extract Solution Values +- Get the objective value using `float(pyo.value(model.obj))`. +- Iterate over all variable indices and extract values using `float(pyo.value(model.x[s, p, d]))`. +- Collect results into a dictionary with descriptive keys. + +### Step 4 - Verify Solution Consistency +- After extraction, verify that all demand constraints are satisfied by summing supplies per product-destination and comparing to the original demand data. +- Recompute the objective value manually from the solution to confirm consistency with the solver-reported value. + +### Step 5 - Format and Output Results +- Build a result dictionary with keys "status", "objective", and "solution". +- Output as JSON string prefixed with `RESULT_JSON:` for automated parsing. + +### Code Usage +```python +import pyomo.environ as pyo +import json + +# Data preparation +sources = ["source_1", "source_2"] +products = ["product_1", "product_2"] +destinations = ["dest_1", "dest_2"] + +profit_data = {("source_1", "product_1", "dest_1"): 10.0, ...} +demand_data = {("product_1", "dest_1"): 100.0, ...} + +# Build model +model = pyo.ConcreteModel() +model.SOURCES = pyo.Set(initialize=sources) +model.PRODUCTS = pyo.Set(initialize=products) +model.DESTINATIONS = pyo.Set(initialize=destinations) + +model.profit = pyo.Param(model.SOURCES, model.PRODUCTS, model.DESTINATIONS, initialize=profit_data, within=pyo.Reals) +model.demand = pyo.Param(model.PRODUCTS, model.DESTINATIONS, initialize=demand_data, within=pyo.NonNegativeReals) + +model.x = pyo.Var(model.SOURCES, model.PRODUCTS, model.DESTINATIONS, domain=pyo.NonNegativeReals) + +def demand_rule(mod, p, d): + return sum(mod.x[s, p, d] for s in mod.SOURCES) == mod.demand[p, d] + +model.demand_constraint = pyo.Constraint(model.PRODUCTS, model.DESTINATIONS, rule=demand_rule) + +def obj_rule(mod): + return sum(mod.profit[s, p, d] * mod.x[s, p, d] for s in mod.SOURCES for p in mod.PRODUCTS for d in mod.DESTINATIONS) + +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.maximize) + +# Solve +solver = pyo.SolverFactory("highs") +solver.options["time_limit"] = 60 +solver.options["threads"] = 4 +results = solver.solve(model, tee=False) + +# Extract results +if results.solver.status == pyo.SolverStatus.ok and results.solver.termination_condition in {pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible}: + solution = {} + for s in model.SOURCES: + for p in model.PRODUCTS: + for d in model.DESTINATIONS: + solution[f"x_{s}_{p}_{d}"] = round(float(pyo.value(model.x[s, p, d])), 4) + + result = { + "status": str(results.solver.termination_condition), + "objective": round(float(pyo.value(model.obj)), 4), + "solution": solution + } +else: + result = { + "status": "FAILURE", + "solver_status": str(results.solver.status), + "termination_condition": str(results.solver.termination_condition) + } + +print(f"RESULT_JSON:{json.dumps(result)}") +``` + +### Common Pitfalls +- Not checking termination condition for feasibility when optimal is not achieved. +- Using `pyo.value()` on uninitialized variables after a failed solve. +- Forgetting to convert Pyomo numeric values to Python floats for JSON serialization. diff --git a/skills/optskills/skill_library/time_constrained_routing_with_binary_decisions_and_arrival_time_variables.md b/skills/optskills/skill_library/time_constrained_routing_with_binary_decisions_and_arrival_time_variables.md new file mode 100644 index 0000000..6507ac4 --- /dev/null +++ b/skills/optskills/skill_library/time_constrained_routing_with_binary_decisions_and_arrival_time_variables.md @@ -0,0 +1,343 @@ +--- +name: Time-Constrained Routing with Binary Decisions and Arrival Time Variables +description: | + Models and solves a single-vehicle routing problem with time windows using either a MILP formulation with MTZ constraints or a CP-SAT formulation with circuit and time propagation, minimizing total travel time. +--- + +# Workflow 1 (MILP with MTZ Constraints) + +## Modeling stage + +### Strategy Overview +Formulate the problem as a Mixed-Integer Linear Program using binary arc variables and continuous arrival time variables. Connectivity is enforced via flow conservation, and time window feasibility is ensured through Miller-Tucker-Zemlin (MTZ) constraints that propagate arrival times along selected arcs. + +### Step 1 - Define Sets and Parameters +- Define the set of nodes including the depot (index 0) and customer nodes. +- Define the travel time matrix `dist[i][j]` for all node pairs. +- Define time window bounds `a[i]` (earliest) and `b[i]` (latest) for each node. +- Compute a sufficiently large constant `M = max(b) + max(dist)` for MTZ constraints. + +### Step 2 - Create Decision Variables +- Create binary variables `x[i][j]` for each directed arc `(i,j)`, indicating whether the vehicle travels directly from node `i` to node `j`. +- Create continuous variables `t[i]` representing the arrival time at each node. + +### Step 3 - Add Flow Conservation Constraints +- For each node `j` (including depot), enforce exactly one incoming arc: `sum_i x[i][j] == 1`. +- For each node `i` (including depot), enforce exactly one outgoing arc: `sum_j x[i][j] == 1`. + +### Step 4 - Add Time Window Constraints +- Bound arrival times: `a[i] <= t[i] <= b[i]` for all nodes `i`. + +### Step 5 - Add Travel Time Propagation (MTZ) +- For each arc `(i,j)`, add: `t[j] >= t[i] + dist[i][j] - M * (1 - x[i][j])`, where `M` is a sufficiently large number (e.g., `max(b) + max(dist)`). + +### Step 6 - Fix Depot Start Time +- Set `t[0] == 0` to fix departure from the depot. + +### Formulation Template +```json +{ + "sets": ["N: set of nodes (0 = depot, 1..n = customers)"], + "parameters": [ + "dist[i][j]: travel time from i to j", + "a[i]: earliest arrival time at node i", + "b[i]: latest arrival time at node i", + "M: large constant, e.g., max(b) + max(dist)" + ], + "decision_variables": [ + "x[i][j] ∈ {0,1}: 1 if vehicle travels from i to j", + "t[i] ≥ 0: arrival time at node i" + ], + "objective": { + "sense": "min", + "expression": "sum_{i,j} dist[i][j] * x[i][j]" + }, + "constraints": [ + "sum_{i} x[i][j] == 1 for all j ∈ N", + "sum_{j} x[i][j] == 1 for all i ∈ N", + "a[i] <= t[i] <= b[i] for all i ∈ N", + "t[j] >= t[i] + dist[i][j] - M * (1 - x[i][j]) for all i,j ∈ N", + "t[0] == 0" + ] +} +``` + +### Common Pitfalls +- Using a too-small `M` value can incorrectly bind constraints when `x[i][j]=0`, causing infeasibility. Always set `M` to at least `max(b) + max(dist)`. +- Forgetting to include the depot in flow conservation constraints (both incoming and outgoing) will break tour completion. +- Using strict inequality constraints (e.g., `t[i] > a[i]`) is not supported; use non-strict bounds and rely on integer domains. + +## Solving stage + +### Strategy Overview +Use a MILP solver (e.g., Gurobi, CPLEX) with explicit parameter control. Solve to optimality with a MIP gap of zero, then extract the route and verify arrival times. + +### Step 1 - Initialize Solver and Set Parameters +- Create a solver instance (e.g., `gp.Model()` for Gurobi). +- Set parameters: `TimeLimit = [TIME_LIMIT]`, `MIPGap = 0.0`, `Threads`, `Seed` for reproducibility. + +### Step 2 - Build Model from Formulation +- Add decision variables with appropriate types and bounds. +- Add constraints using the formulation template. +- Set the objective function. + +### Step 3 - Solve and Check Status +- Call `optimize()`. +- Check solver status: if `GRB.OPTIMAL` or `GRB.TIME_LIMIT` with a feasible solution (`model.SolCount > 0`), proceed. Otherwise, report infeasibility or error. + +### Step 4 - Extract Solution +- Retrieve the route by following `x[i][j] > 0.5` starting from the depot. +- Retrieve arrival times `t[i]` and verify they satisfy time windows. +- Compute total travel time from the objective value. + +### Code Usage +```python +import gurobipy as gp + +# --- Data (placeholders) --- +N = list(range(num_nodes)) +dist = [[...] for _ in N] +a = [...] +b = [...] +M = max(b) + max(max(row) for row in dist) + +# --- Model --- +model = gp.Model("VRPTW_MTZ") +model.Params.TimeLimit = [TIME_LIMIT] +model.Params.MIPGap = 0.0 +model.Params.Threads = 4 +model.Params.Seed = 42 + +# Variables +x = model.addVars(N, N, vtype=gp.GRB.BINARY, name="x") +t = model.addVars(N, vtype=gp.GRB.CONTINUOUS, lb=0, name="t") + +# Flow conservation +for j in N: + model.addConstr(gp.quicksum(x[i, j] for i in N) == 1, name=f"in_{j}") +for i in N: + model.addConstr(gp.quicksum(x[i, j] for j in N) == 1, name=f"out_{i}") + +# Time windows +for i in N: + model.addConstr(t[i] >= a[i], name=f"tw_lb_{i}") + model.addConstr(t[i] <= b[i], name=f"tw_ub_{i}") + +# MTZ constraints +for i in N: + for j in N: + if i != j: + model.addConstr(t[j] >= t[i] + dist[i][j] - M * (1 - x[i, j]), name=f"mtz_{i}_{j}") + +# Depot start +model.addConstr(t[0] == 0, name="depot_start") + +# Objective +model.setObjective(gp.quicksum(dist[i][j] * x[i, j] for i in N for j in N), gp.GRB.MINIMIZE) + +# Solve +model.optimize() + +# --- Result parsing --- +if model.status == gp.GRB.OPTIMAL or (model.status == gp.GRB.TIME_LIMIT and model.SolCount > 0): + route = [0] + current = 0 + while True: + next_node = [j for j in N if x[current, j].X > 0.5][0] + if next_node == 0: + break + route.append(next_node) + current = next_node + arrival_times = [t[i].X for i in route] + print(f"RESULT:{model.ObjVal}") +else: + print('{"status": "infeasible", "error": "No feasible solution found"}') +``` + +### Common Pitfalls +- Not checking `SolCount > 0` when status is `TIME_LIMIT`; the solver may have no feasible solution yet. +- Assuming the solver will find a solution quickly; always set a reasonable time limit and handle timeouts gracefully. +- Using `x[i,j].X` without verifying the variable is in the solution; check `model.SolCount` first. + +# Workflow 2 (CP-SAT with Circuit Constraint) + +## Modeling stage + +### Strategy Overview +Formulate the problem using Google OR-Tools CP-SAT solver with a circuit constraint to enforce a single tour, and a time dimension to handle arrival times and waiting. The solver natively supports time windows and slack (waiting) variables. + +### Step 1 - Define Nodes and Travel Times +- Create a list of nodes including the depot (index 0). +- Define a travel time callback function that returns `dist[i][j]` for any node pair. + +### Step 2 - Create Routing Model and Parameters +- Instantiate `RoutingIndexManager` with the number of nodes and 1 vehicle. +- Set the depot index to 0. +- Register the travel time callback with `RegisterTransitCallback`. + +### Step 3 - Add Time Dimension with Slack +- Add a time dimension using `AddDimension` with: + - `slack_max` set to a large value (e.g., `horizon`) to allow waiting. + - `capacity` set to the maximum time horizon (e.g., `max(b)`). + - `fix_start_cumul_to_zero=True` to fix depot departure to 0. + +### Step 4 - Set Time Window Constraints +- For each node, call `time_dimension.CumulVar(index).SetRange(a[i], b[i])`. +- For the depot, set the start cumul to `[0, 0]` and the end cumul to `[a[0], b[0]]`. + +### Step 5 - Set Objective and Add Finalizers +- Set the arc cost evaluator to the travel time callback. +- Set the objective to minimize total travel time. +- Add `routing.AddVariableMinimizedByFinalizer(time_dimension.CumulVar(index))` for all nodes to encourage earlier arrivals and reduce waiting time. + +### Formulation Template +```json +{ + "sets": ["N: set of nodes (0 = depot, 1..n = customers)"], + "parameters": [ + "dist[i][j]: travel time from i to j", + "a[i]: earliest arrival time at node i", + "b[i]: latest arrival time at node i", + "horizon: max(b) + max(dist)" + ], + "decision_variables": [ + "NextVar[i]: next node visited after i (implicit circuit)", + "CumulVar[i]: arrival time at node i (includes waiting)" + ], + "objective": { + "sense": "min", + "expression": "sum of dist[i][j] over selected arcs" + }, + "constraints": [ + "Circuit constraint: each node has exactly one successor and one predecessor", + "CumulVar[i] ∈ [a[i], b[i]] for all i ∈ N", + "CumulVar[0] == 0 at start", + "CumulVar[j] >= CumulVar[i] + dist[i][j] when NextVar[i] == j" + ] +} +``` + +### Common Pitfalls +- Setting `slack_max = 0` prevents waiting, which can make the model infeasible if time windows require early arrival. Always allow slack. +- Confusing physical arrival time with service start time; the cumul variable represents service start (including waiting), not physical arrival. +- Not setting `fix_start_cumul_to_zero=True` and then manually setting the depot cumul range inconsistently (e.g., `SetRange(0, 0)` without fixing start). +- Not adding `AddVariableMinimizedByFinalizer` for time window nodes; the solver may not properly enforce cumul bounds during local search. + +## Solving stage + +### Strategy Overview +Use OR-Tools routing solver with multiple first solution strategies and local search metaheuristics. Parse the solution by reading cumul variables directly from the solver to get accurate arrival times including waiting. + +### Step 1 - Set Search Parameters +- Create `DefaultRoutingSearchParameters`. +- Set `first_solution_strategy` to `PATH_CHEAPEST_ARC` initially. +- Set `local_search_metaheuristic` to `GUIDED_LOCAL_SEARCH`. +- Set `time_limit` to a reasonable value (e.g., `[TIME_LIMIT]` seconds). +- Enable logging if debugging. + +### Step 2 - Solve with Fallback Strategies +- Call `SolveWithParameters`. +- If status is not `ROUTING_SUCCESS`, try alternative first solution strategies in order: `SAVINGS`, `SWEEP`, `CHRISTOFIDES`, `BEST_INSERTION`. + +### Step 3 - Extract and Verify Solution +- Use `solution.Value(time_dimension.CumulVar(index))` to get arrival times for each node. +- Use `solution.Value(routing.NextVar(index))` to reconstruct the route. +- Verify arrival times against time windows. +- Compute total travel time as the sum of arc distances along the route, not the solver-reported objective which may include waiting time. + +### Code Usage +```python +from ortools.constraint_solver import routing_enums_pb2, pywrapcp + +# --- Data (placeholders) --- +num_nodes = len(a) +dist = [[...] for _ in range(num_nodes)] +a = [...] +b = [...] +horizon = max(b) + max(max(row) for row in dist) + +# --- Model --- +manager = pywrapcp.RoutingIndexManager(num_nodes, 1, 0) +routing = pywrapcp.RoutingModel(manager) + +def travel_time_callback(from_index, to_index): + from_node = manager.IndexToNode(from_index) + to_node = manager.IndexToNode(to_index) + return dist[from_node][to_node] + +transit_callback_index = routing.RegisterTransitCallback(travel_time_callback) +routing.SetArcCostEvaluatorOfAllVehicles(transit_callback_index) + +# Time dimension +time_dimension_name = 'Time' +routing.AddDimension( + transit_callback_index, + horizon, # slack_max (allow waiting) + horizon, # capacity (max time) + True, # fix_start_cumul_to_zero + time_dimension_name +) +time_dimension = routing.GetDimensionOrDie(time_dimension_name) + +# Time windows +for node in range(num_nodes): + index = manager.NodeToIndex(node) + time_dimension.CumulVar(index).SetRange(a[node], b[node]) + +# Depot start/end +depot_index = manager.NodeToIndex(0) +time_dimension.CumulVar(depot_index).SetRange(0, 0) # start at 0 +routing.AddVariableMinimizedByFinalizer(time_dimension.CumulVar(depot_index)) + +# Add finalizers for all nodes +for node in range(num_nodes): + index = manager.NodeToIndex(node) + routing.AddVariableMinimizedByFinalizer(time_dimension.CumulVar(index)) + +# --- Solve --- +search_parameters = pywrapcp.DefaultRoutingSearchParameters() +search_parameters.first_solution_strategy = ( + routing_enums_pb2.FirstSolutionStrategy.PATH_CHEAPEST_ARC) +search_parameters.local_search_metaheuristic = ( + routing_enums_pb2.LocalSearchMetaheuristic.GUIDED_LOCAL_SEARCH) +search_parameters.time_limit.seconds = [TIME_LIMIT] +search_parameters.log_search = False + +solution = routing.SolveWithParameters(search_parameters) + +# Fallback strategies +if not solution: + for strategy in [ + routing_enums_pb2.FirstSolutionStrategy.SAVINGS, + routing_enums_pb2.FirstSolutionStrategy.SWEEP, + routing_enums_pb2.FirstSolutionStrategy.CHRISTOFIDES, + routing_enums_pb2.FirstSolutionStrategy.BEST_INSERTION, + ]: + search_parameters.first_solution_strategy = strategy + solution = routing.SolveWithParameters(search_parameters) + if solution: + break + +# --- Result parsing --- +if solution: + route = [] + index = routing.Start(0) + total_time = 0 + while not routing.IsEnd(index): + node = manager.IndexToNode(index) + route.append(node) + arrival = solution.Value(time_dimension.CumulVar(index)) + next_index = solution.Value(routing.NextVar(index)) + if not routing.IsEnd(next_index): + total_time += dist[node][manager.IndexToNode(next_index)] + index = next_index + route.append(0) # return to depot + print(f"RESULT:{total_time}") +else: + print('{"status": "infeasible", "error": "No feasible solution found"}') +``` + +### Common Pitfalls +- Assuming a single first solution strategy is sufficient; always try multiple strategies before concluding infeasibility. +- Manually summing travel times for validation instead of reading cumul variables; cumul values include waiting and are the actual constrained values. +- Not verifying solution feasibility by cross-checking arrival times against time windows. diff --git a/skills/optskills/skill_library/transportation_flow_optimizer.md b/skills/optskills/skill_library/transportation_flow_optimizer.md new file mode 100644 index 0000000..5325802 --- /dev/null +++ b/skills/optskills/skill_library/transportation_flow_optimizer.md @@ -0,0 +1,201 @@ +--- +name: Transportation Flow Optimizer +description: | + Model and solve capacitated transportation problems with supply limits, demand satisfaction, and arc capacity constraints to minimize total flow cost. + +--- +# Workflow 1 (Direct Solver API) + +## Modeling stage + +### Strategy Overview +Formulate the problem directly within a solver's native API (e.g., OR-Tools) by defining variables with bounds, adding constraints via summation, and setting a linear objective. This approach is procedural and tightly couples the model to the solver's construction methods. + +### Step 1 - Define Problem Dimensions and Data +- Identify the number of origins (`num_origins`) and destinations (`num_dests`). +- Organize input data into 2D lists/arrays for `cost` and `capacity`, and 1D lists for `supply` and `demand`. Ensure all dimensions align. +- **Check Problem Balance**: Verify total supply equals total demand. If not balanced, introduce dummy nodes or adjust constraints accordingly. + +### Step 2 - Create Solver and Decision Variables +- Instantiate the solver (e.g., `solver = pywraplp.Solver.CreateSolver('GLOP')`). +- Create a 2D array of continuous decision variables `flow[i][j]`. Set the lower bound to 0 and the upper bound to `capacity[i][j]` directly in the variable creation to implicitly handle arc capacities. + +### Step 3 - Add Supply and Demand Constraints +- For each origin `i`, add a constraint: `sum(flow[i][j] for j in destinations) == supply[i]`. +- For each destination `j`, add a constraint: `sum(flow[i][j] for i in origins) == demand[j]`. + +### Step 4 - Set the Objective Function +- Define the objective as the sum of `flow[i][j] * cost[i][j]` across all arcs. +- Set the objective for minimization using the solver's method (e.g., `solver.Minimize(objective_expression)`). + +### Formulation Template +```json +{ + "sets": ["origins", "destinations"], + "parameters": ["supply[origins]", "demand[destinations]", "cost[origins][destinations]", "capacity[origins][destinations]"], + "decision_variables": ["flow[origins][destinations]"], + "objective": { + "sense": "min", + "expression": "sum(cost[i][j] * flow[i][j] for i in origins for j in destinations)" + }, + "constraints": [ + "sum(flow[i][j] for j in destinations) == supply[i] for all i in origins", + "sum(flow[i][j] for i in origins) == demand[j] for all j in destinations", + "0 <= flow[i][j] <= capacity[i][j] for all i in origins, j in destinations" + ] +} +``` + +### Common Pitfalls +- Forgetting to enforce non-negativity; ensure variable lower bounds are set to 0. +- Mismatching indices between supply/demand lists and the variable matrix, causing constraint errors. +- Using `<=` for supply/demand constraints when exact fulfillment (`==`) is required for a balanced problem. + +## Solving stage + +### Strategy Overview +Solve the constructed model using the solver's native methods, check the solution status, extract results, and perform post-solution validation to ensure all constraints are satisfied within a tolerance. + +### Step 1 - Invoke the Solver +- Call the solver's `Solve()` method. +- Store the returned status code (e.g., `OPTIMAL`, `FEASIBLE`, `INFEASIBLE`). + +### Step 2 - Check Solution Status and Extract Results +- If status is `OPTIMAL` or `FEASIBLE`, retrieve the objective value. +- Iterate through all `flow` variables to collect their `solution_value()`. +- If the status indicates infeasibility or error, exit with a structured error message. + +### Step 3 - Validate the Solution +- Recalculate total flow from each origin and into each destination. +- Verify these values are within tolerance of the `supply` (`==`) and `demand` (`==`) parameters. +- Check that no flow exceeds its `capacity`. + +### Step 4 - Report the Solution +- Print the objective value in a parseable format: `RESULT:{objective_value}`. +- Optionally, output non-zero flows or a summary of constraint satisfaction for debugging. + +### Code Usage +```python +# build model from formulation +solver = pywraplp.Solver.CreateSolver('GLOP') +flow = [[solver.NumVar(0, capacity[i][j], f'flow_{i}_{j}') for j in range(num_dests)] for i in range(num_origins)] +# Add supply & demand constraints +for i in range(num_origins): + solver.Add(sum(flow[i][j] for j in range(num_dests)) == supply[i]) +for j in range(num_dests): + solver.Add(sum(flow[i][j] for i in range(num_origins)) == demand[j]) +# Set objective +objective = solver.Objective() +for i in range(num_origins): + for j in range(num_dests): + objective.SetCoefficient(flow[i][j], cost[i][j]) +objective.SetMinimization() + +# solve with status / termination checks +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + print(f'RESULT:{objective.Value()}') +else: + print(f'{{"status": "FAILED", "reason": "Solver returned status {status}"}}') +``` + +### Common Pitfalls +- Not checking for `FEASIBLE` status in addition to `OPTIMAL`, potentially missing valid solutions. +- Failing to handle solver instantiation errors (e.g., if `GLOP` is not available). +- Assuming variable values exist without checking the solve status first. + +# Workflow 2 (Modeling Language with Solver Factory) + +## Modeling stage + +### Strategy Overview +Use a modeling abstraction (e.g., Pyomo) to declaratively define sets, parameters, variables, and constraints. This separates the problem formulation from the solver interface, allowing for solver agnosticism and easier model inspection. + +### Step 1 - Declare Abstract Model Components +- Define Pyomo `Set` components for `origins` and `destinations`. +- Define `Param` components for `supply`, `demand`, `cost`, and `capacity`, indexed by the appropriate sets. +- **Check Problem Balance**: Verify total supply equals total demand. If not balanced, introduce dummy nodes or adjust constraints accordingly. + +### Step 2 - Define Decision Variables and Bounds +- Define a `Var` component `flow`, indexed over `(origins, destinations)`, within `pyo.NonNegativeReals`. +- Apply upper bounds (`flow[i,j].ub = capacity[i,j]`) to enforce arc capacities, either during variable creation or via a rule. + +### Step 3 - Construct Constraints Declaratively +- Define a `ConstraintList` or use rule-based `Constraint` components. +- Add supply limit constraint: `sum(flow[i,j] for j in destinations) == supply[i]`. +- Add demand satisfaction constraint: `sum(flow[i,j] for i in origins) == demand[j]`. + +### Step 4 - Define the Objective Function +- Define an `Objective` rule to minimize `sum(cost[i,j] * flow[i,j] for i in origins for j in destinations)`. + +### Formulation Template +```json +{ + "sets": ["origins", "destinations"], + "parameters": ["supply[origins]", "demand[destinations]", "cost[origins][destinations]", "capacity[origins][destinations]"], + "decision_variables": ["flow[origins][destinations]"], + "objective": { + "sense": "min", + "expression": "sum(cost[i][j] * flow[i][j] for i in origins for j in destinations)" + }, + "constraints": [ + "sum(flow[i][j] for j in destinations) == supply[i] for all i in origins", + "sum(flow[i][j] for i in origins) == demand[j] for all j in destinations", + "flow[i][j] <= capacity[i][j] for all i in origins, j in destinations" + ] +} +``` + +### Common Pitfalls +- Confusing Pyomo's `AbstractModel` with `ConcreteModel`; choose based on whether data is provided at model creation or later. +- Incorrectly indexing parameters within constraint rules, leading to `KeyError`. +- Forgetting to set the `sense` (minimize/maximize) on the objective. + +## Solving stage + +### Strategy Overview +Use a solver factory to interface with different solvers (e.g., HiGHS, CBC). Configure solver options, solve the model, and then programmatically interrogate the solver results and model instance to extract and validate the solution. + +### Step 1 - Select and Configure the Solver +- Use `SolverFactory('solver_name')` to create a solver interface. +- Set options like time limit (`sec`), optimality gap (`ratio`), and number of threads (`threads`) if applicable. + +### Step 2 - Solve and Check Termination Status +- Execute `solver.solve(model)`. +- Check the solver status (`model.solver.status`) and termination condition (`model.solver.termination_condition`) to confirm optimality or feasibility. + +### Step 3 - Extract and Validate Solution +- If solve was successful, retrieve the objective value via `pyo.value(model.obj)`. +- Iterate through the `flow` variable to collect its `value`. +- Perform post-solve validation: recalculate sums and compare against `supply` and `demand` with a tolerance (e.g., `1e-6`). + +### Step 4 - Output and Error Handling +- Output the objective value as `RESULT:{objective_value}`. +- If the solve failed, output a JSON object containing the solver status and termination condition for diagnostics. + +### Code Usage +```python +# build model from formulation +model = pyo.ConcreteModel() +model.origins = pyo.Set(initialize=origins_list) +model.dests = pyo.Set(initialize=dests_list) +model.flow = pyo.Var(model.origins, model.dests, within=pyo.NonNegativeReals, bounds=lambda m, i, j: (0, capacity[i][j])) +# Define constraints +model.supply_con = pyo.Constraint(model.origins, rule=lambda m, i: sum(m.flow[i,j] for j in m.dests) == supply[i]) +model.demand_con = pyo.Constraint(model.dests, rule=lambda m, j: sum(m.flow[i,j] for i in m.origins) == demand[j]) +# Define objective +model.obj = pyo.Objective(expr=sum(cost[i][j] * model.flow[i,j] for i in model.origins for j in model.dests), sense=pyo.minimize) + +# solve with status / termination checks +solver = SolverFactory('highs') +results = solver.solve(model) +if model.solver.termination_condition == pyo.TerminationCondition.optimal: + print(f'RESULT:{pyo.value(model.obj)}') +else: + print(f'{{"status": "{model.solver.status}", "termination_condition": "{model.solver.termination_condition}"}}') +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition`; a status of `ok` does not guarantee optimality. +- Attempting to access variable values from an unsolved or infeasible model, which may raise an error. +- Misconfiguring solver options specific to the chosen solver (e.g., using `timeLimit` for one solver vs. `sec` for another). diff --git a/skills/optskills/skill_library/transportationproblemsolver.md b/skills/optskills/skill_library/transportationproblemsolver.md new file mode 100644 index 0000000..9517da9 --- /dev/null +++ b/skills/optskills/skill_library/transportationproblemsolver.md @@ -0,0 +1,257 @@ +--- +name: TransportationProblemSolver +description: | + Model and solve balanced or capacitated transportation problems with linear costs using either direct LP solver APIs or algebraic modeling frameworks. +--- + +# Workflow 1 (Direct LP Solver API) + +## Modeling stage + +### Strategy Overview +Formulate the transportation problem directly within a solver's API (e.g., OR-Tools GLOP) by creating variables, constraints, and objective using procedural loops. This approach is efficient for standard problems and offers fine-grained control over variable bounds and constraint coefficients. + +### Step 1 - Define Data Structures +- Organize problem data into lists or arrays for supply, demand, cost, and optional capacity. +- Use consistent indexing: `origins = range(num_origins)`, `destinations = range(num_destinations)`. +- Verify that total supply equals total demand for a balanced problem. + +### Step 2 - Create Flow Variables +- Instantiate non-negative continuous variables for each origin-destination pair. +- Use `solver.NumVar(lower_bound, upper_bound, name)`. +- For capacitated problems, set the upper bound to `capacity[i][j]`; otherwise, use `solver.infinity()`. + +### Step 3 - Add Supply and Demand Constraints +- For each origin `i`, create an equality constraint: `sum(x[i][j] for all j) == supply[i]`. +- For each destination `j`, create an equality constraint: `sum(x[i][j] for all i) == demand[j]`. +- Use `solver.Constraint(value, value)` for equality or `solver.Constraint(lb, ub)` for inequalities. + +### Step 4 - Set Linear Cost Objective +- Create an objective function: `minimize sum(cost[i][j] * x[i][j] for all i,j)`. +- Set coefficients via `objective.SetCoefficient(x[i][j], cost[i][j])` and call `objective.SetMinimization()`. + +### Formulation Template +```json +{ + "sets": ["origins", "destinations"], + "parameters": ["supply[origins]", "demand[destinations]", "cost[origins][destinations]", "capacity[origins][destinations] (optional)"], + "decision_variables": ["flow[origins][destinations] >= 0"], + "objective": { + "sense": "min", + "expression": "sum(cost[i][j] * flow[i][j] for all i,j)" + }, + "constraints": [ + "sum(flow[i][j] for j in destinations) == supply[i] for each i", + "sum(flow[i][j] for i in origins) == demand[j] for each j", + "flow[i][j] <= capacity[i][j] for each i,j (optional)" + ] +} +``` + +### Common Pitfalls +- Forgetting to check if the solver instance was created successfully (`solver` is not `None`). +- Using inequality constraints (`<=`, `>=`) when the problem is balanced and requires exact equality, leading to unexpected surplus or shortage. +- Not setting a time limit for large instances, risking excessive runtime. + +## Solving stage + +### Strategy Overview +Solve the constructed model using a dedicated LP solver (e.g., GLOP, CBC). Extract the solution, verify its correctness, and handle solver statuses robustly. + +### Step 1 - Initialize Solver and Set Limits +- Create solver: `solver = pywraplp.Solver.CreateSolver("GLOP")`. +- Set a reasonable time limit: `solver.SetTimeLimit(30000)` (30 seconds). + +### Step 2 - Solve and Check Status +- Call `status = solver.Solve()`. +- Check if `status` is `OPTIMAL` or `FEASIBLE`. If not, handle the failure (e.g., output error JSON). + +### Step 3 - Extract and Verify Solution +- Retrieve the objective value: `total_cost = objective.Value()`. +- Extract variable values: `flow_val = x[i][j].solution_value()`. +- Implement a verification loop to ensure supply/demand constraints are satisfied within a small tolerance (e.g., `1e-6`). + +### Step 4 - Output Results +- For successful solves, output the objective value in a parseable format (e.g., `RESULT:{total_cost}`). +- Optionally, output a structured JSON with non-zero flows and verification metrics. + +### Code Usage +```python +# Example using OR-Tools GLOP +from ortools.linear_solver import pywraplp + +# 1. Initialize solver +solver = pywraplp.Solver.CreateSolver("GLOP") +if not solver: + raise RuntimeError("Solver backend not available.") + +# 2. Create variables +x = {} +for i in range(num_origins): + for j in range(num_destinations): + ub = capacity[i][j] if capacitated else solver.infinity() + x[i, j] = solver.NumVar(0, ub, f"flow_{i}_{j}") + +# 3. Add constraints +# Supply constraints +for i in range(num_origins): + constraint = solver.Constraint(supply[i], supply[i]) + for j in range(num_destinations): + constraint.SetCoefficient(x[i, j], 1) +# Demand constraints +for j in range(num_destinations): + constraint = solver.Constraint(demand[j], demand[j]) + for i in range(num_origins): + constraint.SetCoefficient(x[i, j], 1) + +# 4. Set objective +objective = solver.Objective() +for i in range(num_origins): + for j in range(num_destinations): + objective.SetCoefficient(x[i, j], cost[i][j]) +objective.SetMinimization() + +# 5. Solve and check status +status = solver.Solve() +if status in (solver.OPTIMAL, solver.FEASIBLE): + total_cost = objective.Value() + # Verification (optional) + for i in range(num_origins): + shipped = sum(x[i, j].solution_value() for j in range(num_destinations)) + assert abs(shipped - supply[i]) < 1e-6 + print(f"RESULT:{total_cost}") +else: + print(f'{{"status":"failed","solver_status":{status}}}') +``` + +### Common Pitfalls +- Not checking solver status before extracting values, leading to runtime errors on infeasible models. +- Ignoring numerical precision issues when comparing constraint satisfaction; always use a tolerance. +- Forgetting to set the objective sense to minimization. + +# Workflow 2 (Algebraic Modeling with Pyomo) + +## Modeling stage + +### Strategy Overview +Use an algebraic modeling language (Pyomo) to declaratively define sets, parameters, variables, and constraints. This approach separates model logic from solver interaction, improving readability and maintainability for complex or large-scale problems. + +### Step 1 - Define Abstract Sets and Parameters +- Create Pyomo `Set` objects for origins and destinations. +- Define `Param` objects for supply, demand, cost, and optional capacity, using dictionaries or rules for initialization. + +### Step 2 - Declare Decision Variables +- Create a non-negative continuous variable indexed over origin and destination sets: `model.x = pyo.Var(model.I, model.J, domain=pyo.NonNegativeReals)`. +- For capacitated problems, variable bounds can be set via a rule or a separate constraint. + +### Step 3 - Construct Objective Function +- Define a linear objective to minimize total cost using a summation expression over the indexed sets. + +### Step 4 - Implement Constraint Rules +- Create supply balance constraints: for each origin, sum of outgoing flows equals supply. +- Create demand balance constraints: for each destination, sum of incoming flows equals demand. +- For capacitated problems, add capacity constraints as inequalities. + +### Formulation Template +```json +{ + "sets": ["I (origins)", "J (destinations)"], + "parameters": ["supply[I]", "demand[J]", "cost[I,J]", "capacity[I,J] (optional)"], + "decision_variables": ["x[I,J] >= 0"], + "objective": { + "sense": "min", + "expression": "sum(cost[i,j] * x[i,j] for i in I, j in J)" + }, + "constraints": [ + "sum(x[i,j] for j in J) == supply[i] for each i in I", + "sum(x[i,j] for i in I) == demand[j] for each j in J", + "x[i,j] <= capacity[i,j] for each i in I, j in J (optional)" + ] +} +``` + +### Common Pitfalls +- Incorrectly initializing parameters with mutable defaults; use `initialize` with a dictionary or function. +- Defining constraint rules that modify global state or have side effects. +- Not verifying that total supply equals total demand before solving, which can lead to infeasibility. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using an external LP solver (e.g., HiGHS, CBC). Configure solver options, check termination status rigorously, and load the solution only after confirming success. + +### Step 1 - Instantiate Solver and Set Options +- Create solver: `solver = pyo.SolverFactory("highs")` (or `"cbc"`). +- Set options: `solver.options["time_limit"] = 30`, `solver.options["threads"] = 4`. + +### Step 2 - Solve with Status Control +- Call `results = solver.solve(model, tee=False, load_solutions=False)`. +- Check `results.solver.status` is `SolverStatus.ok` and `results.solver.termination_condition` is `optimal` or `feasible`. + +### Step 3 - Load and Extract Solution +- If status checks pass, load the solution: `model.solutions.load_from(results)`. +- Retrieve objective value: `total_cost = float(pyo.value(model.obj))`. +- Extract non-zero flows by iterating over variables with a tolerance (e.g., `val > 1e-6`). + +### Step 4 - Validate Solution +- Programmatically verify that supply and demand constraints are satisfied by recomputing sums from variable values. +- Recalculate total cost from extracted flows to cross-check the solver's objective. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# 1. Build model +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=range(num_origins)) +model.J = pyo.Set(initialize=range(num_destinations)) + +model.supply = pyo.Param(model.I, initialize=supply_dict) +model.demand = pyo.Param(model.J, initialize=demand_dict) +model.cost = pyo.Param(model.I, model.J, initialize=cost_dict) + +model.x = pyo.Var(model.I, model.J, domain=pyo.NonNegativeReals) + +def obj_rule(m): + return sum(m.cost[i, j] * m.x[i, j] for i in m.I for j in m.J) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + +def supply_rule(m, i): + return sum(m.x[i, j] for j in m.J) == m.supply[i] +model.supply_con = pyo.Constraint(model.I, rule=supply_rule) + +def demand_rule(m, j): + return sum(m.x[i, j] for i in m.I) == m.demand[j] +model.demand_con = pyo.Constraint(model.J, rule=demand_rule) + +# Optional capacity constraints +if capacitated: + def capacity_rule(m, i, j): + return m.x[i, j] <= capacity_dict[i, j] + model.cap_con = pyo.Constraint(model.I, model.J, rule=capacity_rule) + +# 2. Solve +solver = pyo.SolverFactory("highs") +solver.options["time_limit"] = 30 +results = solver.solve(model, tee=False, load_solutions=False) + +# 3. Check status and extract +status = results.solver.status +term = results.solver.termination_condition +if status == SolverStatus.ok and term in (TerminationCondition.optimal, TerminationCondition.feasible): + model.solutions.load_from(results) + total_cost = float(pyo.value(model.obj)) + # Verification + for i in model.I: + shipped = sum(pyo.value(model.x[i, j]) for j in model.J) + assert abs(shipped - pyo.value(model.supply[i])) < 1e-6 + print(f"RESULT:{total_cost}") +else: + print(f'{{"status":"failed","solver_status":"{status}","termination":"{term}"}}') +``` + +### Common Pitfalls +- Loading solutions without checking termination condition, which may raise `NoFeasibleSolutionError`. +- Using `tee=True` in production, which clutters output with solver logs. +- Not converting the objective value to a standard Python float, which can cause serialization issues. diff --git a/skills/optskills/skill_library/tsp_mtz_formulation.md b/skills/optskills/skill_library/tsp_mtz_formulation.md new file mode 100644 index 0000000..1a205cc --- /dev/null +++ b/skills/optskills/skill_library/tsp_mtz_formulation.md @@ -0,0 +1,248 @@ +--- +name: TSP_MTZ_Formulation +description: | + Model and solve the Traveling Salesperson Problem using the Miller-Tucker-Zemlin (MTZ) formulation with binary arc and integer position variables, producing exact or feasible tours via MIP/CP-SAT solvers. +--- + +# Workflow 1 (CP-SAT with OR-Tools) + +## Modeling stage + +### Strategy Overview +Formulate the TSP as a Mixed-Integer Program (MIP) using the OR-Tools CP-SAT solver interface. This workflow leverages the solver's native constraint programming strengths for combinatorial problems, using a linearized MTZ formulation suitable for exact solving with parallel search. + +### Step 1 - Define Problem Data +- Define the set of nodes `N = {0, 1, ..., n-1}` where `0` is the depot. +- Define a distance matrix `dist[i][j]` for all ordered pairs `(i, j)` where `i != j`. +- Calculate the Big-M parameter as `M = len(N)` (number of nodes). + +### Step 2 - Create Decision Variables +- Create binary arc variables `x[i][j]` for all `i, j in N, i != j`. Each variable equals 1 if arc `(i, j)` is in the tour. +- Create integer position variables `u[i]` for all `i in N`, with domain `[0, n-1]`. These represent the visit order. + +### Step 3 - Formulate Degree Constraints +- For each node `j` in `N`, enforce exactly one incoming arc: `sum(x[i][j] for i in N if i != j) == 1`. +- For each node `i` in `N`, enforce exactly one outgoing arc: `sum(x[i][j] for j in N if j != i) == 1`. + +### Step 4 - Apply MTZ Subtour Elimination +- For all `i, j in N` where `i != j` and `j != depot` (i.e., `j != 0`), add constraint: `u[i] - u[j] + 1 <= M * (1 - x[i][j])`. +- Fix the depot's position to break symmetry: `u[depot] == 0`. + +### Step 5 - Define Objective +- Minimize total tour distance: `minimize sum(dist[i][j] * x[i][j] for i in N for j in N if i != j)`. + +### Formulation Template +```json +{ + "sets": [ + "N: set of nodes (0 is depot)" + ], + "parameters": [ + "dist[i][j]: distance from node i to j, for i, j in N, i != j", + "M: Big-M constant, M = |N|" + ], + "decision_variables": [ + "x[i][j]: binary, 1 if arc (i,j) is selected", + "u[i]: integer, position of node i in tour, domain [0, |N|-1]" + ], + "objective": { + "sense": "min", + "expression": "sum_{i in N} sum_{j in N, j != i} dist[i][j] * x[i][j]" + }, + "constraints": [ + "in_degree: forall j in N: sum_{i in N, i != j} x[i][j] == 1", + "out_degree: forall i in N: sum_{j in N, j != i} x[i][j] == 1", + "mtz: forall i in N, j in N, i != j, j != depot: u[i] - u[j] + 1 <= M * (1 - x[i][j])", + "fix_depot: u[depot] == 0" + ] +} +``` + +### Common Pitfalls +- Using a Big-M value that is too small, which can cut off valid solutions. Use `M = n` (number of nodes). +- Forgetting to exclude the depot (`j != depot`) in the MTZ constraints, which would incorrectly prevent the return arc to the start. +- Not adding symmetry-breaking constraints (like fixing `u[depot]`), which can slow down the solver. + +## Solving stage + +### Strategy Overview +Solve the MIP model using OR-Tools' CP-SAT solver, configured for parallel search and a time limit. Extract and validate the tour from the binary arc variables. + +### Step 1 - Configure Solver +- Instantiate the CP-SAT solver. +- Set solver parameters: `num_search_workers` for parallelism, `max_time_in_seconds` for time limit, and `random_seed` for reproducibility. +- Set `relative_gap_limit` to `0.0` to seek an optimal solution. + +### Step 2 - Solve and Check Status +- Invoke the solver's `Solve` method. +- Check the status is `OPTIMAL` or `FEASIBLE`. Handle `INFEASIBLE` or `MODEL_INVALID` with appropriate error messages. + +### Step 3 - Extract Solution +- If a feasible solution exists, reconstruct the tour sequence. +- Start at the depot. Iteratively find the next node `j` where `x[current][j]` equals 1 in the solution (check `solver.Value(x[current][j]) > 0.5`). +- Append nodes to the tour until all are visited, verifying the final arc returns to the depot. + +### Step 4 - Validate and Report +- Compute the total distance from the extracted tour sequence and compare it to the solver's objective value for consistency. +- Output the tour, total distance, and solver status. + +### Code Usage +```python +# build model from formulation +from ortools.sat.python import cp_model + +model = cp_model.CpModel() +# ... create variables, add constraints and objective as per modeling stage + +# solve with status / termination checks +solver = cp_model.CpSolver() +solver.parameters.num_search_workers = 8 +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.random_seed = 42 +status = solver.Solve(model) + +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + # Extract tour from solver.Value(x[i][j]) variables + tour = [depot] + current = depot + while len(tour) < len(N): + for j in N: + if j != current and solver.Value(x[current][j]) > 0.5: + tour.append(j) + current = j + break + # Validate return to depot + if solver.Value(x[current][depot]) > 0.5: + tour.append(depot) + print(f"Tour: {tour}") +else: + print("No feasible solution found.") +``` + +### Common Pitfalls +- Assuming `OPTIMAL` status within a time limit; always check for `FEASIBLE` as well. +- Incorrect tour reconstruction due to not checking variable solution values against a tolerance (e.g., `> 0.5`). +- Not verifying the final arc closes the cycle, which can happen if the MTZ constraints are misapplied. + +# Workflow 2 (MIP with Pyomo and External Solver) + +## Modeling stage + +### Strategy Overview +Formulate the TSP as a MIP using the Pyomo modeling language, which provides an abstract, solver-agnostic interface. This workflow is designed for use with high-performance external MIP solvers (e.g., Gurobi, HiGHS, CBC) and emphasizes a clean separation of model and solver. + +### Step 1 - Define Abstract Sets and Parameters +- Define a Pyomo `Set` for nodes `model.N`. +- Define a Pyomo `Param` `model.dist` indexed by `(i, j)` for `i != j` to store distances. +- Define parameter `model.M` (Big-M) as the number of nodes. + +### Step 2 - Create Decision Variables +- Create binary variables `model.x[i, j]` for all `i, j in model.N, i != j`. +- Create integer variables `model.u[i]` for all `i in model.N` with bounds `(0, len(model.N)-1)`. + +### Step 3 - Formulate Degree Constraints +- Add constraints for each node `j`: `sum(model.x[i, j] for i in model.N if i != j) == 1`. +- Add constraints for each node `i`: `sum(model.x[i, j] for j in model.N if j != i) == 1`. + +### Step 4 - Apply MTZ Subtour Elimination +- For all `i, j in model.N` where `i != j` and `j != model.depot`, add constraint: `model.u[i] - model.u[j] + 1 <= model.M * (1 - model.x[i, j])`. +- Fix the depot's position: `model.u[model.depot] == 0`. + +### Step 5 - Define Objective +- Minimize total distance: `model.obj = Objective(expr=sum(model.dist[i, j] * model.x[i, j] for i in model.N for j in model.N if i != j), sense=minimize)`. + +### Formulation Template +```json +{ + "sets": [ + "N: Pyomo Set of nodes" + ], + "parameters": [ + "dist[i,j]: Pyomo Param, distance matrix", + "M: Pyomo Param, Big-M constant, M = |N|" + ], + "decision_variables": [ + "x[i,j]: Pyomo Var, domain=Binary", + "u[i]: Pyomo Var, domain=NonNegativeIntegers, bounds=(0, |N|-1)" + ], + "objective": { + "sense": "min", + "expression": "sum_{i in N} sum_{j in N, j != i} dist[i,j] * x[i,j]" + }, + "constraints": [ + "in_degree: forall j in N: sum_{i in N, i != j} x[i,j] == 1", + "out_degree: forall i in N: sum_{j in N, j != i} x[i,j] == 1", + "mtz: forall i in N, j in N, i != j, j != depot: u[i] - u[j] + 1 <= M * (1 - x[i,j])", + "fix_depot: u[depot] == 0" + ] +} +``` + +### Common Pitfalls +- Using `model.N` in list comprehensions inside Pyomo expressions, which can cause errors. Use Pyomo's `model.N` component directly in summations. +- Not setting proper bounds on integer variables `u[i]`, which can lead to unbounded problems. +- Defining the distance parameter for `i == j`, which should be excluded to avoid self-loops. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using an external MIP solver via a solver manager (e.g., `SolverFactory`). Configure solver-specific options for time limit and optimality gap, then extract and validate the solution. + +### Step 1 - Select and Configure Solver +- Use `SolverFactory('solver_name')` (e.g., `'gurobi'`, `'highs'`, `'cbc'`). +- Pass solver options: `timelimit`, `mipgap` (or `relgap`), `threads`, and `seed` for reproducibility. + +### Step 2 - Solve and Check Termination +- Invoke `solver.solve(model, options=...)`. +- Check both the solver status (`solver.status`) and model termination condition (`model.termination_condition`). Accept `optimal` or `feasible` results. + +### Step 3 - Extract Solution +- Access variable values using `value(model.x[i, j])` or `model.x[i, j].value`. +- Reconstruct the tour by starting at the depot and following arcs where the variable value is approximately 1 (check `> 0.5`). + +### Step 4 - Validate and Report +- Compute the tour's total distance from the sequence and compare to `value(model.obj)`. +- Output the tour, objective value, and solver statistics. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo + +model = pyo.ConcreteModel() +model.N = pyo.Set(initialize=node_list) +model.depot = node_list[0] +model.dist = pyo.Param(model.N, model.N, initialize=dist_dict, default=0) +model.M = pyo.Param(initialize=len(model.N)) +# ... create variables, constraints, and objective as per modeling stage + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') # or 'gurobi', 'cbc' +results = solver.solve(model, options={'time_limit': 30, 'threads': 4}) + +if (results.solver.status == pyo.SolverStatus.ok and + results.solver.termination_condition in (pyo.TerminationCondition.optimal, + pyo.TerminationCondition.feasible)): + # Extract tour + tour = [model.depot] + current = model.depot + visited = set(tour) + while len(visited) < len(model.N): + for j in model.N: + if j != current and pyo.value(model.x[current, j]) > 0.5: + tour.append(j) + current = j + visited.add(j) + break + # Check return arc + if pyo.value(model.x[current, model.depot]) > 0.5: + tour.append(model.depot) + print(f"Tour: {tour}") +else: + print("Solver did not find a feasible solution.") +``` + +### Common Pitfalls +- Confusing Pyomo's `solver.status` (solver process) with `termination_condition` (solution quality). Both must be checked. +- Not using `pyo.value()` to access variable values in the solution object. +- Assuming the solver returns an optimal solution within the time limit; always handle feasible solutions gracefully. diff --git a/skills/optskills/skill_library/tsp_mtz_mip.md b/skills/optskills/skill_library/tsp_mtz_mip.md new file mode 100644 index 0000000..e67abb7 --- /dev/null +++ b/skills/optskills/skill_library/tsp_mtz_mip.md @@ -0,0 +1,247 @@ +--- +name: TSP-MTZ-MIP +description: | + Model and solve the Traveling Salesperson Problem (TSP) as a Mixed-Integer Program (MIP) using binary arc selection and Miller-Tucker-Zemlin (MTZ) subtour elimination constraints, with workflows for both commercial and open-source solver backends. + +--- +# Workflow 1 (Commercial Solver via Pyomo) + +## Modeling stage + +### Strategy Overview +Model the TSP as a MIP using the MTZ formulation and solve it using a high-performance commercial solver (e.g., Gurobi, CPLEX) via Pyomo, prioritizing solution speed and advanced MIP features. + +### Step 1 - Define Sets and Parameters +- Define the set of nodes `NODES`. +- Define a cost parameter `cost[i, j]` for the travel distance from node `i` to node `j`. Ensure the cost matrix is complete for all ordered pairs `(i, j)` where `i != j`. + +### Step 2 - Create Decision Variables +- Create binary decision variables `x[i, j]` for each ordered pair `(i, j)` where `i != j`. `x[i, j] = 1` indicates the arc from `i` to `j` is included in the tour. +- Create continuous (or integer) position variables `u[i]` for each node `i`. `u[i]` represents the visit order (position) of node `i` in the tour. + +### Step 3 - Formulate Degree Constraints +- For each node `j`, enforce exactly one incoming arc: `sum(x[i, j] for i in NODES if i != j) == 1`. +- For each node `i`, enforce exactly one outgoing arc: `sum(x[i, j] for j in NODES if j != i) == 1`. + +### Step 4 - Apply MTZ Subtour Elimination +- Fix the position of the starting node (node 0): `u[0] == 0`. +- For all other nodes `i != 0`, set bounds: `1 <= u[i] <= len(NODES) - 1`. +- For all pairs `i, j` where `i != j` and both are not the starting node, add the MTZ constraint: `u[i] - u[j] + n * x[i, j] <= n - 1`, where `n = len(NODES)`. + +### Step 5 - Define the Objective +- Minimize the total travel cost: `sum(cost[i, j] * x[i, j] for all i, j where i != j)`. + +### Formulation Template +```json +{ + "sets": [ + "NODES" + ], + "parameters": [ + "cost[i, j] for i, j in NODES, i != j" + ], + "decision_variables": [ + "x[i, j] ∈ {0, 1} for i, j in NODES, i != j", + "u[i] ∈ ℝ (or ℤ) for i in NODES" + ], + "objective": { + "sense": "min", + "expression": "sum(cost[i, j] * x[i, j])" + }, + "constraints": [ + "in_degree: sum(x[i, j] for i in NODES if i != j) == 1, for each j in NODES", + "out_degree: sum(x[i, j] for j in NODES if j != i) == 1, for each i in NODES", + "mtz: u[i] - u[j] + n * x[i, j] <= n - 1, for i, j in NODES \\ {0}, i != j", + "u_start: u[0] == 0", + "u_bounds: 1 <= u[i] <= n - 1, for i in NODES \\ {0}" + ] +} +``` + +### Common Pitfalls +- Adding MTZ constraints for pairs where `i` or `j` is 0, which are redundant and increase problem size. +- Defining the cost matrix only for `i < j` in an asymmetric TSP, missing the cost for the reverse direction. +- Not fixing `u[0]` or setting its bounds incorrectly, which can lead to symmetric, equivalent solutions and slower solving. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using a commercial solver interface, configure it for reproducibility and performance, and rigorously validate the solution by reconstructing and checking the Hamiltonian tour. + +### Step 1 - Instantiate Solver and Set Parameters +- Create a solver object using `SolverFactory('[SOLVER_NAME]')`. +- Configure key parameters: set a time limit (`TimeLimit`), optimality gap tolerance (`MIPGap`), number of threads (`Threads`), and a random seed (`Seed`) for deterministic results. + +### Step 2 - Solve and Check Status +- Execute the solve command on the model instance. +- Check the solver status (`SolverStatus.ok`) and termination condition (`TerminationCondition.optimal` or `TerminationCondition.feasible`) before proceeding to extract the solution. + +### Step 3 - Extract and Reconstruct the Tour +- Collect all arcs where the solution value of `x[i, j]` is greater than a tolerance (e.g., `0.5`). +- Build a successor dictionary mapping each node `i` to its unique outgoing node `j` from the selected arcs. +- Starting from the designated start node (0), follow the successor chain to construct an ordered list of visited nodes, verifying that a single cycle is formed. + +### Step 4 - Validate Solution Integrity +- Verify that the reconstructed tour visits each node exactly once. +- Recalculate the total cost by summing `cost[i, j]` for consecutive nodes in the tour (including the return to start). Compare this with the solver's reported objective value to catch discrepancies. + +### Code Usage +```python +import pyomo.environ as pyo + +# 1. Build model (model) from the formulation template above. +# ... (Pyomo model creation code) + +# 2. Solve with status / termination checks +solver = pyo.SolverFactory('[SOLVER_NAME]') +solver.options['TimeLimit'] = [TIME_LIMIT] +solver.options['MIPGap'] = [MIP_GAP] +solver.options['Threads'] = [THREADS] +solver.options['Seed'] = [RANDOM_SEED] + +results = solver.solve(model, tee=False) # tee=True for solver log + +# Check status +from pyomo.opt import SolverStatus, TerminationCondition +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in [TerminationCondition.optimal, TerminationCondition.feasible]): + # Proceed to extract solution + successor = {i: j for i in model.NODES for j in model.NODES if i != j and pyo.value(model.x[i, j]) > 0.5} + tour = [0] + current = 0 + while len(tour) < len(model.NODES): + current = successor[current] + tour.append(current) + calculated_cost = sum(model.cost[tour[i], tour[i+1]] for i in range(len(tour)-1)) + model.cost[tour[-1], tour[0]] + # Validate + if abs(pyo.value(model.obj) - calculated_cost) > 1e-6: + raise ValueError("Cost mismatch between solver and reconstructed tour.") +else: + # Handle suboptimal or failed solve + print(f"Solver did not find a feasible solution. Status: {results.solver.termination_condition}") +``` + +### Common Pitfalls +- Extracting variable values without checking if the solver found a feasible solution, leading to `None` values or errors. +- Using a loose optimality gap (`MIPGap`) for a problem where the true optimum is required, potentially accepting suboptimal tours. +- Not setting a random seed, causing non-reproducible results across runs due to the solver's internal heuristics. + +# Workflow 2 (Open-Source Solver via OR-Tools) + +## Modeling stage + +### Strategy Overview +Model the TSP as a MIP using the MTZ formulation and solve it using the open-source SCIP solver via Google's OR-Tools `pywraplp` interface, providing a free, capable alternative without commercial licenses. + +### Step 1 - Define Problem Data +- Define the number of nodes `n` and a list or range of node indices `N = range(n)`. +- Define the cost matrix as a 2D list or dictionary `cost[i][j]` accessible by node indices. + +### Step 2 - Create Solver and Variables +- Instantiate a MIP solver object: `solver = pywraplp.Solver.CreateSolver('SCIP')`. +- Create binary variables `x[i][j]` for all `i != j` using `solver.IntVar(0, 1, '')`. +- Create integer position variables `u[i]` with bounds `0` to `n-1` using `solver.IntVar(0, n-1, '')`. + +### Step 3 - Add Degree Constraints +- For each node `j`, create a constraint: `sum(x[i][j] for i in N if i != j) == 1`. +- For each node `i`, create a constraint: `sum(x[i][j] for j in N if j != i) == 1`. + +### Step 4 - Add MTZ Subtour Elimination Constraints +- Set the position of the start node: `solver.Add(u[0] == 0)`. +- For all `i` in `{1,...,n-1}` and `j` in `{1,...,n-1}` where `i != j`, add constraint: `u[i] - u[j] + n * x[i][j] <= n - 1`. + +### Step 5 - Set the Objective +- Create the objective expression: `sum(cost[i][j] * x[i][j] for i in N for j in N if i != j)`. +- Set the solver to minimize this expression. + +### Formulation Template +```json +{ + "sets": [ + "n (number of nodes)", + "N = {0, 1, ..., n-1}" + ], + "parameters": [ + "cost[i][j] for i, j in N, i != j" + ], + "decision_variables": [ + "x[i][j] ∈ {0, 1} for i, j in N, i != j", + "u[i] ∈ {0, ..., n-1} for i in N" + ], + "objective": { + "sense": "min", + "expression": "sum(cost[i][j] * x[i][j])" + }, + "constraints": [ + "in_degree: sum(x[i][j] for i in N if i != j) == 1, for each j in N", + "out_degree: sum(x[i][j] for j in N if j != i) == 1, for each i in N", + "mtz: u[i] - u[j] + n * x[i][j] <= n - 1, for i, j in N \\ {0}, i != j", + "u_start: u[0] == 0" + ] +} +``` + +### Common Pitfalls +- Creating variables `x[i][i]` (self-loops), which waste memory and must be excluded from sums. +- Adding MTZ constraints for pairs where `i` or `j` is 0, which are unnecessary and increase problem size. + +## Solving stage + +### Strategy Overview +Solve the model using the OR-Tools wrapper, configure basic solver settings, extract the solution, and implement a robust tour reconstruction algorithm that handles potential numerical tolerances. + +### Step 1 - Configure and Execute Solve +- Set solver parameters like time limit (`solver.SetTimeLimit([TIME_LIMIT_MS])`). +- Call `solver.Solve()` to initiate the optimization. + +### Step 2 - Verify Solution Status +- Check the solver result status: `pywraplp.Solver.OPTIMAL` or `pywraplp.Solver.FEASIBLE`. Handle `pywraplp.Solver.NOT_SOLVED` or other statuses appropriately. + +### Step 3 - Extract Arc Selections and Reconstruct Tour +- Build a successor dictionary mapping each node `i` to its unique outgoing node `j` where `x[i][j].solution_value() > 0.5`. +- Starting from node 0, iteratively follow the successor chain to build the tour sequence. Use a visited set to detect completion. + +### Step 4 - Validate and Output Results +- Verify the tour length equals `n`. +- Recompute the objective from the extracted tour and compare with `solver.Objective().Value()`. +- Package results (status, objective value, tour list) in a structured format (e.g., dictionary, JSON). + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# 1. Build model +solver = pywraplp.Solver.CreateSolver('SCIP') +n = len(NODES) +# ... Create variables x, u and add constraints as per modeling stage + +# 2. Solve with status / termination checks +solver.SetTimeLimit([TIME_LIMIT_MS]) # milliseconds +status = solver.Solve() + +# Check status +if status in [pywraplp.Solver.OPTIMAL, pywraplp.Solver.FEASIBLE]: + # Build successor dictionary and reconstruct tour + successor = {i: j for i in range(n) for j in range(n) if i != j and x[i][j].solution_value() > 0.5} + tour = [0] + current = 0 + while len(tour) < n: + current = successor[current] + tour.append(current) + # Validate + calculated_obj = sum(cost[tour[i]][tour[i+1]] for i in range(n-1)) + cost[tour[-1]][tour[0]] + if abs(solver.Objective().Value() - calculated_obj) > 1e-6: + raise ValueError("Objective value mismatch.") + result = { + 'status': 'OPTIMAL' if status == pywraplp.Solver.OPTIMAL else 'FEASIBLE', + 'objective': solver.Objective().Value(), + 'tour': tour + } +else: + result = {'status': 'NOT_SOLVED', 'objective': None, 'tour': None} +``` + +### Common Pitfalls +- Using a strict equality (`== 0.5`) to check binary variable values, failing due to floating-point precision. Use a tolerance (e.g., `> 0.5`). +- An inefficient O(n²) tour reconstruction that searches all arcs at each step. A dictionary mapping `i` to its successor `j` built once is more efficient. +- Not handling the case where the solver finds a feasible but not optimal solution, potentially leading to incorrect assumptions about solution quality. diff --git a/skills/optskills/skill_library/tsp_with_position_variables.md b/skills/optskills/skill_library/tsp_with_position_variables.md new file mode 100644 index 0000000..773e422 --- /dev/null +++ b/skills/optskills/skill_library/tsp_with_position_variables.md @@ -0,0 +1,258 @@ +--- +name: TSP with Position Variables +description: | + Model and solve the Traveling Salesperson Problem using binary arc selection and integer position assignment variables, with explicit subtour elimination constraints, to produce a minimal-distance Hamiltonian cycle. +--- + +# Workflow 1 (MIP with MTZ using Pyomo/Gurobi) + +## Modeling stage + +### Strategy Overview +This workflow formulates the TSP as a Mixed-Integer Program (MIP) using the Miller-Tucker-Zemlin (MTZ) subtour elimination constraints. It is implemented in Pyomo and solved with the Gurobi solver, which is well-suited for medium to large-scale routing problems. + +### Step 1 - Define Sets and Parameters +- Define a set `CITIES` representing all locations to be visited. +- Define a parameter `distance[i][j]` representing the travel cost from city `i` to city `j`. Ensure `distance[i][i]` is set to a large value or zero, as self-loops are prohibited. + +### Step 2 - Create Decision Variables +- Create binary variable `x[i][j]` for all `i, j` in `CITIES`. `x[i][j] = 1` indicates the arc from city `i` to city `j` is selected in the tour. +- Create integer variable `u[i]` for all `i` in `CITIES`. `u[i]` represents the position of city `i` in the tour sequence, starting from 1. + +### Step 3 - Formulate Degree and Assignment Constraints +- Add constraints to ensure each city has exactly one outgoing arc: `sum_{j in CITIES, j != i} x[i][j] == 1` for all `i`. +- Add constraints to ensure each city has exactly one incoming arc: `sum_{i in CITIES, i != j} x[i][j] == 1` for all `j`. +- Explicitly forbid self-loops: `x[i][i] == 0` for all `i`. + +### Step 4 - Implement Subtour Elimination (MTZ) +- For all `i, j` in `CITIES` where `i != 0` and `j != 0` and `i != j`, add the MTZ constraint: `u[i] - u[j] + n * x[i][j] <= n - 1`. Here, `n` is the total number of cities. +- Fix the position of the start city (city `0`) to break symmetry: `u[0] == 1`. +- Set bounds for position variables: `1 <= u[i] <= n` for all `i`. + +### Step 5 - Define the Objective +- Minimize the total travel distance: `sum_{i in CITIES} sum_{j in CITIES, j != i} distance[i][j] * x[i][j]`. + +### Formulation Template +```json +{ + "sets": [ + {"name": "CITIES", "description": "Set of all cities/nodes to visit."} + ], + "parameters": [ + {"name": "distance", "index": ["CITIES", "CITIES"], "description": "Cost matrix for travel between cities."}, + {"name": "n", "value": "len(CITIES)", "description": "Number of cities."} + ], + "decision_variables": [ + {"name": "x", "index": ["CITIES", "CITIES"], "type": "binary", "description": "Arc selection variable."}, + {"name": "u", "index": ["CITIES"], "type": "integer", "bounds": "[1, n]", "description": "Position assignment variable."} + ], + "objective": { + "sense": "min", + "expression": "sum_{i in CITIES} sum_{j in CITIES, j != i} distance[i][j] * x[i][j]" + }, + "constraints": [ + {"name": "outgoing_arc", "expression": "sum_{j in CITIES, j != i} x[i][j] == 1", "for": "i in CITIES"}, + {"name": "incoming_arc", "expression": "sum_{i in CITIES, i != j} x[i][j] == 1", "for": "j in CITIES"}, + {"name": "no_self_loop", "expression": "x[i][i] == 0", "for": "i in CITIES"}, + {"name": "mtz", "expression": "u[i] - u[j] + n * x[i][j] <= n - 1", "for": "i in CITIES, j in CITIES where i != 0 and j != 0 and i != j"}, + {"name": "fix_start", "expression": "u[0] == 1"} + ] +} +``` + +### Common Pitfalls +- Applying MTZ constraints to the start city (`i=0` or `j=0`), which can make the model infeasible. The start city's position is fixed, so these constraints are not needed for arcs involving it. +- Forgetting to exclude self-loops (`i=j`) from the objective summation, which could incorrectly add zero or large penalty costs. +- Using an incorrect coefficient (like `n-1`) in the MTZ constraint, which may not correctly eliminate all subtours. The standard form uses `n` for problems where the start city position is fixed to 1. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the Gurobi solver with configured parameters for reproducibility and performance. After solving, rigorously check the status, extract the solution, and reconstruct the tour sequence. + +### Step 1 - Configure and Execute the Solver +- Instantiate the Gurobi solver via Pyomo's `SolverFactory`. +- Set key parameters: `TimeLimit=30`, `MIPGap=0.0` (for optimality), `Threads=4`, and `Seed=42` for deterministic behavior. +- Execute the solve with `tee=True` to output the solver log for debugging. + +### Step 2 - Validate Solver Status and Termination +- Check the solver status (`solver.status`). It should be `ok`. +- Check the model termination condition (`model.termination_condition`). Accept `optimal` or `feasible` (for early termination). If the status is not acceptable, do not load the solution and report the condition. + +### Step 3 - Extract and Verify the Solution +- Load the solution into the model if the status checks pass. +- Extract the selected arcs by finding all `x[i][j]` variables with a value > 0.5. +- Extract the position assignments from the `u[i]` variables. +- Reconstruct the tour by starting at the designated start city (city `0`) and following the selected arcs until returning to the start. +- Optionally, verify the objective value by manually summing the `distance` along the extracted tour sequence. + +### Step 4 - Handle Infeasibility or Errors +- If the model is infeasible, inspect the MTZ constraint formulation and the bounds on position variables. A common error is incorrect indexing in the MTZ constraints. +- If the solver hits the time limit, the best solution found can still be used if the termination condition is `feasible`. + +### Code Usage +```python +import pyomo.environ as pyo + +# build model from formulation +model = pyo.ConcreteModel() +# ... (define sets, params, variables, constraints, objective as per modeling stage) + +# solve with status / termination checks +solver = pyo.SolverFactory('gurobi') +solver.options['TimeLimit'] = 30 +solver.options['MIPGap'] = 0.0 +solver.options['Threads'] = 4 +solver.options['Seed'] = 42 + +results = solver.solve(model, tee=True) # tee=True for logs + +# Check solver status +if results.solver.status != pyo.SolverStatus.ok: + raise RuntimeError(f"Solver failed with status: {results.solver.status}") + +# Check termination condition +if model.termination_condition not in [pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible]: + raise RuntimeError(f"No acceptable solution found. Termination: {model.termination_condition}") + +# Load and process solution +model.solutions.load_from(results) +# ... (extract x[i][j] and u[i] values, reconstruct tour) +``` + +### Common Pitfalls +- Trusting a non-zero solver return code or an `unknown` termination condition as a valid solution. Always perform explicit checks. +- Loading solutions before verifying the solver status, which can lead to errors if the solve failed. +- Not verifying the reconstructed tour against the objective value, which might catch extraction errors. + +# Workflow 2 (MIP with MTZ using Pyomo/HiGHS) + +## Modeling stage + +### Strategy Overview +This workflow also uses the MTZ formulation but is designed for use with the open-source HiGHS solver via Pyomo. It emphasizes parameterization for reusability and includes explicit handling for the start city in constraints. + +### Step 1 - Parameterize Model Components +- Define `n = len(CITIES)` dynamically. Use this parameter in constraint coefficients and variable bounds to make the model adaptable. +- Define the `distance` matrix, ensuring it is square and `distance[i][i]` is handled appropriately (e.g., set to 0). + +### Step 2 - Instantiate Variables with Bounds +- Create binary variable `x[i][j]` for arc selection. +- Create integer variable `u[i]` for position, with explicit lower bound `1` and upper bound `n`. + +### Step 3 - Apply Standard TSP Constraints +- Enforce single outgoing and incoming arc per city using summations that exclude the `i=j` case. +- Explicitly set `x[i][i].fix(0)` to prevent self-loops. + +### Step 4 - Apply Subtour Elimination with Start City Handling +- For all `i, j` where `i != 0`, `j != 0`, and `i != j`, add the MTZ constraint: `u[i] - u[j] + (n-1) * x[i][j] <= n-2`. This is an equivalent variant that works when `u[0]` is fixed to 1. +- Fix the start city's position: `u[0].fix(1)`. + +### Step 5 - Set the Minimization Objective +- Minimize `sum_{i in CITIES} sum_{j in CITIES} distance[i][j] * x[i][j]`, relying on the fixed zero values on the diagonal to ignore self-loops. + +### Formulation Template +```json +{ + "sets": [ + {"name": "CITIES", "description": "Set of all cities/nodes."} + ], + "parameters": [ + {"name": "distance", "index": ["CITIES", "CITIES"], "description": "Cost matrix. distance[i][i] should be 0."}, + {"name": "n", "value": "len(CITIES)", "description": "Cardinality of CITIES."} + ], + "decision_variables": [ + {"name": "x", "index": ["CITIES", "CITIES"], "type": "binary", "description": "1 if arc i->j is in the tour."}, + {"name": "u", "index": ["CITIES"], "type": "integer", "bounds": "[1, n]", "description": "Visit order of city i."} + ], + "objective": { + "sense": "min", + "expression": "sum_{i in CITIES} sum_{j in CITIES} distance[i][j] * x[i][j]" + }, + "constraints": [ + {"name": "flow_out", "expression": "sum_{j in CITIES} x[i][j] == 1", "for": "i in CITIES"}, + {"name": "flow_in", "expression": "sum_{i in CITIES} x[i][j] == 1", "for": "j in CITIES"}, + {"name": "mtz", "expression": "u[i] - u[j] + (n-1) * x[i][j] <= n-2", "for": "i in CITIES, j in CITIES where i != 0 and j != 0 and i != j"}, + {"name": "start_pos", "expression": "u[0] == 1"} + ] +} +``` +*Note: The `flow_out` and `flow_in` constraints sum over all `j` and `i`, respectively, assuming `x[i][i]` is fixed to 0.* + +### Common Pitfalls +- Using the `(n-1)` coefficient in the MTZ constraint but not adjusting the right-hand side to `n-2`, leading to an incorrect formulation. +- Omitting the condition `i != j` in the MTZ constraint, which is unnecessary as `x[i][i]` is fixed to 0 but can cause a modeling error if the constraint is evaluated for `i=j`. +- Not fixing `x[i][i]` to 0, which would allow the solver to select self-loops to satisfy degree constraints trivially. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the HiGHS solver, focusing on robust solution loading and verification. This workflow includes explicit checks for solver status and uses a verification strategy for small instances. + +### Step 1 - Configure HiGHS Solver +- Instantiate the HiGHS solver via `SolverFactory('appsi_highs')` or the appropriate interface. +- Set parameters: `time_limit=30`, `mip_rel_gap=0.0`, `threads=4`, `presolve="on"`. +- Use `load_solutions=False` in the initial solve command to prevent automatic loading before status checks. + +### Step 2 - Check Solution Status Systematically +- After solving, check `results.solver.status`. It must be `ok`. +- Check `model.termination_condition`. Accept `optimal` or `feasible`. For other conditions (like `infeasible` or `maxTimeLimit`), handle accordingly without loading the solution. + +### Step 3 - Load Results and Reconstruct Tour +- If status checks pass, load the solution using `model.solutions.load_from(results)`. +- Extract the tour by iterating from the start city: `current = 0`, then repeatedly find `next_city` such that `pyo.value(model.x[current, next_city]) > 0.5`. +- Record the sequence to form the Hamiltonian cycle. + +### Step 4 - Implement Verification and Error Handling +- For small `n`, implement a brute-force verification to confirm the extracted tour is a valid cycle and its cost matches the solver's objective value. This is a strong sanity check. +- Structure the output (e.g., as a JSON object) to clearly indicate success/failure, the tour, total distance, and solver status details. + +### Code Usage +```python +import pyomo.environ as pyo + +# build model from formulation +model = pyo.ConcreteModel() +# ... (define sets, params, variables, constraints, objective as per modeling stage) +# Fix self-loops +for i in model.CITIES: + model.x[i, i].fix(0) + +# solve with status / termination checks +solver = pyo.SolverFactory('appsi_highs') # or 'highs' +solver.options['time_limit'] = 30 +solver.options['mip_rel_gap'] = 0.0 +solver.options['threads'] = 4 +solver.options['presolve'] = 'on' + +# Solve without auto-loading +results = solver.solve(model, load_solutions=False) + +# Status verification +if results.solver.status != pyo.SolverStatus.ok: + raise RuntimeError(f"Solver failed: {results.solver.status}") + +tc = model.termination_condition +if tc not in [pyo.TerminationCondition.optimal, pyo.TerminationCondition.feasible]: + # Handle non-success, e.g., log and return empty result + output = {"status": "failure", "termination_condition": str(tc)} + # return or raise +else: + # Load and process solution + model.solutions.load_from(results) + tour = [0] + current = 0 + for _ in range(len(model.CITIES)-1): + for j in model.CITIES: + if j != current and pyo.value(model.x[current, j]) > 0.5: + tour.append(j) + current = j + break + # tour should end back at start city 0 + output = {"status": "success", "tour": tour, "distance": pyo.value(model.obj)} +``` + +### Common Pitfalls +- Assuming `load_solutions=True` (the default) and trying to access variable values after an unsuccessful solve, which may raise exceptions or return stale data. +- Not verifying that the reconstructed tour forms a complete cycle back to the start city, which could indicate an error in extraction logic. +- For HiGHS, using an incorrect solver factory name or not having the appropriate Pyomo extension installed, leading to a `SolverFactory` error. diff --git a/skills/optskills/skill_library/weighted_set_cover_optimization.md b/skills/optskills/skill_library/weighted_set_cover_optimization.md new file mode 100644 index 0000000..d11d53e --- /dev/null +++ b/skills/optskills/skill_library/weighted_set_cover_optimization.md @@ -0,0 +1,236 @@ +--- +name: Weighted Set Cover Optimization +description: | + Model binary selection problems with coverage requirements as weighted set cover MILP, then solve with open-source solvers via Pyomo or OR-Tools. +--- + +# Workflow 1 (Pyomo with HiGHS/CBC) + +## Modeling stage + +### Strategy Overview +Model the problem as a concrete Pyomo model using structured sets and rules, enabling clear separation of data and logic for maintainability and solver portability. + +### Step 1 - Define Sets and Parameters +- Identify the set of selectable items (e.g., facilities, cameras, teams) and the set of elements requiring coverage (e.g., areas, tasks). +- Define a linear cost parameter for each item and a binary coverage mapping for each element, listing which items can cover it. + +### Step 2 - Create Binary Decision Variables +- Instantiate a binary variable for each selectable item, where a value of 1 indicates selection. + +### Step 3 - Formulate Linear Objective +- Define the objective to minimize the total linear cost: the sum of each item's cost multiplied by its binary variable. + +### Step 4 - Implement Coverage Constraints +- For each element requiring coverage, add a constraint ensuring the sum of the binary variables for items that cover it is at least one. + +### Formulation Template +```json +{ + "sets": [ + "I: Set of selectable items.", + "J: Set of elements requiring coverage." + ], + "parameters": [ + "cost[i ∈ I]: Linear cost of selecting item i.", + "cover[j ∈ J]: List of items i ∈ I that can cover element j." + ], + "decision_variables": [ + "x[i ∈ I] ∈ {0, 1}: 1 if item i is selected." + ], + "objective": { + "sense": "min", + "expression": "∑_{i ∈ I} cost[i] * x[i]" + }, + "constraints": [ + "Coverage: ∑_{i ∈ cover[j]} x[i] ≥ 1, ∀ j ∈ J" + ] +} +``` + +### Common Pitfalls +- Using inconsistent indexing between cost dictionary and coverage mapping, leading to missing coefficients. +- Forgetting to verify the coverage mapping is complete for all elements, resulting in infeasible models. +- Defining the coverage parameter as a dense matrix for sparse problems, causing unnecessary memory overhead. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using an open-source MILP solver (HiGHS or CBC), configuring performance limits and robustly checking solver status before extracting and verifying the solution. + +### Step 1 - Configure and Execute Solver +- Instantiate a solver factory (e.g., `"highs"` or `"cbc"`). Set practical options: time limit, optimality gap tolerance (e.g., 0.0), and thread count for parallelism. +- Call the solver on the model, suppressing the log output (`tee=False`) unless debugging. + +### Step 2 - Check Solver Status and Termination +- Inspect the solver status (`SolverStatus.ok`) and termination condition (`TerminationCondition.optimal` or `.feasible`). Proceed only if the solve was successful. + +### Step 3 - Extract and Verify Solution +- Retrieve selected items where the variable value exceeds 0.5 (accounting for numerical tolerance). Compute the total cost from the objective value. +- Programmatically verify that every element is covered by at least one selected item using the coverage mapping. + +### Step 4 - Output Standardized Results +- Package the solution status, objective value, list of selected items, and verification result into a structured dictionary (e.g., JSON) for downstream use. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# Build model from formulation +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=items) +model.J = pyo.Set(initialize=elements) +model.x = pyo.Var(model.I, domain=pyo.Binary) +model.obj = pyo.Objective(expr=sum(cost[i] * model.x[i] for i in model.I), sense=pyo.minimize) +def cover_rule(m, j): + return sum(m.x[i] for i in coverage[j]) >= 1 +model.cover = pyo.Constraint(model.J, rule=cover_rule) + +# Solve with status / termination checks +solver = pyo.SolverFactory("highs") # or "cbc" +solver.options["time_limit"] = 30 +solver.options["mip_rel_gap"] = 0.0 +solver.options["threads"] = 4 +results = solver.solve(model, tee=False) + +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in {TerminationCondition.optimal, TerminationCondition.feasible}: + selected = [i for i in model.I if pyo.value(model.x[i]) > 0.5] + total_cost = float(pyo.value(model.obj)) + # Verification + verified = all(any(pyo.value(model.x[i]) > 0.5 for i in coverage[j]) for j in elements) + solution_payload = { + "status": "optimal" if term == TerminationCondition.optimal else "feasible", + "objective_value": total_cost, + "selected_items": selected, + "coverage_verified": verified + } +else: + solution_payload = {"status": "failed", "solver_status": str(status), "termination": str(term)} +``` + +### Common Pitfalls +- Not checking both solver status and termination condition, potentially extracting invalid solutions from interrupted solves. +- Using a loose optimality gap (`mip_rel_gap`) when an exact solution is required, leading to suboptimal selections. +- Failing to implement post-solve verification, which can miss modeling errors that the solver tolerated. + +# Workflow 2 (OR-Tools MIP/CP-SAT) + +## Modeling stage + +### Strategy Overview +Model the set cover problem directly using the OR-Tools linear solver (MIP) or CP-SAT API, which is efficient for binary programs and offers fine-grained control over the solving process. + +### Step 1 - Map Problem Data +- Define lists/dictionaries for item costs and for coverage, where each element maps to a list of covering item indices. + +### Step 2 - Instantiate Solver and Variables +- Create a solver instance (e.g., `"SCIP"` for MIP or `CpModel()` for CP-SAT). For MIP, create binary integer variables. For CP-SAT, create Boolean variables. + +### Step 3 - Build Coverage Constraints +- For each element, create a linear constraint with a lower bound of 1. Add the binary/Boolean variable for each covering item with a coefficient of 1. + +### Step 4 - Set Linear Objective +- Define the objective as the sum of each item's cost multiplied by its variable, and set the sense to minimization. + +### Formulation Template +```json +{ + "sets": [ + "I: Index set of selectable items.", + "J: Index set of elements requiring coverage." + ], + "parameters": [ + "cost[i ∈ I]: Linear cost of item i.", + "cover[j ∈ J]: List of item indices i that cover element j." + ], + "decision_variables": [ + "x[i ∈ I] ∈ {0, 1}: Selection variable for item i." + ], + "objective": { + "sense": "min", + "expression": "∑_{i ∈ I} cost[i] * x[i]" + }, + "constraints": [ + "Coverage: ∑_{i ∈ cover[j]} x[i] ≥ 1, ∀ j ∈ J" + ] +} +``` + +### Common Pitfalls +- Using the wrong OR-Tools solver type for the problem (e.g., MIP for very large combinatorial problems where CP-SAT may be more effective). +- Incorrectly setting constraint bounds (e.g., using `solver.infinity()` as the upper bound for a '≥1' constraint). +- Not naming variables and constraints, making debugging difficult for larger instances. + +## Solving stage + +### Strategy Overview +Solve the model using OR-Tools' MIP or CP-SAT solver, configure performance settings, extract the solution with numerical tolerance checks, and validate coverage. + +### Step 1 - Configure Solver Settings +- Set a time limit (in milliseconds for MIP, seconds for CP-SAT) and the number of parallel workers (`SetNumThreads` for MIP, `num_search_workers` for CP-SAT) to balance speed and resource use. + +### Step 2 - Execute Solve and Check Status +- Call the solver's `Solve()` method. Check the return status against `OPTIMAL` and `FEASIBLE` constants (MIP) or `OPTIMAL` and `FEASIBLE` (CP-SAT). + +### Step 3 - Extract Selected Items +- Iterate through decision variables. For MIP, use `solution_value() > 0.5`. For CP-SAT, use `BooleanValue()` or `Value()`. Collect indices of selected items. + +### Step 4 - Verify and Package Results +- Verify coverage by checking each element's covering list against the selected items. Assemble a result dictionary containing status, objective value, selection list, and verification flag. + +### Code Usage +```python +# Option A: Using OR-Tools MIP Solver (SCIP/CBC) +from ortools.linear_solver import pywraplp + +solver = pywraplp.Solver.CreateSolver("SCIP") +# Create variables +x = {i: solver.IntVar(0, 1, f'x_{i}') for i in items} +# Add coverage constraints +for j in elements: + constraint = solver.Constraint(1, solver.infinity(), f'cover_{j}') + for i in coverage[j]: + constraint.SetCoefficient(x[i], 1) +# Set objective +objective = solver.Objective() +for i in items: + objective.SetCoefficient(x[i], cost[i]) +objective.SetMinimization() +# Configure and solve +solver.SetTimeLimit(30000) # milliseconds +solver.SetNumThreads(4) +status = solver.Solve() +# Check status and extract +if status in (solver.OPTIMAL, solver.FEASIBLE): + selected = [i for i in items if x[i].solution_value() > 0.5] + total_cost = objective.Value() + verified = all(any(x[i].solution_value() > 0.5 for i in coverage[j]) for j in elements) + +# Option B: Using OR-Tools CP-SAT +from ortools.sat.python import cp_model + +model = cp_model.CpModel() +x = {i: model.NewBoolVar(f'x_{i}') for i in items} +# Objective +model.Minimize(sum(cost[i] * x[i] for i in items)) +# Constraints +for j in elements: + model.Add(sum(x[i] for i in coverage[j]) >= 1) +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 4 +status = solver.Solve(model) +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + selected = [i for i in items if solver.Value(x[i]) == 1] + total_cost = solver.ObjectiveValue() + verified = all(any(solver.Value(x[i]) == 1 for i in coverage[j]) for j in elements) +``` + +### Common Pitfalls +- Confusing MIP and CP-SAT status codes or solution value access methods, leading to runtime errors. +- Setting an overly restrictive time limit that prevents finding a feasible solution for larger instances. +- Neglecting to implement a fallback solver strategy if the primary solver fails or is unavailable. diff --git a/skills/optskills/skill_library/weighted_set_cover_optimization_2.md b/skills/optskills/skill_library/weighted_set_cover_optimization_2.md new file mode 100644 index 0000000..5c0f782 --- /dev/null +++ b/skills/optskills/skill_library/weighted_set_cover_optimization_2.md @@ -0,0 +1,273 @@ +--- +name: Weighted Set Cover Optimization +description: | + Model and solve weighted set cover problems using binary selection variables, coverage constraints, and weighted sum minimization, with implementation options for both CP-SAT and MIP solvers. +--- + +# Workflow 1 (CP-SAT / OR-Tools) + +## Modeling stage + +### Strategy Overview +Formulate the weighted set cover problem as a binary integer program using the OR-Tools CP-SAT solver interface. This approach is efficient for pure 0-1 problems and leverages the solver's native constraint propagation. + +### Step 1 - Define Data Structures +- Represent the problem using two core dictionaries: one mapping each selectable item to its cost, and another mapping each element that must be covered to the list of items that can cover it. +- Use consistent, hashable keys (e.g., integers or strings) for both items and elements. + +### Step 2 - Create Binary Variables +- Instantiate a `CpModel`. +- Create one binary (BoolVar) decision variable for each selectable item. A value of 1 indicates the item is selected. + +### Step 3 - Formulate the Objective +- Define the objective to minimize the total weighted cost: the sum over all items of `(cost[item] * binary_variable[item])`. + +### Step 4 - Add Coverage Constraints +- For each element requiring coverage, add a linear constraint: the sum of the binary variables for all items that cover that element must be greater than or equal to 1. + +### Formulation Template +```json +{ + "sets": [ + "I: Set of selectable items (e.g., facilities, crews).", + "J: Set of elements requiring coverage (e.g., locations, tasks)." + ], + "parameters": [ + "c_i: Cost/weight of selecting item i ∈ I.", + "cover_j: List of item indices i ∈ I that can cover element j ∈ J." + ], + "decision_variables": [ + "x_i ∈ {0, 1}: Binary variable indicating if item i is selected." + ], + "objective": { + "sense": "min", + "expression": "Σ_{i ∈ I} c_i * x_i" + }, + "constraints": [ + "Coverage: For each j ∈ J, Σ_{i ∈ cover_j} x_i ≥ 1." + ] +} +``` + +### Common Pitfalls +- Inefficiently iterating over all items for each coverage constraint. Use the precomputed `cover_j` list for direct access. +- Forgetting to set `relative_gap_limit = 0.0` when an exact optimal solution is required, which may allow early termination with a gap. +- Using float costs with CP-SAT; scale to integers if necessary for exact arithmetic. + +## Solving stage + +### Strategy Overview +Configure and run the CP-SAT solver with parameters for timelimit, parallelism, and optimality guarantee. Extract the solution, verify its feasibility, and output structured results. + +### Step 1 - Configure Solver Parameters +- Set a `max_time_in_seconds` to prevent indefinite runs. +- Specify `num_search_workers` for parallel search (often equal to available CPU cores). +- Set `random_seed` for reproducibility. +- Enforce exact optimization by setting `relative_gap_limit = 0.0`. + +### Step 2 - Solve and Check Status +- Call `solver.Solve(model)`. +- Check if the status is `OPTIMAL` or `FEASIBLE`. Handle `INFEASIBLE` or `UNKNOWN` statuses appropriately. + +### Step 3 - Extract and Verify Solution +- Collect all items where the solver's value of the binary variable equals 1. +- Compute the total objective value from selected items. +- Programmatically verify that every element is covered by at least one selected item as a sanity check. + +### Step 4 - Output Structured Results +- Package the results (status, objective value, selected items, verification flag) into a JSON object. +- Print the JSON with a consistent prefix (e.g., `RESULT_JSON:`) for automated parsing. + +### Code Usage +```python +from ortools.sat.python import cp_model +import json + +# --- Data Placeholders --- +# costs = {item_id: cost_value, ...} +# coverage = {element_id: [item_ids], ...} + +model = cp_model.CpModel() +x = {i: model.NewBoolVar(f"x_{i}") for i in costs} + +# Objective +model.Minimize(sum(costs[i] * x[i] for i in costs)) + +# Constraints +for elem, covering_items in coverage.items(): + model.Add(sum(x[item] for item in covering_items) >= 1) + +# Solver Setup +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30 +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 +solver.parameters.relative_gap_limit = 0.0 + +# Solve +status = solver.Solve(model) + +# Result Processing +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + selected = [i for i in costs if solver.Value(x[i]) == 1] + total_cost = sum(costs[i] for i in selected) + # Verification + coverage_ok = all( + any(solver.Value(x[item]) == 1 for item in coverage[elem]) + for elem in coverage + ) + result = { + "status": "optimal" if status == cp_model.OPTIMAL else "feasible", + "objective": float(total_cost), + "selected_items": selected, + "coverage_verified": coverage_ok + } +else: + result = {"status": "failed", "reason": "infeasible_or_timeout"} +print(f"RESULT_JSON:{json.dumps(result)}") +``` + +### Common Pitfalls +- Not checking for both `OPTIMAL` and `FEASIBLE` statuses, potentially discarding good feasible solutions when time limits are hit. +- Assuming variable values are integers; always use `solver.Value()` to query the solution. +- Omitting solution verification, which can mask subtle solver or modeling errors. + +# Workflow 2 (Pyomo with MIP Solver) + +## Modeling stage + +### Strategy Overview +Model the problem using Pyomo's abstract modeling components (Sets, Params, Vars), creating a portable MILP formulation. This decouples the model logic from the solver, allowing easy switching between open-source (HiGHS, CBC) and commercial (Gurobi) solvers. + +### Step 1 - Define Pyomo Sets and Parameters +- Create a `ConcreteModel`. +- Define `Set` objects for items and elements. +- Define a `Param` for costs, indexed by the item set. +- Define a binary `Param` for the coverage relationship, indexed by (element, item), initialized via a rule or dictionary. + +### Step 2 - Declare Binary Variables +- Create a `Var` indexed by the item set, with `domain=pyo.Binary`. + +### Step 3 - Formulate the Objective +- Define an `Objective` to minimize the sum of `cost[item] * variable[item]` over all items. + +### Step 4 - Implement Coverage Constraints +- Define a `Constraint` indexed by the element set. +- The rule for each constraint returns the expression: sum over items of `(coverage[element, item] * variable[item]) >= 1`. + +### Formulation Template +```json +{ + "sets": [ + "I: Set of selectable items.", + "J: Set of elements requiring coverage." + ], + "parameters": [ + "c_i: Cost of item i ∈ I.", + "a_ji: Binary parameter, 1 if item i covers element j ∈ J, else 0." + ], + "decision_variables": [ + "x_i ∈ {0, 1}: Binary selection variable for item i." + ], + "objective": { + "sense": "min", + "expression": "Σ_{i ∈ I} c_i * x_i" + }, + "constraints": [ + "Coverage: For each j ∈ J, Σ_{i ∈ I} a_ji * x_i ≥ 1." + ] +} +``` + +### Common Pitfalls +- Initializing the large, sparse coverage parameter `a_ji` inefficiently. Use a rule that checks membership in a precomputed dictionary `cover[j]` to avoid dense storage. +- Using 1-based indexing from the problem data without converting to 0-based for internal use, causing index errors. +- Defining constraints with a `rule` function that has incorrect signature or side effects. + +## Solving stage + +### Strategy Overview +Use Pyomo's `SolverFactory` to interface with a chosen MIP solver. Configure solver options for time limit and optimality tolerance. Robustly check termination status, extract the solution, and verify feasibility. + +### Step 1 - Select and Configure Solver +- Instantiate a solver via `SolverFactory("solver_name")` (e.g., "highs", "cbc", "gurobi"). +- Set key options: `time_limit`, `mip_rel_gap` (set to 0.0 for exact optimality), and `threads`. + +### Step 2 - Solve and Inspect Termination +- Call `solver.solve(model, tee=False)` (set `tee=True` for log output). +- Check the solver status (`SolverStatus.ok`) and the termination condition (`TerminationCondition.optimal` or `.feasible`). + +### Step 3 - Extract Solution Values +- If the solve was successful, iterate over the item set and collect indices where `pyo.value(variable[item]) > 0.5`. +- Compute the objective value using `pyo.value(model.obj)`. + +### Step 4 - Verify and Output Results +- Re-evaluate each coverage constraint with the selected solution to ensure numerical feasibility. +- Assemble results into a structured dictionary and output as JSON. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition +import json + +# --- Data Placeholders --- +# items = [item_ids] +# elements = [element_ids] +# cost_dict = {item_id: cost_value} +# coverage_dict = {element_id: [item_ids]} + +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=items) +model.J = pyo.Set(initialize=elements) + +model.c = pyo.Param(model.I, initialize=cost_dict) + +def coverage_rule(m, j, i): + return 1 if i in coverage_dict.get(j, []) else 0 +model.a = pyo.Param(model.J, model.I, initialize=coverage_rule) + +model.x = pyo.Var(model.I, domain=pyo.Binary) + +model.obj = pyo.Objective( + expr=sum(model.c[i] * model.x[i] for i in model.I), + sense=pyo.minimize +) + +def cover_con_rule(m, j): + return sum(m.a[j, i] * m.x[i] for i in m.I) >= 1 +model.cover_con = pyo.Constraint(model.J, rule=cover_con_rule) + +# Solve +solver = pyo.SolverFactory("highs") # or "cbc", "gurobi" +solver.options["time_limit"] = 30 +solver.options["mip_rel_gap"] = 0.0 +results = solver.solve(model, tee=False) + +# Process Results +status = results.solver.status +term = results.solver.termination_condition + +if status == SolverStatus.ok and term in (TerminationCondition.optimal, TerminationCondition.feasible): + selected = [i for i in model.I if pyo.value(model.x[i]) > 0.5] + obj_val = pyo.value(model.obj) + # Verification + verified = all( + sum(pyo.value(model.a[j, i]) * pyo.value(model.x[i]) for i in model.I) >= 0.999 + for j in model.J + ) + result = { + "status": "optimal" if term == TerminationCondition.optimal else "feasible", + "objective": float(obj_val), + "selected_items": selected, + "coverage_verified": verified + } +else: + result = {"status": "failed", "reason": f"solver_status: {status}, termination: {term}"} +print(f"RESULT_JSON:{json.dumps(result)}") +``` + +### Common Pitfalls +- Not checking both `solver.status` and `termination_condition`, leading to misinterpretation of results (e.g., `ok` status with `infeasible` termination). +- Using a loose tolerance (e.g., `> 0.5`) for binary variable value extraction without considering solver integrality tolerances. +- Forgetting to scale objective values if costs were scaled to integers for the solver, resulting in incorrect reported costs. diff --git a/skills/optskills/skill_library/weighted_set_cover_solver.md b/skills/optskills/skill_library/weighted_set_cover_solver.md new file mode 100644 index 0000000..befac01 --- /dev/null +++ b/skills/optskills/skill_library/weighted_set_cover_solver.md @@ -0,0 +1,236 @@ +--- +name: Weighted Set Cover Solver +description: | + Model and solve weighted set covering problems with binary selection variables, coverage constraints, and fixed-cost minimization using either CP-SAT or MILP frameworks. +--- + +# Workflow 1 (CP-SAT for Exact Binary Optimization) + +## Modeling stage + +### Strategy Overview +Use OR-Tools CP-SAT, a constraint programming solver optimized for binary integer problems. Ideal for pure set covering where all variables are Boolean and an exact optimal solution is required. + +### Step 1 - Define Problem Structure +- Identify the collection of selectable items (sets) and the elements that must be covered. +- Precompute coverage mapping: for each element, generate a list of all items that can cover it (e.g., based on a distance threshold or adjacency matrix). This reduces constraint size and improves solver performance. +- Define a fixed cost for selecting each item. +- **Prerequisite Check**: Validate that every element has at least one covering item; if any element has an empty covering list, the problem is infeasible and should be flagged before solving. + +### Step 2 - Build CP-SAT Model +- Instantiate a `CpModel` object. +- Create one binary decision variable (`NewBoolVar`) for each selectable item. +- Formulate the objective: minimize the sum of selection costs. +- Add coverage constraints: for each element, the sum of its covering variables must be at least 1. + +### Formulation Template +```json +{ + "sets": [ + "I: Set of selectable items (e.g., facilities, locations).", + "J: Set of elements that must be covered (e.g., zones, requirements)." + ], + "parameters": [ + "cost_i: Fixed cost of selecting item i ∈ I.", + "cover_j: Precomputed list of items i ∈ I that cover element j ∈ J." + ], + "decision_variables": [ + "x_i ∈ {0, 1}: 1 if item i is selected, 0 otherwise." + ], + "objective": { + "sense": "min", + "expression": "∑_{i ∈ I} cost_i * x_i" + }, + "constraints": [ + "Coverage: ∑_{i ∈ cover_j} x_i ≥ 1, ∀ j ∈ J" + ] +} +``` + +### Common Pitfalls +- Forgetting to map coverage for all elements, leading to missing constraints. +- Using integer variables instead of Boolean variables, which reduces solver efficiency. +- Not verifying the coverage mapping; an element with an empty covering list makes the problem infeasible. + +## Solving stage + +### Strategy Overview +Configure and run the CP-SAT solver with parameters for deterministic performance and time management. Extract and rigorously verify the solution against the original coverage requirements. + +### Step 1 - Configure Solver +- Create a `CpSolver` instance. +- Set a time limit (`max_time_in_seconds = [TIME_LIMIT]`) to prevent excessive runtime. +- Enable parallel search (`num_search_workers`) for speed. +- Set a random seed (`random_seed`) for reproducibility. +- Set the relative optimality gap to zero (`relative_gap_limit = 0.0`) for an exact solution. + +### Step 2 - Solve and Validate +- Call `solver.Solve(model)` and capture the status. +- If status is `OPTIMAL` or `FEASIBLE`, extract selected items where the variable value equals 1. +- **Mandatory Verification**: Programmatically verify that every element is covered by at least one selected item. +- Compute the total cost from the extracted solution for output consistency. +- **Confidence Check**: For small instances (e.g., ≤10 candidate items), consider performing an exhaustive search to confirm the solver's optimal solution. + +### Code Usage +```python +from ortools.sat.python import cp_model + +# 1. Define data (placeholders) +costs = {i: cost_value for i in items} # item -> cost +coverage = {j: [list_of_covering_items] for j in elements} # element -> precomputed covering items + +# 2. Build Model +model = cp_model.CpModel() +x = {i: model.NewBoolVar(f"x_{i}") for i in items} + +# Objective +model.Minimize(sum(costs[i] * x[i] for i in items)) + +# Coverage Constraints +for j in elements: + model.Add(sum(x[i] for i in coverage[j]) >= 1) + +# 3. Configure and Solve +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30.0 +solver.parameters.num_search_workers = 8 +solver.parameters.random_seed = 42 +solver.parameters.relative_gap_limit = 0.0 + +status = solver.Solve(model) + +# 4. Extract and Verify +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + selected = [i for i in items if solver.Value(x[i]) == 1] + total_cost = sum(costs[i] for i in selected) + + # Mandatory Verification + for j in elements: + if not any(i in selected for i in coverage[j]): + raise AssertionError(f"Element {j} is not covered.") + # Output results (e.g., as JSON) +``` + +### Common Pitfalls +- Not checking for `FEASIBLE` status, which may provide a valid but suboptimal solution if time runs out. +- Assuming solver status `OPTIMAL` guarantees coverage without explicit verification. +- Using floating-point arithmetic for cost summation in verification; use integer costs or exact types if possible. + +# Workflow 2 (Pyomo MILP with Open-Source Solver) + +## Modeling stage + +### Strategy Overview +Use Pyomo, an algebraic modeling language, to formulate the set cover as a Mixed-Integer Linear Program (MILP). Connects to solvers like HiGHS or CBC, offering flexibility and detailed model inspection. + +### Step 1 - Structure Model with Sets +- Define Pyomo `Set` objects for the indices of items and elements. +- Declare cost and coverage as parameters, typically using Python dictionaries. +- Create binary decision variables (`Var(domain=Binary)`) for each item. + +### Step 2 - Formulate Objective and Constraints +- Define the objective function as a `sum` of cost times variable. +- Implement coverage constraints via a `Constraint` rule that iterates over elements, ensuring the sum of relevant variables is at least 1. + +### Formulation Template +```json +{ + "sets": [ + "I: Pyomo Set of selectable items.", + "J: Pyomo Set of elements to cover." + ], + "parameters": [ + "cost: Pyomo Param or dict, cost[i] for i ∈ I.", + "cover: Dict, cover[j] = precomputed list of i ∈ I for each j ∈ J." + ], + "decision_variables": [ + "x[i] ∈ {0, 1}: Pyomo Var with domain=Binary." + ], + "objective": { + "sense": "min", + "expression": "sum(cost[i] * x[i] for i in I)" + }, + "constraints": [ + "Coverage: sum(x[i] for i in cover[j]) >= 1, for each j in J" + ] +} +``` + +### Common Pitfalls +- Defining Pyomo sets with incorrect initialization, leading to indexing errors. +- Using mutable default arguments (like `[]`) in constraint rule functions. +- Not separating model data from model structure, reducing reusability. + +## Solving stage + +### Strategy Overview +Instantiate a solver via Pyomo's `SolverFactory`, configure it for performance and exact solutions, solve, and then check termination conditions before extracting and validating results. + +### Step 1 - Configure and Execute Solver +- Create a solver object (e.g., `SolverFactory("highs")` or `SolverFactory("cbc")`). +- Set key options: time limit (`time_limit = [TIME_LIMIT]`), optimality gap (`mip_rel_gap = 0.0`), and threads (`threads`). +- Call `solver.solve(model, tee=False)` and capture the results object. + +### Step 2 - Process and Verify Solution +- Check that the solver status is `ok` and the termination condition is `optimal` or `feasible`. +- Extract selected items by iterating over variables where `value(x[i]) > 0.5`. +- **Mandatory Verification**: Verify coverage by checking each element against the selected items. +- Compute the objective value from the model or the results. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# 1. Define data (placeholders) +cost_data = {i: cost_value for i in items} +coverage_data = {j: [list_of_covering_items] for j in elements} + +# 2. Build Pyomo Concrete Model +model = pyo.ConcreteModel() +model.I = pyo.Set(initialize=items) +model.J = pyo.Set(initialize=elements) +model.cost = pyo.Param(model.I, initialize=cost_data) +model.x = pyo.Var(model.I, domain=pyo.Binary) + +# Objective +model.obj = pyo.Objective( + expr=sum(model.cost[i] * model.x[i] for i in model.I), + sense=pyo.minimize +) + +# Coverage Constraints +def cover_rule(m, j): + return sum(m.x[i] for i in coverage_data[j]) >= 1 +model.cover = pyo.Constraint(model.J, rule=cover_rule) + +# 3. Solve +solver = pyo.SolverFactory("highs") # or "cbc" +solver_options = { + "time_limit": 30, + "mip_rel_gap": 0.0, + "threads": 4 +} +results = solver.solve(model, options=solver_options, tee=False) + +# 4. Check Status and Extract +status_ok = results.solver.status == SolverStatus.ok +term_acceptable = results.solver.termination_condition in ( + TerminationCondition.optimal, TerminationCondition.feasible +) + +if status_ok and term_acceptable: + selected = [i for i in model.I if pyo.value(model.x[i]) > 0.5] + total_cost = pyo.value(model.obj) + + # Mandatory Verification + for j in model.J: + if not any(i in selected for i in coverage_data[j]): + raise AssertionError(f"Element {j} is not covered.") + # Output results (e.g., as JSON) +``` + +### Common Pitfalls +- Confusing solver status (`ok`) with termination condition (`optimal`); both must be checked. +- Using `pyo.value()` on an uninitialized variable if the solve failed. +- Not setting `mip_rel_gap = 0.0`, which may allow early stopping with a suboptimal solution. diff --git a/skills/optskills/skill_library/weighted_set_cover_solver_2.md b/skills/optskills/skill_library/weighted_set_cover_solver_2.md new file mode 100644 index 0000000..f49fd9c --- /dev/null +++ b/skills/optskills/skill_library/weighted_set_cover_solver_2.md @@ -0,0 +1,251 @@ +--- +name: Weighted Set Cover Solver +description: | + Model and solve weighted set cover problems by selecting subsets to cover all elements at minimum cost using binary variables, coverage constraints, and MIP solvers. +--- + +# Workflow 1 (Matrix-Based MIP with OR-Tools) + +## Modeling stage + +### Strategy Overview +Model the problem as a classic set cover integer program using a binary coverage matrix. This approach is efficient for dense coverage relationships and leverages direct coefficient setting in a low-level solver API. + +### Step 1 - Define Problem Data +- Identify the set of elements to be covered (e.g., projects, customers) and the set of selectable subsets (e.g., consultants, facilities). +- Define a cost parameter for each subset, typically as a list or array. +- Construct a binary coverage matrix where rows correspond to elements and columns correspond to subsets. An entry is 1 if the subset covers the element. + +### Step 2 - Create Binary Decision Variables +- Instantiate one binary decision variable for each selectable subset. The variable equals 1 if the subset is selected, 0 otherwise. +- Enforce binary nature by setting variable bounds to 0 and 1. + +### Step 3 - Formulate Coverage Constraints +- For each element, create a linear constraint requiring the sum of selected subsets that cover it to be at least 1. +- Use the coverage matrix to efficiently add coefficients only for non-zero entries. + +### Step 4 - Define Weighted Objective +- Formulate the objective as the minimization of the total cost, which is the weighted sum of the binary variables using the subset costs as coefficients. + +### Formulation Template +```json +{ + "sets": [ + "E: set of elements to cover", + "S: set of selectable subsets" + ], + "parameters": [ + "cost[s ∈ S]: weight/cost of selecting subset s", + "coverage[e ∈ E][s ∈ S]: binary parameter, 1 if subset s covers element e" + ], + "decision_variables": [ + "x[s ∈ S]: binary, 1 if subset s is selected" + ], + "objective": { + "sense": "min", + "expression": "sum_{s in S} cost[s] * x[s]" + }, + "constraints": [ + "Coverage for each element e in E: sum_{s in S} coverage[e][s] * x[s] >= 1" + ] +} +``` + +### Common Pitfalls +- Creating a dense constraint for every (element, subset) pair, even when coverage is 0, leading to unnecessary model bloat. +- Forgetting to verify that the coverage matrix correctly represents the problem's membership relationships. +- Using floating-point equality checks on binary variable solution values; always use a tolerance (e.g., > 0.5). + +## Solving stage + +### Strategy Overview +Solve the MIP model using the OR-Tools wrapper with the SCIP or CBC backend. This workflow provides fine-grained control over the solver and is suitable for prototyping and deployment in environments where commercial solvers are not available. + +### Step 1 - Initialize Solver and Variables +- Create a solver instance (e.g., `SCIP` or `CBC`). +- Declare the list of binary variables using the solver's integer variable method with bounds (0, 1). + +### Step 2 - Build Constraints from Matrix +- Iterate over each element. For each element, create a constraint with a lower bound of 1. +- Within the loop for each element, iterate over all subsets and use the coverage matrix to conditionally set the coefficient of the variable to 1. + +### Step 3 - Set Objective and Solve +- Define the objective function, set all cost coefficients, and specify minimization. +- Set practical solver parameters like a time limit and number of threads. +- Call the solver's `Solve()` method. + +### Step 4 - Extract and Validate Solution +- Check the solver status for `OPTIMAL` or `FEASIBLE`. +- Extract selected subsets by filtering variables with a solution value greater than 0.5. +- Retrieve the objective value as the total cost. +- Optionally, post-solve verification can be performed by checking the coverage of each element against the selected subsets. + +### Code Usage +```python +from ortools.linear_solver import pywraplp + +# 1. Define data (placeholders) +# elements = range(num_elements) +# subsets = range(num_subsets) +# costs = [...] # cost per subset +# coverage = [...] # binary matrix [element][subset] + +solver = pywraplp.Solver.CreateSolver("SCIP") +if not solver: + raise Exception("Solver not available.") + +# 2. Create variables +x = [solver.IntVar(0, 1, f"x_{s}") for s in subsets] + +# 3. Add coverage constraints +for e in elements: + constraint = solver.Constraint(1, solver.infinity()) + for s in subsets: + if coverage[e][s] == 1: + constraint.SetCoefficient(x[s], 1) + +# 4. Set objective +objective = solver.Objective() +for s in subsets: + objective.SetCoefficient(x[s], costs[s]) +objective.SetMinimization() + +# 5. Configure and solve +solver.SetTimeLimit(30000) # milliseconds +solver.SetNumThreads(4) +status = solver.Solve() + +# 6. Extract results +if status in (solver.OPTIMAL, solver.FEASIBLE): + selected = [s for s in subsets if x[s].solution_value() > 0.5] + total_cost = objective.Value() + # Output or return results +else: + # Handle no solution found + pass +``` + +### Common Pitfalls +- Not checking if the solver backend is available, causing runtime errors. +- Misinterpreting the solver status codes; `FEASIBLE` is acceptable for a satisficing solution. +- Ignoring the solver's time limit, which can lead to indefinite hangs on large instances. + +# Workflow 2 (Declarative Modeling with Pyomo and Gurobi) + +## Modeling stage + +### Strategy Overview +Model the problem declaratively using Pyomo's abstract or concrete modeling components. This approach separates problem definition from solver interaction, improves readability, and facilitates integration with high-performance commercial solvers like Gurobi. + +### Step 1 - Define Abstract Sets and Parameters +- Declare Pyomo `Set` objects for the elements and subsets. +- Define a `Param` for subset costs, indexed by the subset set. +- Define a binary `Param` for the coverage relationship, indexed by the element and subset sets. This can be initialized from a sparse data structure. + +### Step 2 - Declare Binary Variables and Objective +- Declare a `Var` object for the selection variables, indexed by subsets, with domain `Binary`. +- Define the objective as a `sum_product` of costs and variables, to be minimized. + +### Step 3 - Express Coverage Constraints Rule-Based +- Define a Pyomo `Constraint` list indexed by the element set. +- For each element, the constraint rule returns the sum of variables for subsets that cover that element, enforcing it to be >= 1. The rule uses the coverage parameter for lookup. + +### Formulation Template +```json +{ + "sets": [ + "model.E: Pyomo Set of elements", + "model.S: Pyomo Set of subsets" + ], + "parameters": [ + "model.cost: Pyomo Param, indexed by model.S", + "model.coverage: Pyomo Param, indexed by model.E x model.S, domain=Binary" + ], + "decision_variables": [ + "model.x: Pyomo Var, indexed by model.S, domain=Binary" + ], + "objective": { + "sense": "min", + "expression": "sum(model.cost[s] * model.x[s] for s in model.S)" + }, + "constraints": [ + "model.cover_rule: for each e in model.E, sum(model.coverage[e,s] * model.x[s] for s in model.S) >= 1" + ] +} +``` + +### Common Pitfalls +- Using concrete model initialization with large, dense data structures, which consumes excessive memory. Prefer sparse data formats. +- Incorrectly defining constraint rules that cause scope errors by not passing the model instance. +- Overlooking the need to pre-process data into the specific formats (e.g., dictionaries) required by Pyomo `Param` initialization. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the Gurobi solver via the `SolverFactory`. This workflow leverages commercial solver performance, advanced MIP tuning, and robust status reporting, suitable for production systems requiring high reliability and speed. + +### Step 1 - Instantiate Model and Load Data +- Create a concrete Pyomo model instance. +- Populate the sets and parameters with the problem-specific data, using dictionaries for efficient sparse data loading. + +### Step 2 - Configure and Execute Solver +- Create a solver object using `SolverFactory('gurobi')`. +- Set solver options such as time limit, optimality gap tolerance, thread count, and a random seed for reproducibility. +- Call the `solve` method on the model instance with the solver. + +### Step 3 - Inspect Solution Status +- Check the solver status (`SolverStatus.ok`) and termination condition (`TerminationCondition.optimal` or `.feasible`). +- Proceed only if the solution is valid. + +### Step 4 - Retrieve and Verify Solution +- Extract the objective function value using `pyo.value(model.obj)`. +- Iterate over the selection variables to collect indices where the variable value exceeds 0.5. +- Implement a verification function to confirm all elements are covered by the selected subsets. + +### Code Usage +```python +import pyomo.environ as pyo +from pyomo.opt import SolverStatus, TerminationCondition + +# 1. Create a concrete model and define data (placeholders) +model = pyo.ConcreteModel() +# model.S = pyo.Set(initialize=subsets_indices) +# model.E = pyo.Set(initialize=elements_indices) +# model.cost = pyo.Param(model.S, initialize=cost_dict) +# model.coverage = pyo.Param(model.E, model.S, initialize=coverage_dict, default=0) + +# 2. Declare variables and objective +model.x = pyo.Var(model.S, domain=pyo.Binary) +def obj_rule(model): + return sum(model.cost[s] * model.x[s] for s in model.S) +model.obj = pyo.Objective(rule=obj_rule, sense=pyo.minimize) + +# 3. Declare coverage constraints +def cover_rule(model, e): + return sum(model.coverage[e, s] * model.x[s] for s in model.S) >= 1 +model.cover = pyo.Constraint(model.E, rule=cover_rule) + +# 4. Solve +solver = pyo.SolverFactory('gurobi') +solver.options['TimeLimit'] = 30 +solver.options['MIPGap'] = 0.0 +solver.options['Threads'] = 4 +solver.options['Seed'] = 42 +results = solver.solve(model) + +# 5. Check status and extract solution +if (results.solver.status == SolverStatus.ok and + results.solver.termination_condition in (TerminationCondition.optimal, + TerminationCondition.feasible)): + total_cost = pyo.value(model.obj) + selected = [s for s in model.S if pyo.value(model.x[s]) > 0.5] + # Output or return results +else: + # Handle infeasible or error status + pass +``` + +### Common Pitfalls +- Assuming the solver is installed and licensed; always have a fallback plan (e.g., CBC). +- Not setting a `MIPGap` or time limit, which can cause the solver to run indefinitely on difficult instances. +- Failing to handle the case where the solver finds a feasible but not proven optimal solution, which is often acceptable for set cover problems. diff --git a/skills/optskills/skill_library/weighted_set_covering_with_cardinality_constraints.md b/skills/optskills/skill_library/weighted_set_covering_with_cardinality_constraints.md new file mode 100644 index 0000000..ec16e67 --- /dev/null +++ b/skills/optskills/skill_library/weighted_set_covering_with_cardinality_constraints.md @@ -0,0 +1,220 @@ +--- +name: Weighted Set Covering with Cardinality Constraints +description: | + Formulate and solve weighted set covering problems where each element must be covered by a minimum required number of selected sets, minimizing total selection cost. + +--- +# Workflow 1 (CP-SAT with OR-Tools) + +## Modeling stage + +### Strategy Overview +Model the problem as a binary integer program using the OR-Tools CP-SAT solver. This approach is ideal for pure 0-1 problems with linear constraints and objectives, leveraging a dedicated constraint programming solver for efficient search. + +### Step 1 - Define Selection Variables +- Create a binary decision variable for each available set. For example, `x[i] = model.NewBoolVar(f"x_{i}")` where `i` is the set identifier. +- These variables indicate whether a set is selected (1) or not (0). + +### Step 2 - Structure Coverage Requirements +- For each element `e` that requires coverage, identify the list of sets `cover_sets[e]` that can cover it. +- Add a linear constraint for each element: `model.Add(sum(x[i] for i in cover_sets[e]) >= required_coverage[e])`. This enforces the cardinality requirement. + +### Step 3 - Formulate the Objective +- Define the objective to minimize the total cost of selected sets: `model.Minimize(sum(cost[i] * x[i] for i in all_sets))`. + +### Formulation Template +```json +{ + "sets": ["S", "E"], + "parameters": [ + {"name": "cost_s", "domain": "S", "description": "Cost of selecting set s"}, + {"name": "cover_sets_e", "domain": "E", "description": "List of set IDs that cover element e"}, + {"name": "required_coverage_e", "domain": "E", "description": "Minimum number of selected sets required to cover element e"} + ], + "decision_variables": [ + {"name": "x_s", "domain": "S", "type": "binary", "description": "1 if set s is selected"} + ], + "objective": { + "sense": "min", + "expression": "sum(cost_s * x_s for s in S)" + }, + "constraints": [ + {"expression": "sum(x_s for s in cover_sets_e) >= required_coverage_e", "for_each": "e in E"} + ] +} +``` + +### Common Pitfalls +- Forgetting to ensure `cover_sets_e` lists are non-empty for each element `e`. +- Using integer variables instead of boolean variables for pure selection decisions, which reduces solver efficiency. +- Incorrectly indexing parameters or variables when populating the model from data dictionaries. + +## Solving stage + +### Strategy Overview +Solve the formulated model using the CP-SAT solver, configuring it for a balance of speed and proof of optimality. Post-solve, rigorously verify that the solution satisfies all coverage requirements. + +### Step 1 - Configure the Solver +- Instantiate the solver: `solver = cp_model.CpSolver()`. +- Set parameters: `solver.parameters.max_time_in_seconds = 30`, `solver.parameters.num_search_workers = 8` for parallelism, and `solver.parameters.random_seed = 42` for reproducibility. +- For an exact solution, ensure `solver.parameters.relative_gap_limit = 0.0`. + +### Step 2 - Solve and Check Status +- Execute the solve: `status = solver.Solve(model)`. +- Check if the status is `cp_model.OPTIMAL` or `cp_model.FEASIBLE`. Handle `cp_model.INFEASIBLE` or `cp_model.MODEL_INVALID` appropriately. + +### Step 3 - Extract and Verify Solution +- If a solution was found, extract selected sets: `selected_sets = [i for i in all_sets if solver.Value(x[i]) == 1]`. +- Perform verification: for each element `e`, calculate `sum(solver.Value(x[i]) for i in cover_sets[e])` and assert it is `>= required_coverage[e]`. +- Compute and report the total cost from the solution values. + +### Step 4 - Confirm Optimality (Optional) +- To prove optimality, add a constraint forcing a better objective: `model.Add(sum(cost[i] * x[i] for i in all_sets) <= best_cost - 1)`. +- Re-solve. If the status is `INFEASIBLE`, the previous solution is optimal. + +### Code Usage +```python +# build model from formulation +model = cp_model.CpModel() +x = {i: model.NewBoolVar(f"x_{i}") for i in all_sets} +# coverage constraints +for e in elements: + model.Add(sum(x[i] for i in cover_sets[e]) >= required_coverage[e]) +# objective +model.Minimize(sum(cost[i] * x[i] for i in all_sets)) + +# solve with status / termination checks +solver = cp_model.CpSolver() +solver.parameters.max_time_in_seconds = 30 +solver.parameters.num_search_workers = 8 +status = solver.Solve(model) + +if status in (cp_model.OPTIMAL, cp_model.FEASIBLE): + selected = [i for i in all_sets if solver.Value(x[i]) == 1] + # verification loop + for e in elements: + coverage_count = sum(solver.Value(x[i]) for i in cover_sets[e]) + assert coverage_count >= required_coverage[e] + print(f"RESULT:{solver.ObjectiveValue()}") +else: + print("RESULT_JSON:{\"status\": \"infeasible_or_error\"}") +``` + +### Common Pitfalls +- Not verifying the solution satisfies all constraints, especially when `status` is `FEASIBLE` but not `OPTIMAL`. +- Misinterpreting the `status` codes; `OPTIMAL` confirms proven optimality, `FEASIBLE` does not. +- Setting an overly restrictive time limit (`max_time_in_seconds`) before a feasible solution is found. + +# Workflow 2 (MILP with Pyomo and HiGHS) + +## Modeling stage + +### Strategy Overview +Model the problem as a Mixed-Integer Linear Program (MILP) using Pyomo's abstract or concrete model paradigm. This provides a declarative, solver-agnostic formulation that can interface with various MILP solvers like HiGHS. + +### Step 1 - Define Model and Index Sets +- Create a Pyomo `ConcreteModel()`. +- Define index sets `model.S = pyo.Set(initialize=all_sets)` and `model.E = pyo.Set(initialize=all_elements)`. + +### Step 2 - Declare Parameters +- Define cost parameter: `model.cost = pyo.Param(model.S, initialize=cost_dict)`. +- Define coverage requirement parameter: `model.req = pyo.Param(model.E, initialize=required_coverage_dict)`. +- Define coverage matrix parameter using a rule: `def cover_rule(m, s, e): return 1 if s in cover_sets_dict[e] else 0`, then `model.cover = pyo.Param(model.S, model.E, initialize=cover_rule)`. + +### Step 3 - Define Decision Variables and Constraints +- Create binary variables: `model.x = pyo.Var(model.S, domain=pyo.Binary)`. +- For each element, add a coverage constraint: `def coverage_rule(m, e): return sum(m.cover[s, e] * m.x[s] for s in m.S) >= m.req[e]`, then `model.coverage_con = pyo.Constraint(model.E, rule=coverage_rule)`. + +### Step 4 - Formulate the Objective +- Set the objective: `model.obj = pyo.Objective(expr=sum(m.cost[s] * m.x[s] for s in m.S), sense=pyo.minimize)`. + +### Formulation Template +```json +{ + "sets": ["S", "E"], + "parameters": [ + {"name": "cost_s", "domain": "S", "description": "Cost of selecting set s"}, + {"name": "req_e", "domain": "E", "description": "Required coverage count for element e"}, + {"name": "cover_s_e", "domain": ["S", "E"], "description": "1 if set s covers element e"} + ], + "decision_variables": [ + {"name": "x_s", "domain": "S", "type": "binary", "description": "Selection variable for set s"} + ], + "objective": { + "sense": "min", + "expression": "sum(cost_s * x_s for s in S)" + }, + "constraints": [ + {"expression": "sum(cover_s_e * x_s for s in S) >= req_e", "for_each": "e in E"} + ] +} +``` + +### Common Pitfalls +- Defining the coverage matrix parameter inefficiently for large, sparse problems; use a rule with dictionary lookups. +- Incorrectly specifying set or parameter domains, leading to model construction errors. +- Using `pyo.Param` without a default value for missing indices in sparse data. + +## Solving stage + +### Strategy Overview +Solve the Pyomo model using the HiGHS MILP solver via the `appsi_highs` interface or `solverfactory('highs')`. Configure for performance and rigorously check the solution status before extracting results. + +### Step 1 - Configure and Execute the Solver +- Instantiate the solver: `solver = pyo.SolverFactory('highs')`. +- Set solver options: `solver.options['time_limit'] = 30`, `solver.options['mip_rel_gap'] = 0.0` for optimality, and `solver.options['threads'] = 4`. +- Solve with `load_solutions=False`: `results = solver.solve(model, load_solutions=False, tee=True)`. + +### Step 2 - Check Solver Status and Load Solution +- Check termination condition: `if results.solver.termination_condition == pyo.TerminationCondition.optimal:`. +- If optimal or feasible, load the solution: `model.solutions.load_from(results)`. +- If not optimal, inspect `results.solver.status` and `results.solver.termination_condition` for diagnostics. + +### Step 3 - Extract and Verify Solution +- Extract selected sets: `selected_sets = [s for s in model.S if pyo.value(model.x[s]) > 0.5]`. +- Verify coverage: for each element `e`, compute `sum(pyo.value(model.cover[s, e]) * pyo.value(model.x[s]) for s in model.S)` and confirm it meets `pyo.value(model.req[e])`. +- Report the objective value: `pyo.value(model.obj)`. + +### Step 4 - Confirm Optimality via Bound Analysis +- Solve the LP relaxation (e.g., by temporarily changing variable domains to `pyo.Reals`). Compare its objective value to the MILP solution. If they match, the MILP solution is optimal. +- Alternatively, add an objective cut `sum(model.cost[s] * model.x[s] for s in model.S) <= current_obj - epsilon` and attempt to solve. Infeasibility confirms optimality. + +### Code Usage +```python +# build model from formulation +import pyomo.environ as pyo +model = pyo.ConcreteModel() +model.S = pyo.Set(initialize=all_sets) +model.E = pyo.Set(initialize=all_elements) +model.cost = pyo.Param(model.S, initialize=cost_dict) +model.req = pyo.Param(model.E, initialize=required_coverage_dict) +def cover_init(m, s, e): + return 1 if s in cover_sets_dict.get(e, []) else 0 +model.cover = pyo.Param(model.S, model.E, initialize=cover_init) +model.x = pyo.Var(model.S, domain=pyo.Binary) +def coverage_rule(m, e): + return sum(m.cover[s, e] * m.x[s] for s in m.S) >= m.req[e] +model.coverage_con = pyo.Constraint(model.E, rule=coverage_rule) +model.obj = pyo.Objective(expr=sum(m.cost[s] * m.x[s] for s in m.S), sense=pyo.minimize) + +# solve with status / termination checks +solver = pyo.SolverFactory('highs') +solver.options['time_limit'] = 30 +solver.options['mip_rel_gap'] = 0.0 +results = solver.solve(model, load_solutions=False) +if results.solver.termination_condition == pyo.TerminationCondition.optimal: + model.solutions.load_from(results) + selected = [s for s in model.S if pyo.value(model.x[s]) > 0.5] + # verification + for e in model.E: + coverage = sum(pyo.value(model.cover[s, e]) * pyo.value(model.x[s]) for s in model.S) + assert coverage >= pyo.value(model.req[e]) + print(f"RESULT:{pyo.value(model.obj)}") +else: + print(f"RESULT_JSON:{'status': 'solver_failed', 'termination_condition': str(results.solver.termination_condition)}") +``` + +### Common Pitfalls +- Forgetting to set `load_solutions=False` and then trying to access variable values before loading the results. +- Not checking both `solver.status` and `termination_condition`; a status of `ok` does not guarantee optimality. +- Using `pyo.value()` on variables or parameters before a solution has been loaded, resulting in `None` or default values.