From 5537ab7f1776a4efd90273836a0c4b5bbd604373 Mon Sep 17 00:00:00 2001
From: Badr Ouali <32390048+oualib@users.noreply.github.com>
Date: Fri, 3 Jul 2026 11:26:57 +0400
Subject: [PATCH 1/2] Update README with trailer video link and poster
Added a link to the trailer video with an image poster.
---
README.md | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/README.md b/README.md
index c1e00eb..063a1d0 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,5 @@
+
+
@@ -18,7 +20,12 @@
## Trailer Video
-▶ Watch the trailer
+
+
+ 
+
+
VAST Orbit is a Python library with scikit-learn-like functionality for conducting data science projects on data stored in VAST DataBase. Train models using familiar ``scikit-learn`` syntax and deploy them directly in the database, leveraging VAST's high-performance analytics capabilities. VAST Orbit offers robust support for the entire data science life cycle, uses a 'pipeline' mechanism to sequentialize data transformation operations, and provides beautiful graphical options.
@@ -551,4 +558,4 @@ Enjoy!
For contribution guidelines and additional support documentation, please refer to the project documentation.
-For questions and community support, join our Slack channel: https://vastsupport.slack.com
\ No newline at end of file
+For questions and community support, join our Slack channel: https://vastsupport.slack.com
From b59024314e02226f43553c9a9d2cbbb23a1fbdd3 Mon Sep 17 00:00:00 2001
From: Badr Ouali
Date: Fri, 3 Jul 2026 11:42:49 +0400
Subject: [PATCH 2/2] correcting the READMEs
---
README.md | 63 +++++++++---------------
docs/README.md | 130 +++++++++++++++++++++++++++++++++++++++++++------
2 files changed, 138 insertions(+), 55 deletions(-)
diff --git a/README.md b/README.md
index 063a1d0..b32d758 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,3 @@
-
-
@@ -9,10 +7,9 @@
# VAST Orbit
[](https://badge.fury.io/py/vastorbit)
-[](https://anaconda.org/conda-forge/vastorbit)
[](https://opensource.org/licenses/Apache-2.0)
[](https://www.python.org/downloads/)
-[](https://codecov.io/gh/vast-data/vastorbit)
+[](https://codecov.io/gh/vast-data/Orbit)
[](https://github.com/psf/black)
[](https://github.com/pylint-dev/pylint)
[](https://vast-data.github.io/Orbit/)
@@ -27,7 +24,7 @@
-VAST Orbit is a Python library with scikit-learn-like functionality for conducting data science projects on data stored in VAST DataBase. Train models using familiar ``scikit-learn`` syntax and deploy them directly in the database, leveraging VAST's high-performance analytics capabilities. VAST Orbit offers robust support for the entire data science life cycle, uses a 'pipeline' mechanism to sequentialize data transformation operations, and provides beautiful graphical options.
+VAST Orbit is a Python library with scikit-learn-like functionality for conducting data science projects on data stored in the VAST DataBase. Train models using familiar ``scikit-learn`` syntax and deploy them directly in the database, leveraging VAST's high-performance analytics capabilities. VAST Orbit offers robust support for the entire data science life cycle, uses a 'pipeline' mechanism to sequentialize data transformation operations, and provides beautiful graphical options.
## Table of Contents
- [Introduction](#introduction)
@@ -51,19 +48,18 @@ VAST DataBase combines enterprise-grade storage with powerful analytics capabili
Python has become the lingua franca of data science, offering unparalleled flexibility through its high-level abstraction and an extensive ecosystem of libraries. The accessibility of Python has led to the development of powerful APIs like pandas and scikit-learn, supported by a vibrant community of data scientists worldwide. Unfortunately, traditional Python tools operate in-memory as single-node processes, creating fundamental limitations when working with large-scale data. While distributed computing frameworks attempt to address these constraints, they still require moving data for processing—an approach that is prohibitively expensive and increasingly impractical in the modern data landscape. On top of these challenges, data scientists face additional complexity in deploying and operationalizing their models. The entire workflow is time-consuming and inefficient.
-**VAST Orbit solves these problems**. The concept is elegant: instead of moving data to compute, VAST Orbit brings the compute logic to where the data lives—in VAST DataBase. Train your models using familiar ``scikit-learn`` syntax in Python, then deploy them directly in the database for high-performance predictions at scale.
+**VAST Orbit solves these problems**. The concept is elegant: instead of moving data to compute, VAST Orbit brings the compute logic to where the data lives—in the VAST DataBase. Train your models using familiar ``scikit-learn`` syntax in Python, then deploy them directly in the database for high-performance predictions at scale.
### Main Advantages
- **Easy Data Exploration**: Interactive exploration of massive datasets without memory constraints
- **Fast Data Preparation**: Leverage Trino's distributed processing for rapid data transformation
- **Familiar Scikit-learn API**: Train models using the ``scikit-learn`` interface you already know
-- **In-Database Deployment**: Deploy trained models directly in VAST DataBase for production workloads _(with some current limitations — see [Project Status & Roadmap](#project-status--roadmap))_
+- **In-Database Deployment**: Deploy trained models directly in the VAST DataBase for production workloads _(with some current limitations — see [Project Status & Roadmap](#project-status--roadmap))_
- **Easy Model Evaluation**: Comprehensive model evaluation tools with visual insights
- **Seamless SQL Integration**: Use Python or SQL interchangeably based on your preference and use case
-
@@ -85,8 +81,8 @@ Your feedback directly shapes what we build next — see [Help and Support](#hel
To install VAST Orbit with pip:
```shell
-# Latest release version
-pip3 install vastorbit[all]
+# Latest release (beta): --pre is required until 1.0.0 is published
+pip3 install --pre vastorbit[all]
# Latest commit on main branch
pip3 install git+https://github.com/vast-data/Orbit.git@main
@@ -95,12 +91,12 @@ pip3 install git+https://github.com/vast-data/Orbit.git@main
To install VAST Orbit from source, run the following command from the root directory:
```shell
-python3 setup.py install
+pip install .
```
## Connecting to the Database
-VAST Orbit currently connects to VAST DataBase through Trino. Ensure you have Trino set up and configured to access your VAST DataBase instance.
+VAST Orbit currently connects to the VAST DataBase through Trino. Ensure you have Trino set up and configured to access your VAST DataBase instance.
Connection example:
@@ -121,7 +117,7 @@ Use the connection:
vo.connect("VAST_Connection")
```
-For more details on connection configuration, refer to the documentation.
+For more details on connection configuration, refer to the [documentation](https://vast-data.github.io/Orbit/).
## Documentation
@@ -146,14 +142,12 @@ VAST Orbit offers users the flexibility to customize their coding experience wit
Dark mode, ideal for extended coding sessions, features a sleek and stylish dark color scheme, providing a comfortable and eye-friendly environment.
-
On the other hand, Light mode serves as the default theme, offering a clean and bright interface for users who prefer a traditional coding ambiance.
-
@@ -187,7 +181,6 @@ SELECT version();
```
-
@@ -205,7 +198,6 @@ To create plots, simply provide the type of plot along with the SQL command.
```
-
@@ -225,19 +217,19 @@ selected_titanic.groupby(columns=["pclass"], expr=["AVG(survived) AS avg_survive
```
-
### Charts
-VAST Orbit comes integrated with two popular plotting libraries: matplotlib and plotly.
-
-A gallery of VAST Orbit-generated charts will be available in the documentation.
+VAST Orbit comes integrated with two popular plotting libraries: matplotlib and Plotly.
+A gallery of VAST Orbit-generated charts is available in the [documentation](https://vast-data.github.io/Orbit/chart_gallery.html).
-
-
+
+
+
### Complete Machine Learning Pipeline
@@ -288,7 +280,6 @@ iris_data.scatter(
```
-
@@ -302,7 +293,6 @@ titanic.corr(method="spearman")
```
-
@@ -321,13 +311,14 @@ titanic.corr(method="spearman", focus="survived")
```
-
#### Data Preparation
-VAST Orbit provides comprehensive data preparation capabilities including joining tables, encoding categorical variables, and filling missing values. Refer to the documentation for detailed examples.
+VAST Orbit provides comprehensive data preparation capabilities including joining
+tables, encoding categorical variables, and filling missing values. Refer to the
+[documentation](https://vast-data.github.io/Orbit/examples.html) for detailed examples.
Outlier Detection Example:
@@ -340,7 +331,6 @@ data.outliers_plot(columns="Heights")
```
-
@@ -374,7 +364,6 @@ cross_validate(
```
-
@@ -393,7 +382,6 @@ iris_data = load_iris()
```
-
@@ -406,7 +394,6 @@ iris_data = vo.VastFrame(input_relation="public.iris")
```
-
@@ -415,7 +402,8 @@ iris_data = vo.VastFrame(input_relation="public.iris")
Install the library with pip:
```shell
-pip3 install vastorbit[all]
+# beta: --pre required until 1.0.0
+pip3 install --pre vastorbit[all]
```
Create a new VAST connection through Trino:
@@ -454,7 +442,6 @@ vdf = load_titanic()
```
-
@@ -465,7 +452,6 @@ vdf.describe()
```
-
@@ -515,7 +501,6 @@ cross_validate(
```
-
@@ -534,7 +519,6 @@ model.features_importance()
```
-
@@ -546,7 +530,6 @@ model.roc_curve()
```
-
@@ -556,6 +539,6 @@ Enjoy!
## Help and Support
-For contribution guidelines and additional support documentation, please refer to the project documentation.
-
-For questions and community support, join our Slack channel: https://vastsupport.slack.com
+ - **Contributing:** see the [contribution guidelines](https://vast-data.github.io/Orbit/contribution_guidelines.html).
+ - **Documentation:** [vast-data.github.io/Orbit](https://vast-data.github.io/Orbit/).
+ - **Community:** join our Slack channel at [vastsupport.slack.com](https://vastsupport.slack.com).
\ No newline at end of file
diff --git a/docs/README.md b/docs/README.md
index 39abea2..66acc18 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -7,8 +7,11 @@ tables, and outputs in the rendered docs are real. That is why the build is
heavier than a typical Sphinx site — and why this folder ships a helper script,
`refresh.sh`, that wraps the whole pipeline.
+The published result of this build is served at
+**[vast-data.github.io/Orbit](https://vast-data.github.io/Orbit/)**.
+
This guide covers the one-time setup, the everyday build commands, where output
-and logs land, and how to diagnose a failed build.
+and logs land, how to publish to the website, and how to diagnose a failed build.
---
@@ -19,6 +22,7 @@ and logs land, and how to diagnose a failed build.
- [Build modes (the four gears)](#build-modes-the-four-gears)
- [Environment knobs](#environment-knobs)
- [Output](#output)
+- [Publishing to the website](#publishing-to-the-website)
- [Logs and diagnosing failures](#logs-and-diagnosing-failures)
- [What the post-processing scripts do](#what-the-post-processing-scripts-do)
- [Running the steps manually](#running-the-steps-manually)
@@ -63,13 +67,21 @@ pip install -r requirements.txt
reStructuredText; without it, every `.ipynb` page fails.
```shell
-apt install pandoc
+# Debian/Ubuntu
+sudo apt install pandoc
+
+# macOS (Homebrew)
+brew install pandoc
```
**5. Install `make`** (if you don't already have it):
```shell
-apt install make
+# Debian/Ubuntu
+sudo apt install make
+
+# macOS — ships with the Xcode Command Line Tools
+xcode-select --install
```
**6. Point the build at your docs directory.** Open `replace_sphinx_dir.py` and
@@ -98,7 +110,9 @@ Once set up, a full release-quality build is a single command from `docs/`:
That reinstalls the package, cleans previous output, executes every example, and
runs all the post-processing. It takes a while (everything is executed live). For
-day-to-day iteration you'll usually want one of the faster gears below.
+day-to-day iteration you'll usually want one of the faster gears below. When the
+build finishes and you want it live, see
+[Publishing to the website](#publishing-to-the-website).
---
@@ -161,7 +175,84 @@ build/html/
```
Open `build/html/index.html` in a browser. (`build` is the Makefile's `BUILDDIR`;
-`source` is the `SOURCEDIR`.)
+`source` is the `SOURCEDIR`.) To put this output online, see
+[Publishing to the website](#publishing-to-the-website).
+
+---
+
+## Publishing to the website
+
+The documentation is served by **GitHub Pages** from the `gh-pages` branch of
+`vast-data/Orbit`, at **[vast-data.github.io/Orbit](https://vast-data.github.io/Orbit/)**.
+
+We publish the **locally built** `build/html/` — we deliberately do *not* rebuild
+in CI, because the live-execution build (running every example against a database)
+is too heavy for a hosted runner. The tool is
+[`ghp-import`](https://github.com/c-w/ghp-import), which force-replaces the
+`gh-pages` branch with the contents of a folder.
+
+### One-time setup
+
+```shell
+pip install ghp-import
+```
+
+Then, in the repository once: **Settings → Pages → Source → Deploy from a branch
+→ `gh-pages` / `/ (root)`**. (The first `ghp-import` push creates the branch;
+set the Pages source after that.)
+
+### Publish
+
+From `docs/`, after a build:
+
+```shell
+./refresh.sh # produce build/html
+ghp-import -n -p -f build/html
+```
+
+- `-n` adds a `.nojekyll` file. **Required** — without it, Pages' Jekyll step
+ strips Sphinx's `_static/`, `_sources/`, and `_images/` folders and all the
+ CSS/JS/icons disappear.
+- `-p` pushes to `origin`.
+- `-f` force-overwrites the generated `gh-pages` branch (it is throwaway output,
+ regenerated on every publish; your source and history stay on `main`).
+
+The site updates after the Pages build completes — check **Actions → "pages build
+and deployment"** (~1–5 min). If the change doesn't show, it's almost always
+browser/CDN cache: hard-refresh (Ctrl/Cmd+Shift+R) or open in a private window.
+
+### Authentication
+
+`ghp-import -p` pushes over your `origin` remote, and GitHub no longer accepts
+account passwords over HTTPS. Use one of:
+
+- a **Personal Access Token** (paste it at the password prompt; on macOS cache it
+ with `git config --global credential.helper osxkeychain`), or
+- **SSH**: `git remote set-url origin git@github.com:vast-data/Orbit.git`.
+
+### Size limit (important)
+
+GitHub Pages hard-caps a published site at **1 GB**, and it cannot be raised on
+any plan. Because these docs bake in live-executed figures, the build can bloat
+fast. Keep heavy media **out** of `build/html`:
+
+- Don't ship the trailer video or large GIFs inside the site — host them on
+ **GitHub Releases** or a CDN and reference them by URL.
+- Check before publishing:
+
+ ```shell
+ du -sh build/html # must be < 1 GB
+ du -ah build/html | sort -rh | head -20 # find the biggest files
+ ```
+
+### Use one publish path
+
+Publish **either** via `ghp-import` (branch method, used here) **or** via a Pages
+Actions workflow (`actions/deploy-pages`) — never both. The Actions path enforces
+the 1 GB *artifact* cap and will time out on a heavy build; the branch method does
+not. Since we publish locally with `ghp-import`, keep **Settings → Pages → Source
+= Deploy from a branch**, and don't add a `deploy-pages` workflow (or disable it
+if one exists) so the two don't fight.
---
@@ -244,21 +335,25 @@ python3 reverse_replace_sphinx_dir.py
black ../.
```
-The finished site is in `build/html/`.
+The finished site is in `build/html/`. To publish it, see
+[Publishing to the website](#publishing-to-the-website).
---
## Troubleshooting
-**`vastorbit.__version__` is missing / metadata looks wrong.** Don't use an
-editable install (`pip install -e`) for the docs. Editable installs skip the real
-build step that generates the dynamic version (`setuptools_scm` / `_version.py`).
-`refresh.sh` deliberately does a regular `pip install ../.`.
+**Autodoc figures don't appear (or a `pip install -e` install misbehaves).** Don't
+use an editable install (`pip install -e`) for the docs. The build patches
+`SPHINX_DIRECTORY` inside the **installed** copy of `vastorbit` in `site-packages`
+so docstring figures resolve; an editable install points that "installed" copy
+back at your working tree, which changes the patch/reverse semantics and can leave
+your source tree in a patched state. `refresh.sh` deliberately does a regular,
+non-editable `pip install ../.`.
-**Autodoc figures don't appear.** The docstrings Sphinx reads come from the
-installed package, whose `SPHINX_DIRECTORY` placeholders must be patched. Run a
-`full` build (it patches the installed copy); `fast`/`file` rely on a previous
-`full` build having done so.
+**Autodoc figures still don't appear after a partial build.** The docstrings
+Sphinx reads come from the installed package, whose `SPHINX_DIRECTORY`
+placeholders must be patched. Run a `full` build (it patches the installed copy);
+`fast`/`file` rely on a previous `full` build having done so.
**`file` mode warns about a missing doctree cache.** Run a `full` (or `fast`)
build once first so `build/doctrees` exists; otherwise the single-file build
@@ -271,4 +366,9 @@ needs it to convert `.ipynb` files.
headless environment variables; if you call `make html` directly, export
`MPLBACKEND=Agg`, `PLOTLY_RENDERER=json`, and `BROWSER=true` yourself.
-**Interleaved or hard-to-read logs.** Build with `JOBS=1`.
\ No newline at end of file
+**Interleaved or hard-to-read logs.** Build with `JOBS=1`.
+
+**Published site is missing styling / shows a 404 / won't update.** See
+[Publishing to the website](#publishing-to-the-website): confirm `.nojekyll` is
+present (`ghp-import -n`), that the Pages build in **Actions** is green, that the
+build is under **1 GB**, and hard-refresh to clear CDN cache.
\ No newline at end of file