Move site docs here for the new docs hub

.github/workflows/notify-docs.yml

@@ -0,0 +1,24 @@
+name: Notify bedbase docs
+
+# Tells databio/doc-hubs that this repo's docs changed, so docs.bedbase.org
+# rebuilds. All the logic is in the reusable workflow; this file only names
+# which site and which spoke. See databio/doc-hubs README, "Add a spoke".
+
+on:
+  push:
+    # Must equal this spoke's `ref` in hubs/bedbase/spokes.toml: the hub builds
+    # that ref, so notifying from any other branch would rebuild the same bytes.
+    branches: [master]
+    paths:
+      - 'docs/guide/**'                      # the spoke's docs tree, docs.yml included
+      - '.github/workflows/notify-docs.yml'  # so a notifier fix proves itself
+  workflow_dispatch:
+
+jobs:
+  notify:
+    uses: databio/actions/.github/workflows/notify-doc-hub.yml@master
+    with:
+      site: bedbase
+      spoke: bedhost
+    secrets:
+      DOCS_DISPATCH_TOKEN: ${{ secrets.DOCS_DISPATCH_TOKEN }}

docs/guide/README.md

@@ -0,0 +1,47 @@
+<h1 align="center">bedhost</h1>
+
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![Github badge](https://img.shields.io/badge/source-github-354a75?logo=github)](https://github.com/databio/bedhost)
+
+
+`bedhost` is a Python FastAPI module for the API that powers BEDbase.
+It needs a path to the *bedbase configuration file*, which can be provided either via `-c`/`--config` argument or read from `$BEDBASE_CONFIG` environment variable. 
+
+## Introduction
+
+You can find the formal OpenAPI documentation and interactive interface at <https://api.bedbase.org/v1/docs>. This document provides more conceptual introduction and explanations to how to use the API effectively.
+
+
+## General API organization
+
+### Data types
+
+BEDbase stores two types of data, which we call *records*. They are 1. BEDs, and 2. BEDsets. BEDsets are simply collections of BEDs. Each record in the database is either a BED or a BEDset.
+
+### Endpoint organization
+
+The endpoints are divided into 3 groups:
+
+1. `/bed` endpoints are used to interact with metadata for BED records.
+2. `/bedset` endpoints are used to interact with metadata for BEDset records.
+3. `/objects` endpoints are used to download metadata and get URLs to retrieve the underlying data itself. These endpoints implement the [GA4GH DRS standard](https://ga4gh.github.io/data-repository-service-schemas/).
+
+Therefore, to get information and statistics about BED or BEDset records, or what is contained in the database, look through the `/bed` and `/bedset` endpoints. But if you need to write a tool that gets the actual underlying files, then you'll need to use the `/objects` endpoints. The type of identifiers used in each case differ.
+
+## Record identifiers vs. object identifiers
+
+Each record has an identifier. For example, `0000120fe8c5334bb0ce759dfcf06c3b` is a BED identifier. You can use this identifier for the metadata endpoints. To download files, you'll need something slightly different -- you need an *object identifier*. This is because each BED record includes multiple files, such as the original BED file, the BigBed file, analysis plots, and so on. To download a file, you will construct what we call the `object_id`, which identifies the specific file.
+
+### How to construct object identifiers
+
+Object IDs take the form `<record_type>.<record_identifier>.<result_id>`. An example of an object_id for a BED file is `bed.0000120fe8c5334bb0ce759dfcf06c3b.bed_file`
+
+So, you can get information about this object like this:
+
+`GET` [https://api.bedbase.org/v1/objects/bed.0000120fe8c5334bb0ce759dfcf06c3b.bed_file](https://api.bedbase.org/v1/objects/bed.0000120fe8c5334bb0ce759dfcf06c3b.bed_file)
+
+Or, you can get a URL to download the actual file with:
+
+`GET` [https://api.bedbase.org/v1/objects/bed.0000120fe8c5334bb0ce759dfcf06c3b.bed_file/access/http](https://api.bedbase.org/v1/objects/bed.0000120fe8c5334bb0ce759dfcf06c3b.bed_file/access/http)
+
+

docs/guide/build_image.md

@@ -0,0 +1,38 @@
+# Build the container locally
+
+Running with `uvicorn` provides auto-reload. To configure, this assumes you have previously set up `databio/secrets`. 
+
+1. Source `.env` file to populate the environment variables referenced in the configuration file.
+2. Start `bedhost` using `uvicorn` and pass the configuration file via the `BEDBASE_CONFIG` env var.
+
+
+```console
+source environment/production.env
+BEDBASE_CONFIG=deployment/config/api.bedbase.org.yaml uvicorn bedhost.main:app --reload
+```
+
+You can change the database you're connecting to by using a different config file:
+- Using a local config: `BEDBASE_CONFIG=../bbconf/tests/data/config.yaml uvicorn bedhost.main:app --reload`
+- With the dev database: `BEDBASE_CONFIG=deployment/config/api-dev.bedbase.org.yaml uvicorn bedhost.main:app --reload`
+
+Now, you can access the service at [http://127.0.0.1:8000](http://127.0.0.1:8000). Example endpoints:
+- http://127.0.0.1:8000/v1/bed/bbad85f21962bb8d972444f7f9a3a932/metadata?full=true
+- http://127.0.0.1:8000/v1/bed/bbad85f21962bb8d972444f7f9a3a932/metadata/plots?full=true
+- http://127.0.0.1:8000/v1/objects/bed.bbad85f21962bb8d972444f7f9a3a932.chrombins
+- http://127.0.0.1:8000/v1/bed/list?limit=10&offset=0
+
+
+## Running the server in Docker
+
+### Building image
+
+- Primary image: `docker build -t databio/bedhost -f Dockerfile .`
+- Dev image `docker build -t databio/bedhost:dev -f dev.Dockerfile .`
+- Test image: `docker build -t databio/bedhost:dev -f test.Dockerfile .`
+
+Existing images can be found [at dockerhub](https://hub.docker.com/r/databio/bedhost).
+
+
+## Deploying updates automatically
+
+- [Deploying bedbase](./deployment.md).

docs/guide/changelog.md

@@ -0,0 +1,180 @@
+# Changelog
+
+This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) and [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) format. 
+
+## [0.13.0] -- 2026-08-20
+### Added:
+- `/v1/objects/exports` and `/v1/objects/files` endpoints listing published bulk-metadata exports and standalone analysis files [API]
+- `BEDHOST_INIT_ML` environment variable to skip loading ML models [API]
+### Changed:
+- Statistics cache moved to app state (thread safe), with a fallback for empty datasets [API]
+- Blocking endpoints run in a threadpool instead of stalling the event loop [API]
+- Missing markdown files return 404 instead of 500 [API]
+- Logging configured once, in `main.py`
+
+## [0.12.7] -- 2026-04-22
+### Changed:
+- UMAP page improvements and fixed link previews [UI]
+- Migrated to Starlette 1.0 and yacman 1.0
+
+## [0.12.6] -- 2026-04-16
+### Fixed:
+- Build fix
+
+## [0.12.5] -- 2026-04-15
+### Changed:
+- UI updates and search improvements
+
+## [0.12.4] -- 2026-02-06
+### Fixed:
+- Search by external id
+### Changed:
+- Updated UMAP UI [UI]
+
+## [0.12.3] -- 2026-01-21
+
+## [0.12.2] -- 2026-01-21
+### Added:
+- BED classifier and reference genome validator in the BED analyzer [UI]
+### Changed:
+- Updated home page [UI]
+
+## [0.12.1] -- 2025-12-22
+### Added:
+- Hybrid text search combining dense and sparse vector search [API]
+### Changed:
+- Backend improvements that reduce container size
+
+## [0.12.0] -- 2025-12-01
+### Added:
+- umap calculation [API]
+- BED-analyzer (qc), which is backed by gtars-wasm package [UI]
+- New improved home page [UI]
+- Added visualization of the UMAP, and interactive [UI]
+
+
+## [0.11.0] -- 2025-09-11
+### Added:
+- Added umap visualizations of bed embeddings
+- Added bedbase-verse metrics
+- Added filtering by assay and genome in the semantic search
+- Added bed file qc check before running bed to bed search
+
+### Changed:
+- Changed bivec search to text semantic search
+- Multiple UI bug fix
+
+
+## [0.10.0] -- 2025-04-21
+### Added:
+- Added usage statistics
+- Added Bed file statistic page
+
+### Changed:
+- Updated bed compliance and data formats
+
+
+## [0.9.0] -- 2025-01-03
+### Added:
+* Added a new class `CreateBEDsetRequest` in `bedhost/data_models.py` to handle BEDset creation requests.
+* Introduced a new API endpoint `/v1/bed/{bed_id}/neighbours` to get nearest neighbors for a BED record in `bedhost/routers/bed_api.py`.
+* Implemented a new API endpoint `/v1/bedset/create/` to create a new BEDset by providing a registry path to the PEPhub project in `bedhost/routers/bedset_api.py`.
+
+### Changed:
+* Refactored `text_to_bed_search` function to include additional logic for handling specific queries in `bedhost/routers/bed_api.py`.
+
+### UI improvements:
+* Added Most similar files table to bed page
+* Improved Mobile friendly ui to both bed and bedset page
+* Improved metadata tables on bed page
+* Added `Download pdf` button for plots
+* Improved search tables
+* Added creation of bedset UI 
+
+
+## [0.8.0] -- 2024-11-07
+
+### Added:
+- Added endpoint showing available genomes
+- Added endpoint listing bed_ids with missing plots
+
+## [0.7.0] -- 2024-10-23
+
+### Added:
+- New text2bed search (bivec search)
+- Added track_hub endpoints and pointing link
+- Added pep generating endpoint for bedsets
+
+## [0.6.0] -- 2024-10-15
+
+- Multiple ui improvements and fixes
+- Updated bed metadata endpoint: added `annotation` to metadata return model, with standard schema
+- Updated metadata in search endpoints.
+- Added embed endpoint. [#136](https://github.com/databio/bedhost/issues/136)
+
+
+## [0.5.0] -- 2024-06-11
+
+- Improved Bed search (speed and quality)
+- Added licenses
+- UI tweaks
+- Added universes and bed tokens to the database
+- Added embedding endpoint
+
+## [0.4.0] -- 2024-04-08
+
+- Support of new bbconf.
+- Updated endpoints.
+
+
+## [0.3.0] -- 2023-03-01
+
+- switch to pydantic2
+- updated requirements
+- updated docs
+
+
+## [0.2.0] -- 2023-10-17
+- remove all graphql
+- remove local static hosting of UI
+- update to new pipestat-based bbconf (pending)
+- major refactor of API that introduces backwards-incompatible changes
+
+## [0.1.3] -- 2023-09-01
+- allow all origins
+
+## [0.1.2] -- 2023-02-06
+### change
+- change `/bedset/my_bedset/file_paths`endpoint from GET to POST
+
+## [0.1.1] -- 2021-10-30
+### change
+- `/bed/genomes` and `bedset/genomes`: improve speed
+
+## [0.1.0] -- 2021-10-25
+### add
+- GraphQL endpoints
+### change
+- endpoints update due to `bbconf` and `pipestat` changes
+
+## [0.0.6] -- 2021-05-17
+### add
+- Add endpoints that serve:
+  - a list of genome assemblies in bedsets and bedfiles table
+  - bed files by search term(s)
+  - remote file path (http / s3)
+
+## [0.0.5] -- 2021-04-15
+### add
+- Add examples of API endpoints
+### fix
+- resolve `/about` page not found when typing/editing url in the address bar. 
+
+## [0.0.4] -- 2021-04-01
+### add
+- add endpoint for region-based query 
+### fix
+- construction of local file/img path
+
+## [0.0.3] -- 2021-02-22
+- Initial project release

docs/guide/deployment.md

@@ -0,0 +1,64 @@
+# Deploying bedbase.org
+
+This repository deploys the API for bedbase. It will run these services:
+
+1. production API: https://api.bedbase.org/
+2. dev API: https://api-dev.bedbase.org/
+
+This repo will deploy a new service by following these steps:
+
+1. Build an image by packaging the bedhost image (from dockerhub) with the bbconf file in this repository.
+2. Push that image to AWS.
+3. Deploy it to yeti cluster with aws task def.
+
+## Build the container
+
+Here we use the `databio/bedhost` container on dockerhub, and just add the configuration file in this repo to it, so build is super fast.
+
+```
+docker build -t databio/bedhost-configured -f deployment/Dockerfiles/primary.Dockerfile .
+```
+
+Or for dev:
+
+```
+docker build -t databio/bedhost-configured-dev -f deployment/Dockerfiles/dev1.Dockerfile .
+```
+
+## Run it locally to test
+
+First, source the .env file to set env vars in the calling environment.
+Then, use `--env-file` to pass those env vars through to the container
+
+```
+source environment/production.env
+docker run --rm --network="host" \
+  --env-file environment/docker.env \
+  databio/bedhost-configured-dev
+```
+
+Here's another example for running the container:
+
+```
+docker run --rm --init -p 8000:8000 --name bedstat-rest-server \
+  --network="host" \
+  --volume "$(pwd)/deployment/config/api.bedbase.org.yaml:/bedbase.yaml" \
+  --env-file environment/docker.env \
+  --env BEDBASE_CONFIG=/bedbase.yaml \
+  databio/bedhost  uvicorn bedhost.main:app --reload
+```
+
+## Building the Amazon-tagged version
+
+You could build and push to ECR like this if you need it... but the github action will do this for you.
+
+Authenticate with AWS ECR:
+```
+aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 235728444054.dkr.ecr.us-east-1.amazonaws.com
+```
+
+Build/tag/push image:
+```
+docker build -t 235728444054.dkr.ecr.us-east-1.amazonaws.com/bedhost -f deployment/Dockerfiles/primary.Dockerfile .
+docker push 235728444054.dkr.ecr.us-east-1.amazonaws.com/bedhost
+```

docs/guide/docs.yml

@@ -0,0 +1,6 @@
+site_name: BEDhost
+nav:
+  - BEDhost overview: README.md
+  - Building docker image: build_image.md
+  - Deploy API: deployment.md
+  - Changelog: changelog.md