diff --git a/README.md b/README.md index 2d421c1cb..6ef1c2ab7 100644 --- a/README.md +++ b/README.md @@ -1,435 +1,201 @@ -

- -

+

+ Unstructured +

-

-

API Announcement!

-

+# Unstructured API -We are thrilled to announce our newly launched [Unstructured API](https://docs.unstructured.io/api-reference/overview). While access to the hosted Unstructured API will remain free, API Keys are required to make requests. To prevent disruption, get yours [here](https://www.unstructured.io/#get-api-key) now and start using it today! Check out the [readme](https://github.com/Unstructured-IO/unstructured-api#--) here to get started making API calls.

+A self-hosted REST API for partitioning documents with the +[`unstructured`](https://github.com/Unstructured-IO/unstructured) open-source library. The API +detects the uploaded file type, selects the corresponding partitioner, and returns document +elements as JSON or CSV. -#### :rocket: Beta Feature: Chipper Model +This repository is for running the Partition Endpoint yourself. For Unstructured's managed API, +current platform features, and hosted quickstarts, see the +[Unstructured API documentation](https://docs.unstructured.io/api-reference/overview). -We are releasing the beta version of our Chipper model to deliver superior performance when processing high-resolution, complex documents. To start using the Chipper model in your API request, you can utilize the `hi_res` strategy. Please refer to the documentation [here](https://unstructured-io.github.io/unstructured/api.html#strategies). +## What it supports -As the Chipper model is in beta version, we welcome feedback and suggestions. For those interested in testing the Chipper model, we encourage you to connect with us on [Slack community](https://join.slack.com/t/unstructuredw-kbe4326/shared_invite/zt-1x7cgo0pg-PTptXWylzPQF9xZolzCnwQ). +- The partitionable document types supported by the pinned `unstructured[all-docs]` dependency. + See the current + [open-source supported file types](https://docs.unstructured.io/open-source/introduction/supported-file-types). +- PDF and image strategies: `auto`, `fast`, `hi_res`, and `ocr_only`. +- Optional table and image-block extraction, OCR language selection, page breaks, coordinates, + deterministic or UUID element IDs, and gzip uploads. +- `by_title` chunking, including overlap controls and optional original-element metadata. +- Single-file and multi-file requests, with JSON, CSV, or `multipart/mixed` responses. -
+The generated API reference lists the complete request schema. After starting the server, open +[http://localhost:8000/general/docs](http://localhost:8000/general/docs), or read the OpenAPI document at +[http://localhost:8000/general/openapi.json](http://localhost:8000/general/openapi.json). - - - +## Local quickstart -
+### Requirements +- Python 3.12 +- [`uv`](https://docs.astral.sh/uv/) +- The system dependencies required by the document types you plan to process. See the + [`unstructured` installation guide](https://docs.unstructured.io/open-source/installation/full-installation). ---- +Install the locked runtime and test dependencies, then start the development server: -

-

General Pre-Processing Pipeline for Documents

-

- -This repo implements a pre-processing pipeline for the following documents. Currently, the pipeline is capable of recognizing the file type and choosing the relevant partition function to process the file. - - -| Category | Document Types | -|-----------|-------------------------------| -| Plaintext | `.txt`, `.eml`, `.msg`, `.xml`, `.html`, `.md`, `.rst`, `.json`, `.rtf` | -| Images | `.jpeg`, `.png` | -| Documents | `.doc`, `.docx`, `.ppt`, `.pptx`, `.pdf`, `.odt`, `.epub`, `.csv`, `.tsv`, `.xlsx` | -| Zipped | `.gz` | - - -## :rocket: Unstructured API - -Try our hosted API! It's freely available to use with any of the filetypes listed above. This is the easiest way to get started. If you'd like to host your own version of the API, jump down to the [Developer Quickstart Guide](#developer-quick-start). - -``` - curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -H 'unstructured-api-key: ' \ - -F 'files=@sample-docs/family-day.eml' \ - | jq -C . | less -R -``` - -### Parameters - -#### Strategies - -Four strategies are available for processing PDF/Images files: `hi_res`, `fast`, `ocr_only` and `auto`. `fast` is the default `strategy` and works well for documents that do not have text embedded in images. - -On the other hand, `hi_res` is the better choice for PDFs that may have text within embedded images, or for achieving greater precision of [element types](https://unstructured-io.github.io/unstructured/getting_started.html#document-elements) in the response JSON. Please be aware that, as of writing, `hi_res` requests may take 20 times longer to process compared to the `fast` option. See the example below for making a `hi_res` request. - -``` - curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/layout-parser-paper.pdf' \ - -F 'strategy=hi_res' \ - | jq -C . | less -R -``` - -The `ocr_only` strategy runs the document through Tesseract for OCR. Currently, `hi_res` has difficulty ordering elements for documents with multiple columns. If you have a document with multiple columns that do not have extractable text, we recommend using the `ocr_only` strategy. Please be aware that `ocr_only` will fall back to another strategy if Tesseract is not available. - -For the best of all worlds, `auto` will determine when a page can be extracted using `fast` or `ocr_only` mode, otherwise it will fall back to `hi_res`. - -#### Hi Res model name - -The `hi_res` strategy supports different models, and the default is `detectron2onnx`. You can also specify `hi_res_model_name` parameter to run `hi_res` strategy with the chipper model while using the host API: - -``` - curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/layout-parser-paper.pdf' \ - -F 'strategy=hi_res' \ - -F 'hi_res_model_name=chipper' \ - | jq -C . | less -R -``` - -We also support models to be used locally, for example, `yolox`. Please refer to the `using-the-api-locally` section for more information on how to use the local API. - -#### OCR languages - -Note: This kwarg will eventually be deprecated. Please use `languages`. -You can also specify what languages to use for OCR with the `ocr_languages` kwarg. See the [Tesseract documentation](https://github.com/tesseract-ocr/tessdata) for a full list of languages and install instructions. OCR is only applied if the text is not already available in the PDF document. - -``` -curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/english-and-korean.png' \ - -F 'strategy=ocr_only' \ - -F 'ocr_languages=eng' \ - -F 'ocr_languages=kor' \ - | jq -C . | less -R -``` - -#### Languages - -You can also specify what languages to use for OCR with the `languages` kwarg. See the [Tesseract documentation](https://github.com/tesseract-ocr/tessdata) for a full list of languages and install instructions. OCR is only applied if the text is not already available in the PDF document. - -``` -curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/english-and-korean.png' \ - -F 'strategy=ocr_only' \ - -F 'languages=eng' \ - -F 'languages=kor' \ - | jq -C . | less -R -``` - -#### Coordinates - -When elements are extracted from PDFs or images, it may be useful to get their bounding boxes as well. Set the `coordinates` parameter to `true` to add this field to the elements in the response. - -``` - curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/layout-parser-paper.pdf' \ - -F 'coordinates=true' \ - | jq -C . | less -R +```bash +make install +make run-web-app ``` -#### Skip Table Extraction +The API listens on `http://localhost:8000`. Verify it is ready: -Currently, we provide support for enabling and disabling table extraction for all file types. Set parameter `skip_infer_table_types` to specify the document types that you want to skip table extraction with. By default, we enable table extraction -for all file types (`skip_infer_table_types=[]`). Again, please note that table extraction only works with `hi_res` strategy. For example, if you want to skip table extraction for images, you can pass a list with matching image file types: - -``` - curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/layout-parser-paper-with-table.jpg' \ - -F 'strategy=hi_res' \ - -F 'skip_infer_table_types=["jpg"]' \ - | jq -C . | less -R +```bash +curl --fail http://localhost:8000/healthcheck ``` -#### Encoding +Partition a sample document: -You can specify the encoding to use to decode the text input. If no value is provided, utf-8 will be used. - -``` -curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/fake-power-point.pptx' \ - -F 'encoding=utf_8' \ - | jq -C . | less -R +```bash +curl --request POST \ + --url http://localhost:8000/general/v0/general \ + --header 'accept: application/json' \ + --form 'files=@sample-docs/family-day.eml' ``` -#### Gzipped files - -You can send gzipped file and api will un-gzip it. +The response is a list of document elements: -``` -curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'gz_uncompressed_content_type=application/pdf' \ - -F 'files=@sample-docs/layout-parser-paper.pdf.gz' +```json +[ + { + "element_id": "db1ca22813f01feda8759ff04a844e56", + "text": "Hi All,", + "type": "UncategorizedText", + "metadata": { + "filename": "family-day.eml" + } + } +] ``` -If field `gz_uncompressed_content_type` is set, the API will use its value as content-type of all files -after uncompressing the .gz files that are sent in single batch. If not set, the API will use -various heuristics to detect the filetypes after uncompressing from .gz. +### Common options -#### XML Tags +The default PDF/image strategy is `auto`. For layout-aware partitioning, request `hi_res`: -When processing XML documents, set the `xml_keep_tags` parameter to `true` to retain the XML tags in the output. If not specified, it will simply extract the text from within the tags. - -``` -curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/fake-xml.xml' \ - -F 'xml_keep_tags=true' \ - | jq -C . | less -R +```bash +curl --request POST \ + --url http://localhost:8000/general/v0/general \ + --header 'accept: application/json' \ + --form 'files=@sample-docs/layout-parser-paper.pdf' \ + --form 'strategy=hi_res' ``` -#### Page Breaks - -For supported filetypes, set the `include_page_breaks` parameter to `true` to include `PageBreak` elements in the output. +Use `languages` to specify OCR languages. The older `ocr_languages` parameter remains available +for compatibility but is deprecated by the underlying library. +```bash +curl --request POST \ + --url http://localhost:8000/general/v0/general \ + --header 'accept: application/json' \ + --form 'files=@sample-docs/english-and-korean.png' \ + --form 'strategy=ocr_only' \ + --form 'languages=eng' \ + --form 'languages=kor' ``` -curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/layout-parser-paper-fast.pdf' \ - -F 'include_page_breaks=true' \ - | jq -C . | less -R -``` - -#### Unique element IDs +Chunk the output by document title. Original elements are included in chunk metadata by default; +set `include_orig_elements=false` when you want a smaller response: -By default, the element ID is a SHA-256 hash of the element text. This is to ensure that -the ID is deterministic. One downside is that the ID is not guaranteed to be unique. -Different elements with the same text will have the same ID, and there could also be hash collisions. -To use UUIDs in the output instead, set ``unique_element_ids=true``. Note: this means that the element IDs -will be random, so with every partition of the same file, you will get different IDs. -This can be helpful if you'd like to use the IDs as a primary key in a database, for example. - -``` -curl -X 'POST' \ - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/layout-parser-paper-fast.pdf' \ - -F 'unique_element_ids=true' \ - | jq -C . | less -R +```bash +curl --request POST \ + --url http://localhost:8000/general/v0/general \ + --header 'accept: application/json' \ + --form 'files=@sample-docs/layout-parser-paper-fast.pdf' \ + --form 'chunking_strategy=by_title' \ + --form 'max_characters=1500' \ + --form 'include_orig_elements=false' ``` +Return CSV instead of JSON: -#### Chunking Elements - -Use the `chunking_strategy` form-field to chunk text into larger or smaller elements. Defaults to `None` which performs no chunking. The available chunking strategies are `basic` and `by_title`. - -The `basic` strategy combines whole consecutive document elements to maximally fill chunks of `max_characters` length. A single element that by itself exceeds `max_characters` is divided into two or more chunks by text-splitting (on a word boundary). - -The `by_title` strategy has the same behaviors except document section boundaries are respected, meaning elements from two different sections never occur in the same chunk. A `Title` (section heading) element introduces a new section, hence the name. - - Additional Parameters (all optional): - - `max_characters` - The hard maximum chunk size. No chunk will exceed this length. Defaults to 500. - - `new_after_n_chars` - A chunk of this length or greater is considered "full" and will not receive an additional element, even if it would fit within `max_characters`. - This "soft-maximum" defaults to `max_characters`. - - `overlap` - Specifies the length of a string ("tail") to be drawn from each chunk and prefixed to the - next chunk as a context-preserving mechanism. By default, this only applies to split-chunks - where an oversized element is divided into multiple chunks by text-splitting. - - `overlap_all` - Default: `False`. When `True`, apply overlap between "normal" chunks formed from whole - elements and not subject to text-splitting. Use this with caution as it entails a certain - level of "pollution" of otherwise clean semantic chunk boundaries. - - `combine_under_n_chars` - Combines elements (for example a series of titles) until a section reaches a - length of n characters. Defaults to 500. Only operative for the "by_title" - strategy. - - `multipage_sections` - If True, sections can span multiple pages. Defaults to True. Only operative for - the "by_title" strategy. - - -``` -curl -X 'POST' - 'https://api.unstructured.io/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/layout-parser-paper-fast.pdf' \ - -F 'chunking_strategy=by_title' \ - | jq -C . | less -R +```bash +curl --request POST \ + --url http://localhost:8000/general/v0/general \ + --header 'accept: text/csv' \ + --form 'files=@sample-docs/family-day.eml' \ + --form 'output_format=text/csv' ``` -## Developer Quick Start - -* Using `pyenv` to manage virtualenv's is recommended - * Mac install instructions. See [here](https://github.com/Unstructured-IO/community#mac--homebrew) for more detailed instructions. - * `brew install pyenv-virtualenv` - * `pyenv install 3.12` - * Linux instructions are available [here](https://github.com/Unstructured-IO/community#linux). - - * Create a virtualenv to work in and activate it, e.g. for one named `document-processing`: - - `pyenv virtualenv 3.12 - unstructured-api`
- `pyenv activate unstructured-api` +## Docker -See the [Unstructured Quick Start](https://github.com/Unstructured-IO/unstructured#eight_pointed_black_star-quick-start) for the many OS dependencies that are required, if the ability to process all file types is desired. +### Run the published image -* Run `make install` -* Start a local jupyter notebook server with `make run-jupyter`
- **OR**
- just start the fast-API locally with `make run-web-app` +Multi-platform images for `amd64` and `arm64` are published to Quay: -#### Using the API locally - -After running `make run-web-app` (or `make docker-start-api` to run in the container), you can now hit the API locally at port 8000. The `sample-docs` directory has a number of example file types that are currently supported. - -For example: -``` - curl -X 'POST' \ - 'http://localhost:8000/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/family-day.eml' \ - | jq -C . | less -R +```bash +docker pull quay.io/unstructured-io/unstructured-api:latest +docker run --rm --name unstructured-api -p 8000:8000 \ + quay.io/unstructured-io/unstructured-api:latest ``` -The response will be a list of the extracted elements: -``` -[ - { - "element_id": "db1ca22813f01feda8759ff04a844e56", - "coordinates": null, - "text": "Hi All,", - "type": "UncategorizedText", - "metadata": { - "date": "2022-12-21T10:28:53-06:00", - "sent_from": [ - "Mallori Harrell " - ], - "sent_to": [ - "Mallori Harrell " - ], - "subject": "Family Day", - "filename": "family-day.eml" - } - }, -... -... -``` +Pin a release or commit tag instead of `latest` for reproducible deployments. -The output format can also be set to `text/csv` to get the data in csv format rather than json: -``` - curl -X 'POST' \ - 'http://localhost:8000/general/v0/general' \ - -H 'accept: application/json' \ - -H 'Content-Type: multipart/form-data' \ - -F 'files=@sample-docs/family-day.eml' \ - -F 'output_format="text/csv"' -``` +### Build locally -The response will be a list of the extracted elements in csv format: -``` -type,element_id,text,filename,sent_from,sent_to,subject,languages,filetype -UncategorizedText,db1ca22813f01feda8759ff04a844e56,"Hi All,",family-day.eml,['Mallori Harrell '],['Mallori Harrell '],Family Day,['eng'],message/rfc822 -NarrativeText,a663c393a5e143c01ef2bb5c98efa2c1,Get excited for our first annual family day! ,family-day.eml,['Mallori Harrell '],['Mallori Harrell '],Family Day,['eng'],message/rfc822 -NarrativeText,ce65ca3bef59957d3f1c2bab5725c82f,"There will be face painting, a petting zoo, funnel cake and more.",family-day.eml,['Mallori Harrell '],['Mallori Harrell '],Family Day,['eng'],message/rfc822 -NarrativeText,d7bcf988af9f06042d83e25c531e5744,Make sure to RSVP!,family-day.eml,['Mallori Harrell '],['Mallori Harrell '],Family Day,['eng'],message/rfc822 -Title,5550577db69c2c8aabcd90979698120a,Best.,family-day.eml,['Mallori Harrell '],['Mallori Harrell '],Family Day,['eng'],message/rfc822 -Title,ca1c571d993b6c1ed8ef56a06c16ba22,Mallori Harrell,family-day.eml,['Mallori Harrell '],['Mallori Harrell '],Family Day,['eng'],message/rfc822 -Title,d5b612de8cd918addd9569b0255b65b2,Unstructured Technologies,family-day.eml,['Mallori Harrell '],['Mallori Harrell '],Family Day,['eng'],message/rfc822 -Title,2e0b9e8ee04b9594a9c26d8535b818ff,Data Scientist,family-day.eml,['Mallori Harrell '],['Mallori Harrell '],Family Day,['eng'],message/rfc822 +```bash +make docker-build +make docker-start-api ``` -#### Parallel Mode for PDFs -As mentioned above, processing a pdf using `hi_res` is currently a slow operation. One workaround is to split the pdf into smaller files, process these asynchronously, and merge the results. You can enable parallel processing mode with the following env variables: - -* `UNSTRUCTURED_PARALLEL_MODE_ENABLED` - set to `true` to process individual pdf pages remotely, default is `false`. -* `UNSTRUCTURED_PARALLEL_MODE_URL` - the location to send pdf page asynchronously, no default setting at the moment. -* `UNSTRUCTURED_PARALLEL_MODE_THREADS` - the number of threads making requests at once, default is `3`. -* `UNSTRUCTURED_PARALLEL_MODE_SPLIT_SIZE` - the number of pages to be processed in one request, default is `1`. -* `UNSTRUCTURED_PARALLEL_RETRY_ATTEMPTS` - the number of retry attempts on a retryable error, default is `2`. (i.e. 3 attempts are made in total) - -Due to the overhead associated with file splitting, parallel processing mode is only recommended for the `hi_res` strategy. Additionally users of the official [Python client](https://github.com/Unstructured-IO/unstructured-python-client?tab=readme-ov-file#splitting-pdf-by-pages) can enable client-side splitting by setting `split_pdf_page=True`. - -#### Security -You may also set the optional `UNSTRUCTURED_API_KEY` env variable to enable request validation for your self-hosted instance of Unstructured. If set, only requests including an `unstructured-api-key` header with the same value will be fulfilled. Otherwise, the server will return a 401 indicating that the request is unauthorized. +## Server configuration -#### Controlling Server Load -Some documents will use a lot of memory as they're being processed. To mitigate OOM errors, the server will return a 503 if the host's available memory drops below 2GB. This is configured with the environment variable `UNSTRUCTURED_MEMORY_FREE_MINIMUM_MB`, which defaults to 2048. You can lower this value to reduce these messages, that is, allow the server to use more memory. Otherwise, you can set to 0 to fully remove this check. +| Variable | Default | Description | +|---|---:|---| +| `PORT` | `8000` | Port used by the container entrypoint. | +| `HOST` | `0.0.0.0` | Interface used by the container entrypoint. | +| `WORKERS` | `1` | Number of Uvicorn workers. | +| `UNSTRUCTURED_API_KEY` | unset | When set, requests must supply the same value in the `unstructured-api-key` header. | +| `UNSTRUCTURED_MEMORY_FREE_MINIMUM_MB` | `2048` | Reject new work with HTTP 503 below this amount of free memory. Set to `0` to disable the check. | +| `ALLOWED_ORIGINS` | unset | Comma-separated origins allowed by the optional CORS middleware. | +| `MAX_LIFETIME_SECONDS` | unset | Begin graceful shutdown after this many seconds: new requests are denied while in-flight requests finish, then the server exits (forcefully after at most 3600 seconds). Requires GNU `timeout`. | -#### Controlling server life time -By default server will run for indefinitely. To change that the `MAX_LIFETIME_SECONDS` environmental variable can be set. If server is run with this variable set, it will enter a graceful shutdown period after `MAX_LIFETIME_SECONDS` from its initialization. Graceful shutdown period lasts for up to 3600 seconds and during it: -- server denies any new requests - they're met with an empty response, -- server continues processing active requests and shuts down (ending graceful period) if all of them are processed. +### Parallel PDF mode (experimental) -After the graceful period is over if server is still running, it is shutdown forcefully, cancelling all active requests and sending empty responses to each of them. +Parallel mode splits PDFs into page ranges, sends those ranges to another Partition Endpoint, and +combines the results. It is most useful for `hi_res` workloads. -*Max lifetime requires gnu [timeout](https://www.gnu.org/software/coreutils/manual/html_node/timeout-invocation.html#timeout-invocation) to be installed, available by default on most linux systems. Downloadable on macOS as gtimeout with gnu coreutils.* +| Variable | Default | Description | +|---|---:|---| +| `UNSTRUCTURED_PARALLEL_MODE_ENABLED` | `false` | Set to `true` to enable remote page processing. | +| `UNSTRUCTURED_PARALLEL_MODE_URL` | unset | Partition Endpoint URL that receives page ranges. Required when parallel mode is enabled. | +| `UNSTRUCTURED_PARALLEL_MODE_THREADS` | `3` | Maximum concurrent remote requests. | +| `UNSTRUCTURED_PARALLEL_MODE_SPLIT_SIZE` | `1` | Pages per remote request. | +| `UNSTRUCTURED_PARALLEL_RETRY_ATTEMPTS` | `2` | Retries after the initial request for retryable errors. | -## :dizzy: Instructions for using the Docker image +Alternatively, the official +[Python client](https://github.com/Unstructured-IO/unstructured-python-client?tab=readme-ov-file#splitting-pdf-by-pages) +can split PDFs client-side with `split_pdf_page=True`. -The following instructions are intended to help you get up and running using Docker to interact with `unstructured-api`. -See [here](https://docs.docker.com/get-docker/) if you don't already have docker installed on your machine. +## Development -NOTE: we build multi-platform images to support both x86_64 and Apple silicon hardware. Docker pull should download the corresponding image for your architecture, but you can specify with `--platform` (e.g. --platform linux/amd64) if needed. - -We build Docker images for all pushes to `main`. We tag each image with the corresponding short commit hash (e.g. `fbc7a69`) and the application version (e.g. `0.5.5-dev1`). We also tag the most recent image with `latest`. To leverage this, `docker pull` from our image repository. - -```bash -docker pull downloads.unstructured.io/unstructured-io/unstructured-api:latest -``` - -Once pulled, you can launch the container as a web app on localhost:8000. +Install the locked development environment: ```bash -docker run -p 8000:8000 -d --rm --name unstructured-api downloads.unstructured.io/unstructured-io/unstructured-api:latest +make install ``` -You can pass in a PORT variable to run the server on a different port in the container. +Run the checks and test suite: ```bash -docker run -p 9500:9500 -d --rm --name unstructured-api -e PORT=9500 downloads.unstructured.io/unstructured-io/unstructured-api:latest +make check +make test ``` -## Security Policy - -See our [security policy](https://github.com/Unstructured-IO/pipeline-emails/security/policy) for -information on how to report security vulnerabilities. +See the +[Unstructured contribution guide](https://github.com/Unstructured-IO/unstructured/blob/main/CONTRIBUTING.md) +before opening a pull request. -## Learn more +## Security -| Section | Description | -|-|-| -| [Unstructured Community GitHub](https://github.com/Unstructured-IO/community) | Information about Unstructured.io community projects | -| [Unstructured GitHub](https://github.com/Unstructured-IO) | Unstructured.io open source repositories | -| [Company Website](https://unstructured.io) | Unstructured.io product and company info | +Report vulnerabilities through this repository's +[security policy](https://github.com/Unstructured-IO/unstructured-api/security/policy). -## :chart_with_upwards_trend: Analytics +## License -We’ve partnered with Scarf (https://scarf.sh) to collect anonymized user statistics to understand which features our community is using and how to prioritize product decision-making in the future. To learn more about how we collect and use this data, please read our [Privacy Policy](https://unstructured.io/privacy-policy). +Licensed under the [Apache License 2.0](LICENSE.md). diff --git a/img/email-screenshot.png b/img/email-screenshot.png deleted file mode 100644 index d78b2f745..000000000 Binary files a/img/email-screenshot.png and /dev/null differ diff --git a/img/unstructured_logo.png b/img/unstructured_logo.png deleted file mode 100644 index c5e36854e..000000000 Binary files a/img/unstructured_logo.png and /dev/null differ diff --git a/img/unstructured_logo.svg b/img/unstructured_logo.svg new file mode 100644 index 000000000..709901a63 --- /dev/null +++ b/img/unstructured_logo.svg @@ -0,0 +1,15 @@ + + + + + + + + + + + + + + +