Skip to content

Commit 632f559

Browse files
committed
[docs] k8s dpl documentation
1 parent a9977c0 commit 632f559

3 files changed

Lines changed: 32 additions & 52 deletions

File tree

‎control-operator/README.md‎

Lines changed: 27 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
# operator
22

3-
Folder with operators regarding Task and Environment deployment.
3+
Folder with operators for Task and Environment deployment. For a better understanding of how the operators are used by ECS and the Control repository, see [docs/kubernetes_ecs.md](../docs/kubernetes_ecs.md).
44

55
## Description
66

7-
In order to deploy Task and Environment workflows to the k8s cluster you need controllers and operators
8-
controlling custom CRDs defining ALICE custom workload. This Folder defines and implements all moving parts together with Makefile
9-
to build, deploy, install CRDs and operators.
7+
In order to deploy Task and Environment workflows to the k8s cluster, you need controllers and operators
8+
controlling the CRDs that define ALICE custom workloads. This folder defines and implements all the moving parts, together with a Makefile
9+
to build, deploy, and install the CRDs and operators.
1010

1111
## Architecture
1212

@@ -18,42 +18,54 @@ The operator is split into two separate binaries with different deployment strat
1818

1919
## Getting Started
2020

21-
You’ll need a Kubernetes cluster to run against. You can use [KIND](https://sigs.k8s.io/kind) to get a local cluster for testing, or run against a remote cluster. Author had the most success with K3s [see](/docs/kubernetes_ecs.md).
22-
**Note:** Your controller will automatically use the current context in your kubeconfig (usually ~/.kube/config) file (i.e. whatever cluster `kubectl cluster-info` shows).
21+
You’ll need a Kubernetes cluster to run against. You can use [KIND](https://sigs.k8s.io/kind) to get a local cluster for testing, or run against a remote cluster. The author had the most success with K3s ([see](/docs/kubernetes_ecs.md)).
22+
**Note:** Your controller will automatically use the current context in your kubeconfig file (usually `~/.kube/config`) (i.e. whatever cluster `kubectl cluster-info` shows).
2323

2424
### Running on the cluster
2525

26-
Following commands show basic use of Makefile. However this isn't exhaustive list.
26+
#### Namespaces
2727

28-
1. Install Instances of Custom Resources:
28+
This project expects two namespaces: `alice-tasks` and `operator-system`. `alice-tasks` is where the actual `Task` resources run, and it must be created before the first Task is deployed. `operator-system` is where all operator pods run, and it is created during deployment to the cluster.
29+
30+
#### Secrets
31+
32+
When deploying operator containers, you are required to set up proper secrets to access the registry where the operator images reside. At the moment they are built by a GitHub pipeline and pushed to the AliceO2Group GitHub [package registry](https://github.com/orgs/AliceO2Group/packages).
33+
34+
The second required secret must be set up to access images used by `Task` CRs. At the moment these images are located in a GitLab registry, and the secret name is hardcoded in this repository as `gitlab-registry-secret`.
35+
36+
#### make commands
37+
38+
The following commands show basic usage of the Makefile. However, this isn't an exhaustive list.
39+
40+
- Install instances of Custom Resources:
2941

3042
```sh
3143
kubectl apply -f config/samples/
3244
```
3345

34-
1. Build and push your images. Default image tags are defined in the Makefile via `TASK_IMG` and `ENVIRONMENT_IMG`. Override them only if you want to use a different registry or tag:
46+
- Build and push your images. Default image tags are defined in the Makefile via `TASK_IMG` and `ENVIRONMENT_IMG`. Override them only if you want to use a different registry or tag:
3547

3648
```sh
3749
make docker-build docker-push TASK_IMG=<some-registry>/task-manager:tag ENVIRONMENT_IMG=<some-registry>/environment-manager:tag
3850
```
3951

40-
1. Deploy the controllers to the cluster. Uses the same `TASK_IMG` and `ENVIRONMENT_IMG` defaults, override them if needed. Images are pulled using the secret named by `PULL_SECRET` (default `ghcr-pull`), which must exist in the `operator-system` namespace:
52+
- Deploy the controllers to the cluster. Uses the same `TASK_IMG` and `ENVIRONMENT_IMG` defaults; override them if needed. Images are pulled using the secret named by `PULL_SECRET` (default `ghcr-pull`), which must exist in the `operator-system` namespace:
4153

4254
```sh
4355
make deploy TASK_IMG=<some-registry>/task-manager:tag ENVIRONMENT_IMG=<some-registry>/environment-manager:tag PULL_SECRET=<secret-name>
4456
```
4557

4658
### Uninstall CRDs
4759

48-
To delete the CRDs from the cluster:
60+
To delete the CRDs from the cluster (careful: this also deletes all `Task`, `Environment` and `TaskTemplate` resources in the whole cluster):
4961

5062
```sh
5163
make uninstall
5264
```
5365

5466
### Undeploy controller
5567

56-
UnDeploy the controller from the cluster:
68+
Undeploy the controllers from the cluster (careful: this deletes the whole `operator-system` namespace, the RBAC resources and the CRDs together with all `Task`, `Environment` and `TaskTemplate` resources; this also applies to `make undeploy-task` and `make undeploy-environment` on their own):
5769

5870
```sh
5971
make undeploy
@@ -90,7 +102,7 @@ The task-manager requires a `NODE_NAME` environment variable to know which node
90102
NODE_NAME=<your-node-name> make run-task
91103
```
92104

93-
**NOTE:** You can also install CRDs and run in one step: `make install run-environment` or `make install run-task`
105+
**NOTE:** You can also install the CRDs and run in one step: `make install run-environment` or `make install run-task`.
94106

95107
### Modifying the API definitions
96108

@@ -100,9 +112,9 @@ If you are editing the API definitions, generate the manifests such as CRs or CR
100112
make manifests
101113
```
102114

103-
**NOTE:** Run `make --help` for more information on all potential `make` targets
115+
**NOTE:** Run `make --help` for more information on all available `make` targets.
104116

105-
More information can be found via the [Kubebuilder Documentation](https://book.kubebuilder.io/introduction.html)
117+
More information can be found in the [Kubebuilder Documentation](https://book.kubebuilder.io/introduction.html).
106118

107119
## License
108120

‎control-operator/ecs-manifests/kubernetes-ecs.md‎

Lines changed: 0 additions & 35 deletions
This file was deleted.

‎docs/kubernetes_ecs.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,10 @@ if you would do the installation with [`o2-flp-setup`](https://alice-flp.docs.ce
2828

2929
Any communication between kubernetes and OCC tasks and ECS and kubernetes is based on [operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/).
3030
All operator pattern code implemented and expected to be used and deployed by ECS is located in the folder `control-operator`
31-
with [README.md](/control-operator/README.md).
31+
with [README.md](/control-operator/README.md).
32+
33+
> ⚠️ **Important**
34+
> `control-operator/README.md` contains usefull commands how to build/test/deploy operator pattern code with included Makefile.
3235
3336
## Direct ECS <-> Kubernetes bridge
3437

@@ -37,7 +40,7 @@ directly to the Kubernetes as well and it is up to Admin/User to decide where th
3740
In order for Task to be deployed in Kubernetes cluster we need to have environment and task managers
3841
running inside the cluster as these are responsible for managing Custom Resource Definitions of Task and Environment.
3942

40-
The bridge works as follows:
43+
The bridge works as following:
4144

4245
1. **ECS core** (running outside the cluster) uses a typed Kubernetes client (`control-operator/pkg/client`) backed by `controller-runtime` to interact with the cluster.
4346
2. When ECS deploys a task whose control mode is `kubernetes_direct` or `kubernetes_fairmq`, it creates a **custom Task object** and, if needed, an **custom Environment object** in the cluster instead of submitting a Mesos offer.

0 commit comments

Comments
 (0)