You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 632f559
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: control-operator/README.md
+27-15Lines changed: 27 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,12 @@
1
1
# operator
2
2
3
-
Folder with operators regarding Task and Environment deployment.
3
+
Folder with operators for Task and Environment deployment. For a better understanding of how the operators are used by ECS and the Control repository, see [docs/kubernetes_ecs.md](../docs/kubernetes_ecs.md).
4
4
5
5
## Description
6
6
7
-
In order to deploy Task and Environment workflows to the k8s cluster you need controllers and operators
8
-
controlling custom CRDs defining ALICE custom workload. This Folder defines and implements all moving parts together with Makefile
9
-
to build, deploy, install CRDs and operators.
7
+
In order to deploy Task and Environment workflows to the k8s cluster, you need controllers and operators
8
+
controlling the CRDs that define ALICE custom workloads. This folder defines and implements all the moving parts, together with a Makefile
9
+
to build, deploy, and install the CRDs and operators.
10
10
11
11
## Architecture
12
12
@@ -18,42 +18,54 @@ The operator is split into two separate binaries with different deployment strat
18
18
19
19
## Getting Started
20
20
21
-
You’ll need a Kubernetes cluster to run against. You can use [KIND](https://sigs.k8s.io/kind) to get a local cluster for testing, or run against a remote cluster. Author had the most success with K3s [see](/docs/kubernetes_ecs.md).
22
-
**Note:** Your controller will automatically use the current context in your kubeconfig (usually ~/.kube/config) file (i.e. whatever cluster `kubectl cluster-info` shows).
21
+
You’ll need a Kubernetes cluster to run against. You can use [KIND](https://sigs.k8s.io/kind) to get a local cluster for testing, or run against a remote cluster. The author had the most success with K3s ([see](/docs/kubernetes_ecs.md)).
22
+
**Note:** Your controller will automatically use the current context in your kubeconfig file (usually `~/.kube/config`) (i.e. whatever cluster `kubectl cluster-info` shows).
23
23
24
24
### Running on the cluster
25
25
26
-
Following commands show basic use of Makefile. However this isn't exhaustive list.
26
+
#### Namespaces
27
27
28
-
1. Install Instances of Custom Resources:
28
+
This project expects two namespaces: `alice-tasks` and `operator-system`. `alice-tasks` is where the actual `Task` resources run, and it must be created before the first Task is deployed. `operator-system` is where all operator pods run, and it is created during deployment to the cluster.
29
+
30
+
#### Secrets
31
+
32
+
When deploying operator containers, you are required to set up proper secrets to access the registry where the operator images reside. At the moment they are built by a GitHub pipeline and pushed to the AliceO2Group GitHub [package registry](https://github.com/orgs/AliceO2Group/packages).
33
+
34
+
The second required secret must be set up to access images used by `Task` CRs. At the moment these images are located in a GitLab registry, and the secret name is hardcoded in this repository as `gitlab-registry-secret`.
35
+
36
+
#### make commands
37
+
38
+
The following commands show basic usage of the Makefile. However, this isn't an exhaustive list.
39
+
40
+
- Install instances of Custom Resources:
29
41
30
42
```sh
31
43
kubectl apply -f config/samples/
32
44
```
33
45
34
-
1. Build and push your images. Default image tags are defined in the Makefile via `TASK_IMG` and `ENVIRONMENT_IMG`. Override them only if you want to use a different registry or tag:
46
+
- Build and push your images. Default image tags are defined in the Makefile via `TASK_IMG` and `ENVIRONMENT_IMG`. Override them only if you want to use a different registry or tag:
35
47
36
48
```sh
37
49
make docker-build docker-push TASK_IMG=<some-registry>/task-manager:tag ENVIRONMENT_IMG=<some-registry>/environment-manager:tag
38
50
```
39
51
40
-
1. Deploy the controllers to the cluster. Uses the same `TASK_IMG` and `ENVIRONMENT_IMG` defaults, override them if needed. Images are pulled using the secret named by `PULL_SECRET` (default `ghcr-pull`), which must exist in the `operator-system` namespace:
52
+
- Deploy the controllers to the cluster. Uses the same `TASK_IMG` and `ENVIRONMENT_IMG` defaults; override them if needed. Images are pulled using the secret named by `PULL_SECRET` (default `ghcr-pull`), which must exist in the `operator-system` namespace:
41
53
42
54
```sh
43
55
make deploy TASK_IMG=<some-registry>/task-manager:tag ENVIRONMENT_IMG=<some-registry>/environment-manager:tag PULL_SECRET=<secret-name>
44
56
```
45
57
46
58
### Uninstall CRDs
47
59
48
-
To delete the CRDs from the cluster:
60
+
To delete the CRDs from the cluster (careful: this also deletes all `Task`, `Environment` and `TaskTemplate` resources in the whole cluster):
49
61
50
62
```sh
51
63
make uninstall
52
64
```
53
65
54
66
### Undeploy controller
55
67
56
-
UnDeploy the controller from the cluster:
68
+
Undeploy the controllers from the cluster (careful: this deletes the whole `operator-system` namespace, the RBAC resources and the CRDs together with all `Task`, `Environment` and `TaskTemplate` resources; this also applies to `make undeploy-task` and `make undeploy-environment` on their own):
57
69
58
70
```sh
59
71
make undeploy
@@ -90,7 +102,7 @@ The task-manager requires a `NODE_NAME` environment variable to know which node
90
102
NODE_NAME=<your-node-name> make run-task
91
103
```
92
104
93
-
**NOTE:** You can also install CRDs and run in one step: `make install run-environment` or `make install run-task`
105
+
**NOTE:** You can also install the CRDs and run in one step: `make install run-environment` or `make install run-task`.
94
106
95
107
### Modifying the API definitions
96
108
@@ -100,9 +112,9 @@ If you are editing the API definitions, generate the manifests such as CRs or CR
100
112
make manifests
101
113
```
102
114
103
-
**NOTE:** Run `make --help` for more information on all potential`make` targets
115
+
**NOTE:** Run `make --help` for more information on all available`make` targets.
104
116
105
-
More information can be found via the [Kubebuilder Documentation](https://book.kubebuilder.io/introduction.html)
117
+
More information can be found in the [Kubebuilder Documentation](https://book.kubebuilder.io/introduction.html).
Copy file name to clipboardExpand all lines: docs/kubernetes_ecs.md
+5-2Lines changed: 5 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,10 @@ if you would do the installation with [`o2-flp-setup`](https://alice-flp.docs.ce
28
28
29
29
Any communication between kubernetes and OCC tasks and ECS and kubernetes is based on [operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/).
30
30
All operator pattern code implemented and expected to be used and deployed by ECS is located in the folder `control-operator`
31
-
with [README.md](/control-operator/README.md).
31
+
with [README.md](/control-operator/README.md).
32
+
33
+
> ⚠️ **Important**
34
+
> `control-operator/README.md` contains usefull commands how to build/test/deploy operator pattern code with included Makefile.
32
35
33
36
## Direct ECS <-> Kubernetes bridge
34
37
@@ -37,7 +40,7 @@ directly to the Kubernetes as well and it is up to Admin/User to decide where th
37
40
In order for Task to be deployed in Kubernetes cluster we need to have environment and task managers
38
41
running inside the cluster as these are responsible for managing Custom Resource Definitions of Task and Environment.
39
42
40
-
The bridge works as follows:
43
+
The bridge works as following:
41
44
42
45
1.**ECS core** (running outside the cluster) uses a typed Kubernetes client (`control-operator/pkg/client`) backed by `controller-runtime` to interact with the cluster.
43
46
2. When ECS deploys a task whose control mode is `kubernetes_direct` or `kubernetes_fairmq`, it creates a **custom Task object** and, if needed, an **custom Environment object** in the cluster instead of submitting a Mesos offer.
0 commit comments