Distribute the monitoring system

[cw75]

Currently, the monitoring system runs single node single threaded, which could be a bottleneck when we increase the scale of the workload. We should make it multi-threaded and distributed!

[Waterdrips]

Whats the current solution? There are K8s ways of including monitoring at a node and pod level if required that could be included