How Cellery Leverages OPA

Hasintha Indrajee
wso2-cellery
Published in
5 min readOct 29, 2019

--

Open Policy Agent (OPA) is a general-purpose policy engine gives you the ability to define fine-grained policy control at all levels of the stack, and more importantly, decouple the definition of policy from the enforcement of it. It’s widely adopted in cloud native space solutions which needs HTTP API Authorization, Data Filtering with Partial Evaluation, Remote Access and Kubernetes Admission Control.

Out of OPA capabilities HTTP API Authorization, is one of the widely used features which exposes an HTTP API for authorization queries. The queries contain input information for decision making where as the policies can be written and deployed in OPA engine with minimum effort. The policies are written using the language Rego.

Istio also makes use of the capabilities of OPA engine. Istio has a dedicated sidecar which runs in addition to the Istio Proxy/Envoy sidecar. When Istio Proxy receives API requests destined for the microservice, it checks with OPA to decide if the request should be allowed. A significant design decision in Istio is to store policies locally in the sidecar so that no hops add to the request flow. The only cost is the GRPC call from istio sidecar to OPA sidecar which is done through GRPC.

A similar mechanism is used in Cellery as well. All invocations of services from Cell to Cell (inter Cell) and within cell (intra Cell) are subjected to authorization checks using Open Policy Agent.

Well, the next question is “where does this OPA engine reside in Cell Architecture ?”. A Cell is an individual entity which acts independently of other cells. A Cell is meant to be developed, managed and maintained by a single team. Hence the policies should also be specific to a Cell as well. Another concern is having per service OPA engines adds an overhead in terms of deployment as well as managing components as well as policies.

Having gone through above facts and concerns, one may ask “What about an OPA engine per Cell ?”. Well, that is what exactly Cellery uses underneath. The fact that single OPA engine is used per Cell may lead to the question about the network hops which a request must go through to reach the single instance of OPA in order to evaluate authorization decision. Before answering this question, go through the below image which illustrates the request flow once the request is reached the Cell gateway. “hr” is a service which is hosted inside Cell.

Cellery Security Flow

A request which reaches a service inside a Cell (hr service) is intercepted by the Istio sidecar (Yes !!, Cellery uses Istio as a dependency). Sidecars can have sidecar filters which are invoked through GRPC. Cellery has such a sidecar filter per Cell in order to handle Cell security. We call it as STS with the meaning of Secure Token Service. In simple terms each request which flow within a Cellery mesh goes through the sidecar in corresponding STS.

OPA is a part of STS

Along with STS service, an OPA instance also runs in the same pod. Whenever a request is intercepted by the STS, the request is sent to OPA HTTP authorization API which is running in the localhost. Running it in the localhost makes the network overhead minimum and optimum.

Communication between STS and OPA

You can write your own policies and deploy them in Cell OPA instance using Cellery. Those rules are getting executed while the services are invoked.

Let’s go through an example.

In order to go through an example start a Cellery runtime, deploy TODO sample and make sure it’s working as expected. Once you confirm things are working as expected, get the configmaps which are in default namespace. There is a configmap with name todo-<instance_name> — sts-policy. In order to list all configmaps in default namespace execute the below command.

kubectl get configmapNAME                                              DATA   AGE
todo-celllatestbdy2-gateway-config 2 13s
todo-celllatestbdy2-mysql-db-init-sql-config 1 12s
todo-celllatestbdy2-sts-config 2 14s
todo-celllatestbdy2-sts-policy 1 13s

Let’s try to investigate what’s there in todo-celllatestbdy2 — sts-policy. Execute the below command to observe the content of the configmap

kubectl edit configmap todo-celllatestbdy2sts-policy

Note : todo-celllatestbdy2sts-policy is the name of the configmap.

This configmap consists of all metadata related to a kubernetes map whereas it also contains the actual content / configurations in this configmap. The below set of data depicts the content of this config map. You can write your policy here and save so that this is effective at the time of request execution. If you are familiar with OPA policy language Rego, it’s pretty straightforward to write policies.

Important : Cellery language does not provide syntax for these policies as per 0.4.0 version. This will be available soon in Cellery.

data:
default.rego: |
package cellery.io
default allow = true

Policies must be written in a specific ways in Cellery. If you are familiar with how Istio does, it’s the same way Cellery also follows. You have a specific format of the policy rule name which is a combination of the destination service and few other facts. A bit more complex policy is depicted below. However this doesn’t show how you exactly need to write OPA policies in Cellery system to be answered for the queries raised by Cellery STS. This will be covered in a different article.

Sample OPA policy to allow and deny requests based on source.

OPA request which is sent from STS to OPA authorization API, consist of below information. Policy writers can write policies based on those information.

Sample request towards OPA engine (input request towards OPA)

Conclusion.

Cellery enables service level security from it’s architecture. Having Open Policy Agent adopted makes Cellery a system with comprehensive fine grained access control at service level. Starting from simple decisions like which Cells are allowed to communicate with what Cells, you can write any complex policy using Rego in order to control request flows within Cellery system.

--

--