Security

Service-to-service zero trust

Intent-based access control makes it easy to implement a zero trust back end. Zero trust for service-to-service calls means that a service is only permitted to invoke an operation over the network on another service when:

1. Each service has a trusted identity.
2. Their identities are authenticated.
3. The calling service is authorized by a policy to invoke the desired operation on the service being called.

Zero trust is in a contrast with perimeter security: trusting any service to make any call to any other service it can reach in the network, based on a presumption that the perimeter of the network is absolutely protected. It’s difficult to make the network perimeter sufficiently secure, given:

• The attack vector landscape is vast and highly automated, and includes not just traditional brute-force methods but also supply chain attacks and 0 day vulnerabilities.
• Modern development teams are increasingly distributed, exercise a lot of independence, and often release changes very frequently via automated CI/CD systems. And that means there are many chances of introducing vulnerabilities.
• Most teams rely extensively on 3rd-party, often open-source software within their code and infrastructure. Despite the great improvements in automated vulnerability scanning, shared vulnerability reporting, and patching frequency, at any given time it’s hard to assume there are no chinks in the armor.

But the biggest issue with perimeter security is the age-old saying that defenders need to win all the time while attackers only need to win once. And once a service in the network has been compromised and turned malicious, every other service in that network is now potentially vulnerable.

Zero trust does not replace perimeter security. And service-to-service zero trust does not replace human-to-service zero trust. All of these need to be in place to realize defense in depth, a bedrock principle of security.

But if service-to-service zero trust is difficult — if it requires every developer to be an expert at multiple access control mechanisms, if it relies on manual provisioning of access keys or configuration of ACLs, if it requires teams to be constantly dependent on each other or on central platform teams to get their work done — then this layer of defense in depth remains a pipedream. It must be easy if it’s to be the reality for most teams.

Note that, unlike perimeter security, service-to-service zero trust can be rolled out gradually. For example, certain sensitive services may be protected while still allowing unrestricted access to other services within the network, while still greatly improving the security posture of the back end.

Intent-based access control (IBAC) trust model

When modeling service-to-service communications, we will refer to the service making the call as a client, and the service receiving the call as a server. For asynchronous service communications, such as through message brokers, we will regard the broker itself as the target server: the service that sends the messages calls the broker to publish the message, and the service that receives the message usually calls the broker to receive the message. (In some cases the broker pushes the message to its subscribers, in which case the roles of client and server are reversed.

Intent-based access control (IBAC) aims to:

1. Make it easy for client developers to access accessing secured servers.
2. Minimize the overhead of maintaining permissions for server teams.
3. Shift access-control configuration left, from a run-time concern to a design-time, infrastructure-as-code concern.
4. Restrict access at all times to the least privilege needed to achieve the required functionality.
5. Align code approval mechanisms and access approval mechanisms.
6. Separate concerns between the access needed by development teams and the access granted by platform or server teams, per environment, either automatically or manually.
7. Separate concerns, to the extent possible, between the access needed by development teams and the access control mechanisms implemented by platform or server teams.

To achieve these, IBAC starts from the client’s (caller’s) intents. An intents file for a client service declares the set of calls the client will be making to various server services to achieve its functionality. Intents files are created and maintained by client developers, within the same repositories as the code files of the client. They are updated when the client code changes, to reflect at any time the needs of the code to call other services (servers).

1. Make secure access easy

An IBAC platform relies on the information in the intents file of a client to automatically configure any access controls between the client and the servers it intends to call such that the intended calls are allowed. If service identities need to be established or credentials need to be provisioned and distributed, the IBAC platform should automate that too. The desired client experience should be: just declare your intent to access, without worrying how to achieve authorization, and the platform will “make it so”. This is the first principle of IBAC, and for a very simple reason: security approaches that aren’t adopted aren’t helping security.

2. Minimize server team overhead

If server teams need to maintain information about who is permitted to access their services, that’s overhead that takes away from their productivity. When should an access request be granted? They would need to establish why the client needs it. When should access be revoked? Rarely would they know when the client no longer needs it. In most cases, they should be able to rely on the client team’s development processes and their organization’s oversight to automatically grant access. With IBAC, the default can be to grant any intended access, with rules to except out certain servers or calls or environments and route those to review and approval processes. In this way, server teams can manage by exception and minimize overhead.

3. Shift-left for access control

In leading engineering organizations, particularly cloud-native ones, all the infrastructure, tests, dependencies, resources, and other aspects needed for code to run are declaratively defined at design time. That way, the development life cycle can be automated to be efficient, resilient, and reliable, while minimizing the time to deliver value. IBAC adds access controls to that list of aspects. The specific permissions for what service can call what other service are no longer provisioned after the fact, maintained outside the automated development lifecycle, and potentially allowed to go stale. Instead, just as Docker manifests determine the operating system, virtual machine architecture, code snapshot, and other layers needed for a container to run; and just as Kubernetes manifests determine the pods, network policies, stateful sets, and other resources needed for your runtime plane to do its work; so intents files determine the access needed for your back end ecosystem to achieve its functional goals. The IBAC system will orchestrate the various access control mechanisms to allow only the intended access at run time.

4. Least privilege

One of the pillars of security is least privilege: in our context, a service should be granted the least privilege needed to accomplish its goals. Any more privilege opens it up to abuse, either unintentional (e.g. calls to the wrong environment, unexpected calls that overload the server, etc.) or intentional (a breached service that inappropriately calls other services or allows the attackers to pivots and attack other services).

But how much privilege is the least, without being too little? This boils down to: what is an intended call vs an unintended one? IBAC solves precisely this: it captures the client developer’s intents, at design time. Unintended calls, even if made by the code (inadvertently), will fail since they were not explicitly declared at design-time and hence not explicitly intended. And calls made by a breached service outside of the original declared intent of the developer will similarly fail.

5. Align code and access approvals

The IBAC security model starts from: trust your developers. You trust them to write the code that powers the business, and specifically to code the calls to other services, so trust them to declare those calls in intents files. (It seems pretty obvious when stated this way.) But responsible organizations don’t stop there: they trust and verify. There are review and approval processes for code, there are tests and monitoring, there are automated scans, there is revision control with history and rollback, and there is ownership of changes with accountability. All of these can and should incorporate intents files. For example, code reviewers should ensure that the declared intents correspond to the calls made in the code, and that changes to the code have corresponding intents file changes.

Once intents files have been approved, along with the approved code that realizes them, it’s known that this approved code won’t work without granting access exactly as stated in the intents files. Access may then be automatically approved accordingly, or further rules may be imposed — see the next section.

6. Separate concerns: access needed and access granted

Once the code and its intents declarations are approved, then automation can take over. The access that needs to be granted to make the code work as intended is now known. The IBAC platform should be configured to overlay the organization’s trust model:

• One example might be to auto-approve (turn intents into actual access) all intents files in dev environments but require secops approvals for any new intents before rolling out to staging, and generate follow-up secops tickets for new intents in prod.
• Another example might be to auto-approve all intents other than specific called-out services which require those service teams to approve.
• Yet a third example would be to auto-approve all but mutating calls (writes, vs reads) to certain sensitive services; the latter would require an additional label to be added by an authorized party to the calling service, such as Finance approved or License verified.

These examples illustrate the balance that may be needed between trusting that code that has been developed to make certain calls should be given the access to complete the calls, and incorporating other compliance and security needs of the business.

7. Separate concerns: access and access controls

IBAC allows strong separation between the definition of the access required by a client and the implementation of access controls in front of the servers. To a large extent, the details of the access control mechanisms and configurations need not be known at design time, in particular to the client developer. There are many advantages that follow:

• Developers can focus their efforts and expertise on functional needs, without needing all the expertise and overhead to also configure access controls.
• Secops engineers can require that access be done securely, and ensure that access controls are implemented correctly and stringently, knowing they are not slowing down the development process when they do so.
• Platform engineers can use the IBAC platform to automate the realization of the intended and approved access within the enforcement systems (network policies, ACLs, gateways, etc.), to meet the requirements set by the secops engineers and the security and compliance team.

Depending on the access control mechanisms, it may not be possible to completely separate the client and server developers from the control mechanisms. For example, in some cases the client must present credentials explicitly in calling the server, and the server might need to explicitly verify them (if using an inline

Otterize OSS security

Otterize OSS components — the intents operator, credentials operator, and network mapper — can only be deployed by cluster admins. Cluster admins can also set the scope in which these components operate, such as the namespaces they watch for intents, or the Kafka topics they will affect (which also requires Kafka admin privileges).

Users of the cluster, i.e. developers, then use pod annotations and custom resources (the Otterize client intents) to instruct the operators to grant them access or credentials. Cluster admins can use Kubernetes RBAC to control who can create and update client intents.

Cluster admins can use Kubernetes RBAC to control which Kubernetes users can apply intents.

Service identity trust model

Services in Otterize are identified through their resolved or declared service name, coupled with the namespace they’re running in. A service in namespace A has a different identity to a service in namespace B — in network policies as well as cryptographically.

Services can then gain access to connect to other services through declared intents that create network policies or Kafka ACLs, as well as request credentials for the instance of the service running in this namespace.

The service names in each namespace are set by the developers, or the platform team - whoever can set the name of the resource that deploys the pods, or set annotations on the pods. It is the cluster admin's responsibility to determine who can set those. In most cases, we expect that restricting access by namespace is sufficient, as service identities are scoped by namespace. Learn more about how service identity resolution works.

Cryptographic credentials

The cryptographic credentials created by the Otterize credentials operator (mTLS certificate and key pairs) are provided in one of three ways, depending on how the operator is configured:

1. By a cert-manager deployment in the same cluster.
2. By the Otterize Cloud managed credentials service. That service is built on a Hashicorp Vault instance with a CA automatically created for you. Using Otterize Cloud for credentials means you do not need to deploy SPIRE on your cluster, which makes for a simpler and lightweight deployment.
3. By a SPIRE server that is deployed alongside the credentials operator.

Note that, by default, the first option (in-cluster SPIRE) is used, even when the credentials operator is connected to Otterize Cloud). The certificateProvider could be set to otterize-cloud or cert-manager to use the other options; see the Helm chart configuration for the credentials operator for more details.

The credentials operator watches for pods starting up in a Kubernetes cluster, and if mTLS credentials are requested, it uses the resolved or declared service name plus the pod’s namespace to generate credentials for that service name, in that namespace. The operator is conceptually similar to a SPIRE agent in that it attests to the identity of pods.

Learn more about how SPIRE works.